[jira] Created: (SOLR-1752) SolrJ fails with exception when passing document ADD and DELETEs in the same request using XML request writer (but not binary request writer)
SolrJ fails with exception when passing document ADD and DELETEs in the same
request using XML request writer (but not binary request writer)
-
Key: SOLR-1752
URL: https://issues.apache.org/jira/browse/SOLR-1752
Project: Solr
Issue Type: Bug
Components: clients - java, update
Affects Versions: 1.4
Reporter: Jayson Minard
Priority: Blocker
Add this test to SolrExampleTests.java and it will fail when using the XML
Request Writer (now default), but not if you change the SolrExampleJettyTest to
use the BinaryRequestWriter.
{code}
public void testAddDeleteInSameRequest() throws Exception {
SolrServer server = getSolrServer();
SolrInputDocument doc3 = new SolrInputDocument();
doc3.addField( "id", "id3", 1.0f );
doc3.addField( "name", "doc3", 1.0f );
doc3.addField( "price", 10 );
UpdateRequest up = new UpdateRequest();
up.add( doc3 );
up.deleteById("id001");
up.setWaitFlush(false);
up.setWaitSearcher(false);
up.process( server );
}
{code}
terminates with exception:
{code}
Feb 3, 2010 8:55:34 AM org.apache.solr.common.SolrException log
SEVERE: org.apache.solr.common.SolrException: Illegal to have multiple roots
(start tag in epilog?).
at [row,col {unknown-source}]: [1,125]
at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:72)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:54)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
at org.mortbay.jetty.Server.handle(Server.java:285)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
at
org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:835)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:723)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:202)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
at
org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
Caused by: com.ctc.wstx.exc.WstxParsingException: Illegal to have multiple
roots (start tag in epilog?).
at [row,col {unknown-source}]: [1,125]
at
com.ctc.wstx.sr.StreamScanner.constructWfcException(StreamScanner.java:630)
at com.ctc.wstx.sr.StreamScanner.throwParseError(StreamScanner.java:461)
at
com.ctc.wstx.sr.BasicStreamReader.handleExtraRoot(BasicStreamReader.java:2155)
at
com.ctc.wstx.sr.BasicStreamReader.nextFromProlog(BasicStreamReader.java:2070)
at
com.ctc.wstx.sr.BasicStreamReader.nextFromTree(BasicStreamReader.java:2647)
at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1019)
at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:90)
at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:69)
... 18 more
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ UpdateRequest does not maintain order of operations when sending mixed types of changes (updates, delete id, delete query, update iterator)
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12776829#action_12776829 ] Jayson Minard commented on SOLR-1162: - Yikes, set to 1.5 -- I better revamp this to current code. > SolrJ UpdateRequest does not maintain order of operations when sending mixed > types of changes (updates, delete id, delete query, update iterator) > - > > Key: SOLR-1162 > URL: https://issues.apache.org/jira/browse/SOLR-1162 > Project: Solr > Issue Type: Improvement > Components: clients - java >Affects Versions: 1.3 >Reporter: Jayson Minard > Fix For: 1.5 > > Attachments: Solr-1162.patch, Solr-1162.patch, Solr-1162.patch > > > In SolrJ UpdateRequest object it maintains separate lists of documents to > add, delete, and delete queries so that the order of those operations is not > known to the caller. It really should execute the items in the same order > they were added to the UpdateRequest. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: SOLR-1155-release1.4-rev834789.patch Added patch for release version of 1.4 (but this is most likely identical to the last trunk patch anyway) > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3, 1.4 >Reporter: Jayson Minard > Attachments: SOLR-1155-release1.4-rev834789.patch, > SOLR-1155-trunk-rev834706.patch, Solr-1155.patch, Solr-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Affects Version/s: 1.4 > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3, 1.4 >Reporter: Jayson Minard > Attachments: SOLR-1155-trunk-rev834706.patch, Solr-1155.patch, > Solr-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: SOLR-1155-trunk-rev834706.patch Updated patch to trunk rev 834706 and up to date with current DirectUpdateHandler2 functionality > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3, 1.4 >Reporter: Jayson Minard > Attachments: SOLR-1155-trunk-rev834706.patch, Solr-1155.patch, > Solr-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1299) In distributed search cannot search on any schema field in ascending order (descending is OK)
[ https://issues.apache.org/jira/browse/SOLR-1299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12734681#action_12734681 ] Jayson Minard commented on SOLR-1299: - Shalin, Are you picking this one up, or should we try to dig into it? Not an area I'm familiar with but can start tracing through... > In distributed search cannot search on any schema field in ascending order > (descending is OK) > - > > Key: SOLR-1299 > URL: https://issues.apache.org/jira/browse/SOLR-1299 > Project: Solr > Issue Type: Bug > Components: search >Affects Versions: 1.4 >Reporter: David Bowen > Fix For: 1.4 > > > Using the example with some of the exampledocs posted, the url: > http://localhost:8983/solr/select/?q=*:*&sort=timestamp+desc&fsv=true > works correctly, but change "desc" to "asc" and you get: > HTTP ERROR: 500 > null > java.lang.NullPointerException > at > org.apache.solr.handler.component.QueryComponent.getComparator(QueryComponent.java:284) > at > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:229) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195) > at > com.proquest.magnolia.solr.plugins.SummonSearchHandler.handleRequestBody(SummonSearchHandler.java:19) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1299) > at > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089) > at > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365) > at > org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) > at > org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) > at > org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712) > at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405) > at > org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211) > at > org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) > at > org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139) > at org.mortbay.jetty.Server.handle(Server.java:285) > at > org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502) > at > org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821) > at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513) > at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208) > at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378) > at > org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226) > at > org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1164) BinaryUpdateRequestHandler and JavaBinUpdateRequestCodec do not maintain order of the commands as serialized in the binary format
[ https://issues.apache.org/jira/browse/SOLR-1164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712128#action_12712128 ] Jayson Minard commented on SOLR-1164: - The UpdateRequest has get methods for getting the individual lists back that can be deprecated but harder to implement their replacement with everything lumped together. So could go either way on that one. Will look at options there when I get back to updating that patch soon. > BinaryUpdateRequestHandler and JavaBinUpdateRequestCodec do not maintain > order of the commands as serialized in the binary format > - > > Key: SOLR-1164 > URL: https://issues.apache.org/jira/browse/SOLR-1164 > Project: Solr > Issue Type: Improvement > Components: update >Affects Versions: 1.3 >Reporter: Jayson Minard >Assignee: Noble Paul > Attachments: SOLR-1164.patch, SOLR-1164.patch > > > When sending commands in the Java binary format to the > BinaryUpdateRequestHandler it does not process them in the order received. > It processes all add/updates then delete by id then delete by query > regardless of what is sent. See SOLR-1162 for related issue and patch that > covers both issues (they are intertwined since some classes are shared on > both sides of the wire) > I wanted a separate issue covering this so that it is seen from the server > viewpoint and not just as a client API issue as other clients writing the > binary form would be unable to maintain order of commands as well. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1164) BinaryUpdateRequestHandler and JavaBinUpdateRequestCodec do not maintain order of the commands as serialized in the binary format
[ https://issues.apache.org/jira/browse/SOLR-1164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711318#action_12711318 ] Jayson Minard commented on SOLR-1164: - So I need to take this patch, then fix SOLR-1162 over the top of it, correct? I'll see what changed across the two and make sure SOLR-1162 depends on this patch and is only the changes beyond it. > BinaryUpdateRequestHandler and JavaBinUpdateRequestCodec do not maintain > order of the commands as serialized in the binary format > - > > Key: SOLR-1164 > URL: https://issues.apache.org/jira/browse/SOLR-1164 > Project: Solr > Issue Type: Improvement > Components: update >Affects Versions: 1.3 >Reporter: Jayson Minard >Assignee: Noble Paul > Attachments: SOLR-1164.patch, SOLR-1164.patch > > > When sending commands in the Java binary format to the > BinaryUpdateRequestHandler it does not process them in the order received. > It processes all add/updates then delete by id then delete by query > regardless of what is sent. See SOLR-1162 for related issue and patch that > covers both issues (they are intertwined since some classes are shared on > both sides of the wire) > I wanted a separate issue covering this so that it is seen from the server > viewpoint and not just as a client API issue as other clients writing the > binary form would be unable to maintain order of commands as well. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ UpdateRequest does not maintain order of operations when sending mixed types of changes (updates, delete id, delete query, update iterator)
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12709503#action_12709503 ] Jayson Minard commented on SOLR-1162: - Sounds good. I'll watch SOLR-1164 for updates. > SolrJ UpdateRequest does not maintain order of operations when sending mixed > types of changes (updates, delete id, delete query, update iterator) > - > > Key: SOLR-1162 > URL: https://issues.apache.org/jira/browse/SOLR-1162 > Project: Solr > Issue Type: Improvement > Components: clients - java >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1162.patch, Solr-1162.patch, Solr-1162.patch > > > In SolrJ UpdateRequest object it maintains separate lists of documents to > add, delete, and delete queries so that the order of those operations is not > known to the caller. It really should execute the items in the same order > they were added to the UpdateRequest. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1162) SolrJ UpdateRequest does not maintain order of operations when sending mixed types of changes (updates, delete id, delete query, update iterator)
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1162: Attachment: Solr-1162.patch Updated patch to remove return value from the unmarshall call as suggested above. > SolrJ UpdateRequest does not maintain order of operations when sending mixed > types of changes (updates, delete id, delete query, update iterator) > - > > Key: SOLR-1162 > URL: https://issues.apache.org/jira/browse/SOLR-1162 > Project: Solr > Issue Type: Improvement > Components: clients - java >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1162.patch, Solr-1162.patch, Solr-1162.patch > > > In SolrJ UpdateRequest object it maintains separate lists of documents to > add, delete, and delete queries so that the order of those operations is not > known to the caller. It really should execute the items in the same order > they were added to the UpdateRequest. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ UpdateRequest does not maintain order of operations when sending mixed types of changes (updates, delete id, delete query, update iterator)
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12709446#action_12709446 ] Jayson Minard commented on SOLR-1162: - Sounds good, didn't like that returned object in the old method anyway... I'll update the unmarshal and unit test that used that object to ensure it was accurate as well. What do you want to do about the deserialization? I think it was unsafe before in that it looked up the lists by names and made assumptions about the parms being first. > SolrJ UpdateRequest does not maintain order of operations when sending mixed > types of changes (updates, delete id, delete query, update iterator) > - > > Key: SOLR-1162 > URL: https://issues.apache.org/jira/browse/SOLR-1162 > Project: Solr > Issue Type: Improvement > Components: clients - java >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1162.patch, Solr-1162.patch > > > In SolrJ UpdateRequest object it maintains separate lists of documents to > add, delete, and delete queries so that the order of those operations is not > known to the caller. It really should execute the items in the same order > they were added to the UpdateRequest. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ UpdateRequest does not maintain order of operations when sending mixed types of changes (updates, delete id, delete query, update iterator)
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12709390#action_12709390 ] Jayson Minard commented on SOLR-1162: - UpdateRequest is used on both ends of the stream potentially so we would want to break out the server pieces to not use the same un-ordered object for binary codec. I could go either way, just never saw a need for unordered to remain but then again there is a whole community out there that might! > SolrJ UpdateRequest does not maintain order of operations when sending mixed > types of changes (updates, delete id, delete query, update iterator) > - > > Key: SOLR-1162 > URL: https://issues.apache.org/jira/browse/SOLR-1162 > Project: Solr > Issue Type: Improvement > Components: clients - java >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1162.patch, Solr-1162.patch > > > In SolrJ UpdateRequest object it maintains separate lists of documents to > add, delete, and delete queries so that the order of those operations is not > known to the caller. It really should execute the items in the same order > they were added to the UpdateRequest. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1164) BinaryUpdateRequestHandler and JavaBinUpdateRequestCodec do not maintain order of the commands as serialized in the binary format
[ https://issues.apache.org/jira/browse/SOLR-1164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12709050#action_12709050 ] Jayson Minard commented on SOLR-1164: - Noticed that SolrJ classes are used in the server to implement the handler side which makes it hard to split the patch. If we want to clean that up, the patch can be split but otherwise leaving it in SOLR-1162 and tracking both sides of the coin by leaving this issue here. > BinaryUpdateRequestHandler and JavaBinUpdateRequestCodec do not maintain > order of the commands as serialized in the binary format > - > > Key: SOLR-1164 > URL: https://issues.apache.org/jira/browse/SOLR-1164 > Project: Solr > Issue Type: Improvement > Components: update >Affects Versions: 1.3 >Reporter: Jayson Minard > > When sending commands in the Java binary format to the > BinaryUpdateRequestHandler it does not process them in the order received. > It processes all add/updates then delete by id then delete by query > regardless of what is sent. See SOLR-1162 for related issue and patch that > covers both issues (they are intertwined since some classes are shared on > both sides of the wire) > I wanted a separate issue covering this so that it is seen from the server > viewpoint and not just as a client API issue as other clients writing the > binary form would be unable to maintain order of commands as well. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (SOLR-1164) BinaryUpdateRequestHandler and JavaBinUpdateRequestCodec do not maintain order of the commands as serialized in the binary format
BinaryUpdateRequestHandler and JavaBinUpdateRequestCodec do not maintain order of the commands as serialized in the binary format - Key: SOLR-1164 URL: https://issues.apache.org/jira/browse/SOLR-1164 Project: Solr Issue Type: Improvement Components: update Affects Versions: 1.3 Reporter: Jayson Minard When sending commands in the Java binary format to the BinaryUpdateRequestHandler it does not process them in the order received. It processes all add/updates then delete by id then delete by query regardless of what is sent. See SOLR-1162 for related issue and patch that covers both issues (they are intertwined since some classes are shared on both sides of the wire) I wanted a separate issue covering this so that it is seen from the server viewpoint and not just as a client API issue as other clients writing the binary form would be unable to maintain order of commands as well. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1162) SolrJ UpdateRequest does not maintain order of operations when sending mixed types of changes (updates, delete id, delete query, update iterator)
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1162: Description: In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest. was:In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest. Summary: SolrJ UpdateRequest does not maintain order of operations when sending mixed types of changes (updates, delete id, delete query, update iterator) (was: SolrJ does not maintain order of operations when using an UpdateRequest object to send them in bulk) > SolrJ UpdateRequest does not maintain order of operations when sending mixed > types of changes (updates, delete id, delete query, update iterator) > - > > Key: SOLR-1162 > URL: https://issues.apache.org/jira/browse/SOLR-1162 > Project: Solr > Issue Type: Improvement > Components: clients - java >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1162.patch, Solr-1162.patch > > > In SolrJ UpdateRequest object it maintains separate lists of documents to > add, delete, and delete queries so that the order of those operations is not > known to the caller. It really should execute the items in the same order > they were added to the UpdateRequest. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ does not maintain order of operations when using an UpdateRequest object to send them in bulk
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12709037#action_12709037 ] Jayson Minard commented on SOLR-1162: - Outside of UpdateRequest if you hand construct a binary codec stream to send to Solr (without this patch) your order of actions would not be maintained within the stream. So the binary streaming update handler is "broken" in this regard as well. So this patch actually resolves two issue: UpdateRequest does not serialize into xml or binary stream the actions in-order. Nor does BinaryUpdateRequestHandler nor did the underlaying JavaBinUpdateRequestCodec so these all become unusable for mixed style actions since none can maintain order. UpdateRequest is just one of the clients to that server-side code that also has the same issue. > SolrJ does not maintain order of operations when using an UpdateRequest > object to send them in bulk > --- > > Key: SOLR-1162 > URL: https://issues.apache.org/jira/browse/SOLR-1162 > Project: Solr > Issue Type: Improvement > Components: clients - java >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1162.patch, Solr-1162.patch > > > In SolrJ UpdateRequest object it maintains separate lists of documents to > add, delete, and delete queries so that the order of those operations is not > known to the caller. It really should execute the items in the same order > they were added to the UpdateRequest. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: Solr-1155.patch Resolve TODO for commitWithin, and updated AutoCommitTrackerTest to validate the fix. > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1155.patch, Solr-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ does not maintain order of operations when using an UpdateRequest object to send them in bulk
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708930#action_12708930 ] Jayson Minard commented on SOLR-1162: - Multiple requests are less efficient than sending large batches together. To be the most efficient with large requests, every user of SolrJ UpdateRequest would need to write the same logic... Place adds into UpdateRequest until you hit the first non-add, then send the UpdateRequest and start writing your deletes until you hit a non-delete, then flush the UpdateRequest and keep adding your new transaction type until you hit the first ... In that case they should avoid using UpdateRequest altogether as calling the SolrServer directly is just as "easy." If we are going to batch on their behalf why wouldn't we do it correctly and be predictable with our ordering. I'm sure if JDBC batches did not maintain order, there would be havoc to pay... Besides that, it isn't clear to users of UpdateRequest as to the order of operations, so someone doing an Add doc 1, Delete doc 1, Add doc 1 may not end up with the expected outcome. It turns into Add doc 1, Add doc 1, Delete doc1 when streaming and similary for XML version of the transaction. If I did a Delete Query *:* then Add doc1, Add doc 2 I end up with no docs as the delete query comes last, but I (the user) does not know that. I've written code to work around UpdateRequest ordering and I usually end up only using it for commitWithin or having a commit tacked on the end of the request due to the above issues. > SolrJ does not maintain order of operations when using an UpdateRequest > object to send them in bulk > --- > > Key: SOLR-1162 > URL: https://issues.apache.org/jira/browse/SOLR-1162 > Project: Solr > Issue Type: Improvement > Components: clients - java >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1162.patch, Solr-1162.patch > > > In SolrJ UpdateRequest object it maintains separate lists of documents to > add, delete, and delete queries so that the order of those operations is not > known to the caller. It really should execute the items in the same order > they were added to the UpdateRequest. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708795#action_12708795 ] Jayson Minard commented on SOLR-1155: - TODO: fix commitWithin, it isn't respected by the DirectUpdateHandler3 > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1162) SolrJ does not maintain order of operations when using an UpdateRequest object to send them in bulk
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1162: Attachment: Solr-1162.patch Updated patch to fix commitWithin parameter in UpdateRequest and failing unit test. > SolrJ does not maintain order of operations when using an UpdateRequest > object to send them in bulk > --- > > Key: SOLR-1162 > URL: https://issues.apache.org/jira/browse/SOLR-1162 > Project: Solr > Issue Type: Improvement > Components: clients - java >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1162.patch, Solr-1162.patch > > > In SolrJ UpdateRequest object it maintains separate lists of documents to > add, delete, and delete queries so that the order of those operations is not > known to the caller. It really should execute the items in the same order > they were added to the UpdateRequest. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ does not maintain order of operations when using an UpdateRequest object to send them in bulk
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708792#action_12708792 ] Jayson Minard commented on SOLR-1162: - Also (for Noble Paul)... UpdateRequest is the invoker of itself via the process method, there is nothing to which you can pass it as a list. UpdateRequest req = new UpdateRequest(); ... add things req.process(solrServer); > SolrJ does not maintain order of operations when using an UpdateRequest > object to send them in bulk > --- > > Key: SOLR-1162 > URL: https://issues.apache.org/jira/browse/SOLR-1162 > Project: Solr > Issue Type: Improvement > Components: clients - java >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1162.patch > > > In SolrJ UpdateRequest object it maintains separate lists of documents to > add, delete, and delete queries so that the order of those operations is not > known to the caller. It really should execute the items in the same order > they were added to the UpdateRequest. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1162) SolrJ does not maintain order of operations when using an UpdateRequest object to send them in bulk
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1162: Issue Type: Improvement (was: Bug) > SolrJ does not maintain order of operations when using an UpdateRequest > object to send them in bulk > --- > > Key: SOLR-1162 > URL: https://issues.apache.org/jira/browse/SOLR-1162 > Project: Solr > Issue Type: Improvement > Components: clients - java >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1162.patch > > > In SolrJ UpdateRequest object it maintains separate lists of documents to > add, delete, and delete queries so that the order of those operations is not > known to the caller. It really should execute the items in the same order > they were added to the UpdateRequest. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ does not maintain order of operations when using an UpdateRequest object to send them in bulk
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708790#action_12708790 ] Jayson Minard commented on SOLR-1162: - Current patch (12-May-2009 11:30PM) is not tested with live server, only against the unit test TestUpdateRequestCodec. > SolrJ does not maintain order of operations when using an UpdateRequest > object to send them in bulk > --- > > Key: SOLR-1162 > URL: https://issues.apache.org/jira/browse/SOLR-1162 > Project: Solr > Issue Type: Bug > Components: clients - java >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1162.patch > > > In SolrJ UpdateRequest object it maintains separate lists of documents to > add, delete, and delete queries so that the order of those operations is not > known to the caller. It really should execute the items in the same order > they were added to the UpdateRequest. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ does not maintain order of operations when using an UpdateRequest object to send them in bulk
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708789#action_12708789 ] Jayson Minard commented on SOLR-1162: - In response to Noble Paul: An UpdateRequest currently contains: a list of adds, a single add via iterator, a list of delete by ids, a list of delete queries. So you can pile them all into the same object currently and then have no idea what order they will actually execute. If you want to stream adds, you cannot intermix deletes or other actions in the same stream. So if UpdateRequest is going to allow a set of different actions it should at least maintain the order in which they were added and execute them similarly. UpdateRequest is the current batching model, so it should be correct. > SolrJ does not maintain order of operations when using an UpdateRequest > object to send them in bulk > --- > > Key: SOLR-1162 > URL: https://issues.apache.org/jira/browse/SOLR-1162 > Project: Solr > Issue Type: Bug > Components: clients - java >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1162.patch > > > In SolrJ UpdateRequest object it maintains separate lists of documents to > add, delete, and delete queries so that the order of those operations is not > known to the caller. It really should execute the items in the same order > they were added to the UpdateRequest. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: (was: SOLR-1155.patch) > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: (was: SOLR-1155.patch) > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: (was: SOLR-1155.patch) > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: (was: SOLR-1155.patch) > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1155.patch, SOLR-1155.patch, SOLR-1155.patch, > SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: (was: SOLR-1155.patch) > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1155.patch, SOLR-1155.patch, SOLR-1155.patch, > SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: (was: SOLR-1155.patch) > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1155.patch, SOLR-1155.patch, SOLR-1155.patch, > SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: (was: SOLR-1155.patch) > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1155.patch, SOLR-1155.patch, SOLR-1155.patch, > SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: (was: SOLR-1155.patch) > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1155.patch, SOLR-1155.patch, SOLR-1155.patch, > SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: (was: SOLR-1155.patch) > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1155.patch, SOLR-1155.patch, SOLR-1155.patch, > SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: Solr-1155.patch Cleaning the patch so that RequestHandlerBase only has the required change and not the accidental reformatting. > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1155.patch, SOLR-1155.patch, SOLR-1155.patch, > SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: (was: SOLR-1155.patch) > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1155.patch, SOLR-1155.patch, SOLR-1155.patch, > SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: (was: SOLR-1155.patch) > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1155.patch, SOLR-1155.patch, SOLR-1155.patch, > SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1162) SolrJ does not maintain order of operations when using an UpdateRequest object to send them in bulk
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1162: Attachment: Solr-1162.patch First round of patch, changed the UpdateRequest to maintain one list of different types of request items, then changed XML serialization to keep the same order, and the same for binary codec. Then updated the streaming side of the binary update handler to stream all request types (not just doc adds) and updated the unit test to verify order is maintained both when streaming or not streaming using the binary codec. > SolrJ does not maintain order of operations when using an UpdateRequest > object to send them in bulk > --- > > Key: SOLR-1162 > URL: https://issues.apache.org/jira/browse/SOLR-1162 > Project: Solr > Issue Type: Bug > Components: clients - java >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: Solr-1162.patch > > > In SolrJ UpdateRequest object it maintains separate lists of documents to > add, delete, and delete queries so that the order of those operations is not > known to the caller. It really should execute the items in the same order > they were added to the UpdateRequest. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ does not maintain order of operations when using an UpdateRequest object to send them in bulk
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708655#action_12708655 ] Jayson Minard commented on SOLR-1162: - Working on this patch now... > SolrJ does not maintain order of operations when using an UpdateRequest > object to send them in bulk > --- > > Key: SOLR-1162 > URL: https://issues.apache.org/jira/browse/SOLR-1162 > Project: Solr > Issue Type: Bug > Components: clients - java >Affects Versions: 1.3 >Reporter: Jayson Minard > > In SolrJ UpdateRequest object it maintains separate lists of documents to > add, delete, and delete queries so that the order of those operations is not > known to the caller. It really should execute the items in the same order > they were added to the UpdateRequest. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (SOLR-1162) SolrJ does not maintain order of operations when using an UpdateRequest object to send them in bulk
SolrJ does not maintain order of operations when using an UpdateRequest object to send them in bulk --- Key: SOLR-1162 URL: https://issues.apache.org/jira/browse/SOLR-1162 Project: Solr Issue Type: Bug Components: clients - java Affects Versions: 1.3 Reporter: Jayson Minard In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: SOLR-1155.patch Updated unit test to compile (was missing method change from previous patch that changed an interface it implemented) > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, > SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, > SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: SOLR-1155.patch Changed external commit interface to not have locking, but rather query if a commit/rollback/optimize is already running to accomplish the same goal. Also ensure no locking around waiting for searcher after commit or optimize. > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, > SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, > SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: SOLR-1155.patch Change openWriter to do a check of null writer before locking to avoid artificial contention between threads due to ensuring a writer is open. > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, > SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, > SOLR-1155.patch, SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: SOLR-1155.patch Bug fixing, and added a new test that test just the AutoCommitTracker in isolation rather than through the DirectUpdateHandlerX > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, > SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, > SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: SOLR-1155.patch Knocked down a few TODO and cleaned up resetting of counts. > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, > SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: SOLR-1155.patch minor change to rollback to only drop and re-open writer if there was one to begin wtih. > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, > SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: SOLR-1155.patch Refactored the AutoCommitTracker out to separate top-level class rather than inner class. Enhanced the interface used by unit tests and other classes to talk to the update handler so make this work. Interface is not final, just a temp answer to the decoupling. > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, > SOLR-1155.patch, SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: SOLR-1155.patch move wait for searcher out of locks in optimize and commit > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, > SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707701#action_12707701 ] Jayson Minard commented on SOLR-1155: - Does anyone know if the UpdateHandler.close() method is called when everything else is shut down and no transactions can come through? If so, I can remove the blocking added to prevent a writer from re-opening after close is called. > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: SOLR-1155.patch New patch resolves the serialization of commit/optimize/close/rollback actions that should not be concurrent with each other. Also protect methods after close to keep other threads from accidentally re-opening writer. > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707692#action_12707692 ] Jayson Minard commented on SOLR-1155: - new TODO... Right now auto commit will eventually fire off a commit that might run into a manual commit or optimize and sit at the syncrhonization block, then fire off on the tail of the other action. I would rather it wait until those running commands clear and THEN evaluate if it needs to do a commit or not since its information it used to make the decision could be moot due to the other action that beat it in. So if a commit or optimize is running it will not evaluate for auto commit until they complete. > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: SOLR-1155.patch, SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: (was: DirectUpdateHandler3.java) > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: SOLR-1155.patch, SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: (was: DirectUpdateHandlerWithAutoCommit.java) > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: SOLR-1155.patch, SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: (was: DirectUpdateHandler3.java) > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: SOLR-1155.patch, SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: (was: DirectUpdateHandler3.java) > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: SOLR-1155.patch, SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: SOLR-1155.patch Fixed some of the issue mentioned in the comment above. Split commit and optimize work out since optimize has a stronger need to lock and be consistent. > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: SOLR-1155.patch, SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707672#action_12707672 ] Jayson Minard commented on SOLR-1155: - Things likely needing further work: * serialize commits? (two shouldn't happen at same time) * optimize should us iwManageWriter lock from optimize through call-back call, so break it away from commit which only needs iwUseWriter?) This also takes care of the contract that optimize callback is still on an optimized index. > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: DirectUpdateHandler3.java, DirectUpdateHandler3.java, > DirectUpdateHandler3.java, DirectUpdateHandlerWithAutoCommit.java, > SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707636#action_12707636 ] Jayson Minard commented on SOLR-1155: - All tests pass with the attached patch file and DirectUpdateHandler3 as the handler for each solrconfig variant in the test-files directory. Needs review for locking holes and some of the TODO comments answered still. And needs a lot of concurrent update testing. > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: DirectUpdateHandler3.java, DirectUpdateHandler3.java, > DirectUpdateHandler3.java, DirectUpdateHandlerWithAutoCommit.java, > SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: SOLR-1155.patch DirectUpdateHandlerWithAutoCommit.java DirectUpdateHandler3.java Updated file, and associated interface used to help DirectUpdateHandler2 and 3 co-exist with tests, SnapPuller and ReplicationHandler classes. A larger patch with all changes to tests, config files, and other items makeing DirectUpdateHandler3 the default (purely for running all of the unit tests using it) is attached as well. Patch includes fix for SOLR-1157 which got in the way of running unit tests. > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: DirectUpdateHandler3.java, DirectUpdateHandler3.java, > DirectUpdateHandler3.java, DirectUpdateHandlerWithAutoCommit.java, > SOLR-1155.patch > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (SOLR-1157) Change for timeouts in RequestHandlerBase (rev674249) broke unit tests due to rsp.getResponse() returning null
Change for timeouts in RequestHandlerBase (rev674249) broke unit tests due to
rsp.getResponse() returning null
--
Key: SOLR-1157
URL: https://issues.apache.org/jira/browse/SOLR-1157
Project: Solr
Issue Type: Bug
Components: search
Affects Versions: 1.4
Reporter: Jayson Minard
Priority: Critical
This code:
boolean timedOut = (Boolean) rsp.getResponseHeader().get("partialResults") ==
null ? false : (Boolean) rsp.getResponseHeader().get("partialResults");
if (timedOut) {
numTimeouts++;
rsp.setHttpCaching(false);
}
in RequestHandlerBase around line 128 breaks unit tests. Needs a null check on
getResponseHeader() before going further.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: DirectUpdateHandler3.java Updated with different locking around the post commit/optimize handlers. There is a gap in time between commit and these calls, is that safe? left TODO's in the code at places of concern. > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: DirectUpdateHandler3.java, DirectUpdateHandler3.java > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayson Minard updated SOLR-1155: Attachment: DirectUpdateHandler3.java I created DirectUpdateHandler3 as a copy of DirectUpdateHandler2 and then did a pass across it rebuilding the locking, stats, and tracker. I am basically at the "IDE show no errors" stage and going to read it a few more times, and start testing. Feedback is welcome as I'm in the dark on a few areas of this code and I am working from a lot of assumptions. Some event placeholder methods remain while I think through the issues, will trim it back when done. Events to the auto commit tracker are within the locks to help avoid cases where we set the counts to 0 after other work has concurrently been done. The stats information has similar concerns and is not yet dealt with, so some stats might reset to 0 on commit when there was further work being done (so they should not go to 0). > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > Attachments: DirectUpdateHandler3.java > > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707602#action_12707602 ] Jayson Minard commented on SOLR-1155: - Preparing a first patch, calling it DirectUpdateHandler3 as there is a lot of reshuffling and a new commit tracker implementation that is lighter-weight. > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707600#action_12707600 ] Jayson Minard commented on SOLR-1155: - Thinking about pending counts as well. If adds are ongoing during a commit, there has to be a point in time where the counters clear the number of documents committed. Currently they are set to 0 which is safe since nothing was ongoing while the commit happened. Something to remember... > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707572#action_12707572 ] Jayson Minard edited comment on SOLR-1155 at 5/8/09 7:52 PM: - This will involve a change to locking as seen in the thread mentioned above between add/delete/commit methods, but there are also synchronized methods on the CommitTracker scehduleCommitWithin method, run method, and getCommitCount method that will cause similar blocking at least for deletes. was (Author: jminard): This will involve a change to locking as seen in the thread mentioned above between add/delete/commit methods, but there are also synchronized methods on the CommitTracker scehduleCommitWithin method, run method, and getCommitCount method that will cause similar blocking. > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707575#action_12707575 ] Jayson Minard commented on SOLR-1155: - Yonik mentioned: "We'd need to think about what type of synchronization may be needed for postCommit and postOptimize hooks too." Is the expectation for those callbacks that nothing will change until the callback completes? (i.e. a script might run a process that scans the index after optimize expecting it to remain optimized during the script execution) Or is that not the contract of those callbacks? > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707572#action_12707572 ] Jayson Minard commented on SOLR-1155: - This will involve a change to locking as seen in the thread mentioned above between add/delete/commit methods, but there are also synchronized methods on the CommitTracker scehduleCommitWithin method, run method, and getCommitCount method that will cause similar blocking. > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit > - > > Key: SOLR-1155 > URL: https://issues.apache.org/jira/browse/SOLR-1155 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.3 >Reporter: Jayson Minard > > Currently DirectUpdateHandler2 will block adds during a commit, and it seems > to be possible with recent changes to Lucene to allow them to run > concurrently. > See: > http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
Change DirectUpdateHandler2 to allow concurrent adds during an autocommit - Key: SOLR-1155 URL: https://issues.apache.org/jira/browse/SOLR-1155 Project: Solr Issue Type: Improvement Components: search Affects Versions: 1.3 Reporter: Jayson Minard Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently. See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1111) fix FieldCache usage in Solr
[ https://issues.apache.org/jira/browse/SOLR-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12705414#action_12705414 ] Jayson Minard commented on SOLR-: - Is this Lucene version in the current 1.4 trunk, or is it a version not-yet integrated into Solr libs? And also, the description makes it sound like an upgrade issue, but really any 1.4 version could blow up due to this problem. Lastly, define "blow up"... Uses double memory, or some other side effect? > fix FieldCache usage in Solr > > > Key: SOLR- > URL: https://issues.apache.org/jira/browse/SOLR- > Project: Solr > Issue Type: Bug >Reporter: Yonik Seeley > Fix For: 1.4 > > > Recent changes in Lucene have altered how the FieldCache is used and as-is > could lead to previously working Solr installations blowing up when they > upgrade to 1.4. We need to fix, or document the affects of these changes. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1030) Facet counts are not correct (or total document count is not correct as they do not match) on some searches
[
https://issues.apache.org/jira/browse/SOLR-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12675466#action_12675466
]
Jayson Minard commented on SOLR-1030:
-
Yeah, we had the assumption that the source data did not have duplicate docs
across shards, turns out that it does. Otherwise we would have checked that
first. I'll keep an eye on this one for a bit, but most likely just the
duplicate doc issue.
> Facet counts are not correct (or total document count is not correct as they
> do not match) on some searches
> ---
>
> Key: SOLR-1030
> URL: https://issues.apache.org/jira/browse/SOLR-1030
> Project: Solr
> Issue Type: Bug
> Components: search
>Affects Versions: 1.4
>Reporter: Jayson Minard
>
> -There isn't much detailed evidence for this one yet, but hopefully it rings
> a bell with someone who made changes in this area recently...-
> -Since updating to the tip from our previous use of the tip from around Jan
> 9, 2009- (seems to be previous to r733656 as well) we are now seeing facet
> counts no longer match total document count. This is through distributed
> search and I have not verified that it only happens on distributed vs. single
> shard search so it could be on both.
> For example, on a single valued field with one facet value set as a fq
> filter, combined with a text search on a simple term "science", the following
> is the facet count:
> 8,294,284
> And the total document count for the same results is:
> 8,294,274
> some debug info (not sure why the filter query is replicated more than once,
> but that shouldn't be harmful):
> {code}
> uerystring(science)
> QParser OldLuceneQParser
> filter_queries[sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article")]
> rawquerystring(science)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1030) Facet counts are not correct (or total document count is not correct as they do not match) on some searches
[
https://issues.apache.org/jira/browse/SOLR-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12675436#action_12675436
]
Jayson Minard commented on SOLR-1030:
-
I think this might indeed be duplicate docs between shards causing the
difference. As we increate page size from 0 to 20, going by 1 each time, we
slowly get a higher and higher count difference. And we find duplicate docs
across shards in this result set for the page of results being shown.
> Facet counts are not correct (or total document count is not correct as they
> do not match) on some searches
> ---
>
> Key: SOLR-1030
> URL: https://issues.apache.org/jira/browse/SOLR-1030
> Project: Solr
> Issue Type: Bug
> Components: search
>Affects Versions: 1.4
>Reporter: Jayson Minard
>
> -There isn't much detailed evidence for this one yet, but hopefully it rings
> a bell with someone who made changes in this area recently...-
> -Since updating to the tip from our previous use of the tip from around Jan
> 9, 2009- (seems to be previous to r733656 as well) we are now seeing facet
> counts no longer match total document count. This is through distributed
> search and I have not verified that it only happens on distributed vs. single
> shard search so it could be on both.
> For example, on a single valued field with one facet value set as a fq
> filter, combined with a text search on a simple term "science", the following
> is the facet count:
> 8,294,284
> And the total document count for the same results is:
> 8,294,274
> some debug info (not sure why the filter query is replicated more than once,
> but that shouldn't be harmful):
> {code}
> uerystring(science)
> QParser OldLuceneQParser
> filter_queries[sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article")]
> rawquerystring(science)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (SOLR-1030) Facet counts are not correct (or total document count is not correct as they do not match) on some searches
[
https://issues.apache.org/jira/browse/SOLR-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12675405#action_12675405
]
jminard edited comment on SOLR-1030 at 2/20/09 10:15 AM:
---
-The last known working revision should be 733,656 as that includes patches we
passed through Shalin and counts matched at that time.- Appears to be previous
to this revision.
was (Author: jminard):
The last known working revision should be 733,656 as that includes patches
we passed through Shalin and counts matched at that time.
> Facet counts are not correct (or total document count is not correct as they
> do not match) on some searches
> ---
>
> Key: SOLR-1030
> URL: https://issues.apache.org/jira/browse/SOLR-1030
> Project: Solr
> Issue Type: Bug
> Components: search
>Affects Versions: 1.4
>Reporter: Jayson Minard
>
> -There isn't much detailed evidence for this one yet, but hopefully it rings
> a bell with someone who made changes in this area recently...-
> -Since updating to the tip from our previous use of the tip from around Jan
> 9, 2009- (seems to be previous to r733656 as well) we are now seeing facet
> counts no longer match total document count. This is through distributed
> search and I have not verified that it only happens on distributed vs. single
> shard search so it could be on both.
> For example, on a single valued field with one facet value set as a fq
> filter, combined with a text search on a simple term "science", the following
> is the facet count:
> 8,294,284
> And the total document count for the same results is:
> 8,294,274
> some debug info (not sure why the filter query is replicated more than once,
> but that shouldn't be harmful):
> {code}
> uerystring(science)
> QParser OldLuceneQParser
> filter_queries[sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article")]
> rawquerystring(science)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1030) Facet counts are not correct (or total document count is not correct as they do not match) on some searches
[
https://issues.apache.org/jira/browse/SOLR-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jayson Minard updated SOLR-1030:
Description:
-There isn't much detailed evidence for this one yet, but hopefully it rings a
bell with someone who made changes in this area recently...-
-Since updating to the tip from our previous use of the tip from around Jan 9,
2009- (seems to be previous to r733656 as well) we are now seeing facet counts
no longer match total document count. This is through distributed search and I
have not verified that it only happens on distributed vs. single shard search
so it could be on both.
For example, on a single valued field with one facet value set as a fq filter,
combined with a text search on a simple term "science", the following is the
facet count:
8,294,284
And the total document count for the same results is:
8,294,274
some debug info (not sure why the filter query is replicated more than once,
but that shouldn't be harmful):
{code}
uerystring (science)
QParser OldLuceneQParser
filter_queries [sys_content_type:("Journal Article"),
sys_content_type:("Journal Article"),
sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
sys_content_type:("Journal Article"), sys_content_type:("Journal Article")]
rawquerystring (science)
{code}
was:
There isn't much detailed evidence for this one yet, but hopefully it rings a
bell with someone who made changes in this area recently...
Since updating to the tip from our previous use of the tip from around Jan 9,
2009 we are now seeing facet counts no longer match total document count. This
is through distributed search and I have not verified that it only happens on
distributed vs. single shard search so it could be on both.
For example, on a single valued field with one facet value set as a fq filter,
combined with a text search on a simple term "science", the following is the
facet count:
8,294,284
And the total document count for the same results is:
8,294,274
some debug info (not sure why the filter query is replicated more than once,
but that shouldn't be harmful):
{code}
uerystring (science)
QParser OldLuceneQParser
filter_queries [sys_content_type:("Journal Article"),
sys_content_type:("Journal Article"),
sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
sys_content_type:("Journal Article"), sys_content_type:("Journal Article")]
rawquerystring (science)
{code}
> Facet counts are not correct (or total document count is not correct as they
> do not match) on some searches
> ---
>
> Key: SOLR-1030
> URL: https://issues.apache.org/jira/browse/SOLR-1030
> Project: Solr
> Issue Type: Bug
> Components: search
>Affects Versions: 1.4
>Reporter: Jayson Minard
>
> -There isn't much detailed evidence for this one yet, but hopefully it rings
> a bell with someone who made changes in this area recently...-
> -Since updating to the tip from our previous use of the tip from around Jan
> 9, 2009- (seems to be previous to r733656 as well) we are now seeing facet
> counts no longer match total document count. This is through distributed
> search and I have not verified that it only happens on distributed vs. single
> shard search so it could be on both.
> For example, on a single valued field with one facet value set as a fq
> filter, combined with a text search on a simple term "science", the following
> is the facet count:
> 8,294,284
> And the total document count for the same results is:
> 8,294,274
> some debug info (not sure why the filter query is replicated more than once,
> but that shouldn't be harmful):
> {code}
> uerystring(science)
> QParser OldLuceneQParser
> filter_queries[sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article")]
> rawquerystring(science)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (SOLR-1030) Facet counts are not correct (or total document count is not correct as they do not match) on some searches
[
https://issues.apache.org/jira/browse/SOLR-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12675416#action_12675416
]
jminard edited comment on SOLR-1030 at 2/20/09 10:16 AM:
---
And the data hasn't changed since the last version of Solr was used (roughly
r733656) and either no one noticed it before, -or the problem didn't previously
exist-. Checking now for timeouts in the logs.
was (Author: jminard):
And the data hasn't changed since the last version of Solr was used
(roughly r733656) and either no one noticed it before, or the problem didn't
previously exist. Checking now for timeouts in the logs.
> Facet counts are not correct (or total document count is not correct as they
> do not match) on some searches
> ---
>
> Key: SOLR-1030
> URL: https://issues.apache.org/jira/browse/SOLR-1030
> Project: Solr
> Issue Type: Bug
> Components: search
>Affects Versions: 1.4
>Reporter: Jayson Minard
>
> -There isn't much detailed evidence for this one yet, but hopefully it rings
> a bell with someone who made changes in this area recently...-
> -Since updating to the tip from our previous use of the tip from around Jan
> 9, 2009- (seems to be previous to r733656 as well) we are now seeing facet
> counts no longer match total document count. This is through distributed
> search and I have not verified that it only happens on distributed vs. single
> shard search so it could be on both.
> For example, on a single valued field with one facet value set as a fq
> filter, combined with a text search on a simple term "science", the following
> is the facet count:
> 8,294,284
> And the total document count for the same results is:
> 8,294,274
> some debug info (not sure why the filter query is replicated more than once,
> but that shouldn't be harmful):
> {code}
> uerystring(science)
> QParser OldLuceneQParser
> filter_queries[sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article")]
> rawquerystring(science)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1030) Facet counts are not correct (or total document count is not correct as they do not match) on some searches
[
https://issues.apache.org/jira/browse/SOLR-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jayson Minard updated SOLR-1030:
Comment: was deleted
> Facet counts are not correct (or total document count is not correct as they
> do not match) on some searches
> ---
>
> Key: SOLR-1030
> URL: https://issues.apache.org/jira/browse/SOLR-1030
> Project: Solr
> Issue Type: Bug
> Components: search
>Affects Versions: 1.4
>Reporter: Jayson Minard
>
> -There isn't much detailed evidence for this one yet, but hopefully it rings
> a bell with someone who made changes in this area recently...-
> -Since updating to the tip from our previous use of the tip from around Jan
> 9, 2009- (seems to be previous to r733656 as well) we are now seeing facet
> counts no longer match total document count. This is through distributed
> search and I have not verified that it only happens on distributed vs. single
> shard search so it could be on both.
> For example, on a single valued field with one facet value set as a fq
> filter, combined with a text search on a simple term "science", the following
> is the facet count:
> 8,294,284
> And the total document count for the same results is:
> 8,294,274
> some debug info (not sure why the filter query is replicated more than once,
> but that shouldn't be harmful):
> {code}
> uerystring(science)
> QParser OldLuceneQParser
> filter_queries[sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article")]
> rawquerystring(science)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1030) Facet counts are not correct (or total document count is not correct as they do not match) on some searches
[
https://issues.apache.org/jira/browse/SOLR-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12675421#action_12675421
]
Jayson Minard commented on SOLR-1030:
-
I just found a bug in our database from the 27th showing this issue dates back
to r733656 as well and before. So it is an older issue.
End up with cases where facets show counts such as 264 where total results are
249.
> Facet counts are not correct (or total document count is not correct as they
> do not match) on some searches
> ---
>
> Key: SOLR-1030
> URL: https://issues.apache.org/jira/browse/SOLR-1030
> Project: Solr
> Issue Type: Bug
> Components: search
>Affects Versions: 1.4
>Reporter: Jayson Minard
>
> There isn't much detailed evidence for this one yet, but hopefully it rings a
> bell with someone who made changes in this area recently...
> Since updating to the tip from our previous use of the tip from around Jan 9,
> 2009 we are now seeing facet counts no longer match total document count.
> This is through distributed search and I have not verified that it only
> happens on distributed vs. single shard search so it could be on both.
> For example, on a single valued field with one facet value set as a fq
> filter, combined with a text search on a simple term "science", the following
> is the facet count:
> 8,294,284
> And the total document count for the same results is:
> 8,294,274
> some debug info (not sure why the filter query is replicated more than once,
> but that shouldn't be harmful):
> {code}
> uerystring(science)
> QParser OldLuceneQParser
> filter_queries[sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article")]
> rawquerystring(science)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1030) Facet counts are not correct (or total document count is not correct as they do not match) on some searches
[
https://issues.apache.org/jira/browse/SOLR-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12675418#action_12675418
]
Jayson Minard commented on SOLR-1030:
-
We are reverting to r733656 to test there again and see if the problem is new
since that revision or if we just now noticed it. There is a bug in our
database showing a count problem from last month but it appears to be related
to duplicate docs on different shards rather than this issue.
> Facet counts are not correct (or total document count is not correct as they
> do not match) on some searches
> ---
>
> Key: SOLR-1030
> URL: https://issues.apache.org/jira/browse/SOLR-1030
> Project: Solr
> Issue Type: Bug
> Components: search
>Affects Versions: 1.4
>Reporter: Jayson Minard
>
> There isn't much detailed evidence for this one yet, but hopefully it rings a
> bell with someone who made changes in this area recently...
> Since updating to the tip from our previous use of the tip from around Jan 9,
> 2009 we are now seeing facet counts no longer match total document count.
> This is through distributed search and I have not verified that it only
> happens on distributed vs. single shard search so it could be on both.
> For example, on a single valued field with one facet value set as a fq
> filter, combined with a text search on a simple term "science", the following
> is the facet count:
> 8,294,284
> And the total document count for the same results is:
> 8,294,274
> some debug info (not sure why the filter query is replicated more than once,
> but that shouldn't be harmful):
> {code}
> uerystring(science)
> QParser OldLuceneQParser
> filter_queries[sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article")]
> rawquerystring(science)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1030) Facet counts are not correct (or total document count is not correct as they do not match) on some searches
[
https://issues.apache.org/jira/browse/SOLR-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12675417#action_12675417
]
Jayson Minard commented on SOLR-1030:
-
Logs on aggregator instance of Solr show no exceptions. Logs on query slaves
show no exceptions. (or errors)
> Facet counts are not correct (or total document count is not correct as they
> do not match) on some searches
> ---
>
> Key: SOLR-1030
> URL: https://issues.apache.org/jira/browse/SOLR-1030
> Project: Solr
> Issue Type: Bug
> Components: search
>Affects Versions: 1.4
>Reporter: Jayson Minard
>
> There isn't much detailed evidence for this one yet, but hopefully it rings a
> bell with someone who made changes in this area recently...
> Since updating to the tip from our previous use of the tip from around Jan 9,
> 2009 we are now seeing facet counts no longer match total document count.
> This is through distributed search and I have not verified that it only
> happens on distributed vs. single shard search so it could be on both.
> For example, on a single valued field with one facet value set as a fq
> filter, combined with a text search on a simple term "science", the following
> is the facet count:
> 8,294,284
> And the total document count for the same results is:
> 8,294,274
> some debug info (not sure why the filter query is replicated more than once,
> but that shouldn't be harmful):
> {code}
> uerystring(science)
> QParser OldLuceneQParser
> filter_queries[sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article")]
> rawquerystring(science)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1030) Facet counts are not correct (or total document count is not correct as they do not match) on some searches
[
https://issues.apache.org/jira/browse/SOLR-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12675416#action_12675416
]
Jayson Minard commented on SOLR-1030:
-
And the data hasn't changed since the last version of Solr was used (roughly
r733656) and either no one noticed it before, or the problem didn't previously
exist. Checking now for timeouts in the logs.
> Facet counts are not correct (or total document count is not correct as they
> do not match) on some searches
> ---
>
> Key: SOLR-1030
> URL: https://issues.apache.org/jira/browse/SOLR-1030
> Project: Solr
> Issue Type: Bug
> Components: search
>Affects Versions: 1.4
>Reporter: Jayson Minard
>
> There isn't much detailed evidence for this one yet, but hopefully it rings a
> bell with someone who made changes in this area recently...
> Since updating to the tip from our previous use of the tip from around Jan 9,
> 2009 we are now seeing facet counts no longer match total document count.
> This is through distributed search and I have not verified that it only
> happens on distributed vs. single shard search so it could be on both.
> For example, on a single valued field with one facet value set as a fq
> filter, combined with a text search on a simple term "science", the following
> is the facet count:
> 8,294,284
> And the total document count for the same results is:
> 8,294,274
> some debug info (not sure why the filter query is replicated more than once,
> but that shouldn't be harmful):
> {code}
> uerystring(science)
> QParser OldLuceneQParser
> filter_queries[sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article")]
> rawquerystring(science)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1030) Facet counts are not correct (or total document count is not correct as they do not match) on some searches
[
https://issues.apache.org/jira/browse/SOLR-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12675415#action_12675415
]
Jayson Minard commented on SOLR-1030:
-
The data is static, no updates.
> Facet counts are not correct (or total document count is not correct as they
> do not match) on some searches
> ---
>
> Key: SOLR-1030
> URL: https://issues.apache.org/jira/browse/SOLR-1030
> Project: Solr
> Issue Type: Bug
> Components: search
>Affects Versions: 1.4
>Reporter: Jayson Minard
>
> There isn't much detailed evidence for this one yet, but hopefully it rings a
> bell with someone who made changes in this area recently...
> Since updating to the tip from our previous use of the tip from around Jan 9,
> 2009 we are now seeing facet counts no longer match total document count.
> This is through distributed search and I have not verified that it only
> happens on distributed vs. single shard search so it could be on both.
> For example, on a single valued field with one facet value set as a fq
> filter, combined with a text search on a simple term "science", the following
> is the facet count:
> 8,294,284
> And the total document count for the same results is:
> 8,294,274
> some debug info (not sure why the filter query is replicated more than once,
> but that shouldn't be harmful):
> {code}
> uerystring(science)
> QParser OldLuceneQParser
> filter_queries[sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article")]
> rawquerystring(science)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1030) Facet counts are not correct (or total document count is not correct as they do not match) on some searches
[
https://issues.apache.org/jira/browse/SOLR-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12675406#action_12675406
]
Jayson Minard commented on SOLR-1030:
-
Possible commits that could impact this area of Solr:
Rev 738606 - Upgrade to Lucene 2.9-dev r738345 by Yonik
Rev 738950 - Upgrade to Lucene 2.9-dev r738622 by Yonik
Rev 739305 - Revert to r738218 of Lucene by Yonik
Rev 740319 - Change SolrIndexSearcher to use insertWithOverflow by Yonik
Rev 741710 - Addition of timeouts for distrubted searching by Shalin (maybe we
time out on some shards?)
Rev 742988 - Make QueryComponent Optional by gsingers
Rev 743196 - Current lucene libs, SolrIndexReader introduction for
FileFloatSource fix by Yonik
So possibly a Lucene bug?
> Facet counts are not correct (or total document count is not correct as they
> do not match) on some searches
> ---
>
> Key: SOLR-1030
> URL: https://issues.apache.org/jira/browse/SOLR-1030
> Project: Solr
> Issue Type: Bug
> Components: search
>Affects Versions: 1.4
>Reporter: Jayson Minard
>
> There isn't much detailed evidence for this one yet, but hopefully it rings a
> bell with someone who made changes in this area recently...
> Since updating to the tip from our previous use of the tip from around Jan 9,
> 2009 we are now seeing facet counts no longer match total document count.
> This is through distributed search and I have not verified that it only
> happens on distributed vs. single shard search so it could be on both.
> For example, on a single valued field with one facet value set as a fq
> filter, combined with a text search on a simple term "science", the following
> is the facet count:
> 8,294,284
> And the total document count for the same results is:
> 8,294,274
> some debug info (not sure why the filter query is replicated more than once,
> but that shouldn't be harmful):
> {code}
> uerystring(science)
> QParser OldLuceneQParser
> filter_queries[sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article")]
> rawquerystring(science)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1030) Facet counts are not correct (or total document count is not correct as they do not match) on some searches
[
https://issues.apache.org/jira/browse/SOLR-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12675405#action_12675405
]
Jayson Minard commented on SOLR-1030:
-
The last known working revision should be 733,656 as that includes patches we
passed through Shalin and counts matched at that time.
> Facet counts are not correct (or total document count is not correct as they
> do not match) on some searches
> ---
>
> Key: SOLR-1030
> URL: https://issues.apache.org/jira/browse/SOLR-1030
> Project: Solr
> Issue Type: Bug
> Components: search
>Affects Versions: 1.4
>Reporter: Jayson Minard
>
> There isn't much detailed evidence for this one yet, but hopefully it rings a
> bell with someone who made changes in this area recently...
> Since updating to the tip from our previous use of the tip from around Jan 9,
> 2009 we are now seeing facet counts no longer match total document count.
> This is through distributed search and I have not verified that it only
> happens on distributed vs. single shard search so it could be on both.
> For example, on a single valued field with one facet value set as a fq
> filter, combined with a text search on a simple term "science", the following
> is the facet count:
> 8,294,284
> And the total document count for the same results is:
> 8,294,274
> some debug info (not sure why the filter query is replicated more than once,
> but that shouldn't be harmful):
> {code}
> uerystring(science)
> QParser OldLuceneQParser
> filter_queries[sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
> sys_content_type:("Journal Article"), sys_content_type:("Journal Article")]
> rawquerystring(science)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Created: (SOLR-1030) Facet counts are not correct (or total document count is not correct as they do not match) on some searches
Facet counts are not correct (or total document count is not correct as they do
not match) on some searches
---
Key: SOLR-1030
URL: https://issues.apache.org/jira/browse/SOLR-1030
Project: Solr
Issue Type: Bug
Components: search
Affects Versions: 1.4
Reporter: Jayson Minard
There isn't much detailed evidence for this one yet, but hopefully it rings a
bell with someone who made changes in this area recently...
Since updating to the tip from our previous use of the tip from around Jan 9,
2009 we are now seeing facet counts no longer match total document count. This
is through distributed search and I have not verified that it only happens on
distributed vs. single shard search so it could be on both.
For example, on a single valued field with one facet value set as a fq filter,
combined with a text search on a simple term "science", the following is the
facet count:
8,294,284
And the total document count for the same results is:
8,294,274
some debug info (not sure why the filter query is replicated more than once,
but that shouldn't be harmful):
{code}
uerystring (science)
QParser OldLuceneQParser
filter_queries [sys_content_type:("Journal Article"),
sys_content_type:("Journal Article"),
sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
sys_content_type:("Journal Article"), sys_content_type:("Journal Article"),
sys_content_type:("Journal Article"), sys_content_type:("Journal Article")]
rawquerystring (science)
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-950) Solr can throw an exception on distributed search due to bad if statement in FacetComponent.java:99
[
https://issues.apache.org/jira/browse/SOLR-950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12662350#action_12662350
]
Jayson Minard commented on SOLR-950:
Shalin, thanks for committing the fix.
> Solr can throw an exception on distributed search due to bad if statement in
> FacetComponent.java:99
> ---
>
> Key: SOLR-950
> URL: https://issues.apache.org/jira/browse/SOLR-950
> Project: Solr
> Issue Type: Bug
> Components: search
>Affects Versions: 1.3
>Reporter: Jayson Minard
>Assignee: Shalin Shekhar Mangar
>Priority: Critical
> Fix For: 1.4
>
>
> When issuing a distributed request, the aggregator node caused an exception
> due to the following code at line 99 in FacetComponent.java
> {code}
> if (refList == null | refList.size()==0) continue;
> {code}
> (!) note the single pipe for a bitwise OR rather than boolean OR
> Change code to:
> {code}
> if (refList == null || refList.size() == 0) continue;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-951) Distributed query fails with tag / exclude pattern used for facets that are multi-valued.
[
https://issues.apache.org/jira/browse/SOLR-951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12662349#action_12662349
]
Jayson Minard commented on SOLR-951:
Ok, FacetComponent.java creates the parameter that is passed down and adds the
"__terms" that is at the end of the original tag. Just below that on line 108
is the place where it appends the new parameters to the existing.
{code}
if (dff.localParams != null) {
facetCommand = commandPrefix+termsKey+dff.facetStr.substring(2);
} else {
facetCommand = commandPrefix+termsKey+'}'+dff.field;
}
{code}
should have white space between the params, such as...
{code}
if (dff.localParams != null) {
facetCommand = commandPrefix+termsKey+" "+dff.facetStr.substring(2);
} else {
facetCommand = commandPrefix+termsKey+'}'+dff.field;
}
{code}
I tested this fix and it no longer throws an exception, I have not tested that
it ensures the tag/exclude feature still works though.
> Distributed query fails with tag / exclude pattern used for facets that are
> multi-valued.
> -
>
> Key: SOLR-951
> URL: https://issues.apache.org/jira/browse/SOLR-951
> Project: Solr
> Issue Type: Bug
> Components: search
>Affects Versions: 1.3.1
>Reporter: Jayson Minard
>Priority: Critical
>
> Running this query:
> {code}
> http://testserver:8985/solr/select/?q=*%3A*&version=2.2&start=0&rows=10&indent=on&facet=true&;
> facet.field={!ex=t1}SubjectTerms_mfacet&fq={!tag=t1}SubjectTerms_mfacet:(testvalue)&
> shards=test1:8985/solr,test2:8985/solr
> {code}
> causes this exception:
> {code}
> java.lang.NullPointerException
> at
> org.apache.solr.handler.component.FacetComponent.refineFacets(FacetComponent.java:330)
> at
> org.apache.solr.handler.component.FacetComponent.handleResponses(FacetComponent.java:231)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:266)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1325)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
> at
> org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
> at org.mortbay.jetty.Server.handle(Server.java:285)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
> at
> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
> at
> org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
> {code}
> which is at this area of the code in the aggregator, but not the root cause...
> {code}
> private void refineFacets(ResponseBuilder rb, ShardRequest sreq) {
> FacetInfo fi = rb._facetInfo;
> for (ShardResponse srsp: sreq.responses) {
> // int shardNum = rb.getShardNum(srsp.shard);
> NamedList facet_counts =
> (NamedList)srsp.getSolrResponse().getResponse().get("facet_counts");
> NamedList facet_fields = (NamedList)facet_counts.get("facet_fields");
>
> for (int i=0; i EXCEPTION LINE 330
> String key = facet_fields.getName(i);
> DistribFieldFacet dff = (DistribFieldFacet)fi.facets.get(key);
> if (dff == null) continue;
> NamedList shardCounts = (NamedList)facet_fields.getVal(i);
> for (int j=0; j String name = shardCounts.getName(j);
> long count = ((Number)shardCounts.getVal(j)).longValue()
[jira] Commented: (SOLR-951) Distributed query fails with tag / exclude pattern used for facets that are multi-valued.
[
https://issues.apache.org/jira/browse/SOLR-951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12662344#action_12662344
]
Jayson Minard commented on SOLR-951:
ok so if the string being parsed is:
{code}
{!terms=$SubjectTerms_mfacet__termsex=t1}SubjectTerms_mfacet
{code}
And this enters parseLocalParams() in QueryParsing...
It would see the "{!" and then start parsing the remainder
"terms=$SubjectTerms_mfacet__termsex=t1}SubjectTerms_mfacet" using the first
getId call which succeeds and finds "terms" and then the "=" via the peek, and
then enters the if that sees the "$" and decides to dereference the parameter.
It immediately calls getId() again which pulls out
"$SubjectTerms_mfacet__termsex" and then loops to the top of the loop and calls
getId() a second time picking up the final "=" which is then an error. So,
something is missing in the parameters. Is "ex=t1" another parameter and
should have a delimiter? Ah, it should have a space between there somewhere.
So the parsing is fine, but the creation of the string isn't. Searching
upwards the stack trace now.
> Distributed query fails with tag / exclude pattern used for facets that are
> multi-valued.
> -
>
> Key: SOLR-951
> URL: https://issues.apache.org/jira/browse/SOLR-951
> Project: Solr
> Issue Type: Bug
> Components: search
>Affects Versions: 1.3.1
>Reporter: Jayson Minard
>Priority: Critical
>
> Running this query:
> {code}
> http://testserver:8985/solr/select/?q=*%3A*&version=2.2&start=0&rows=10&indent=on&facet=true&;
> facet.field={!ex=t1}SubjectTerms_mfacet&fq={!tag=t1}SubjectTerms_mfacet:(testvalue)&
> shards=test1:8985/solr,test2:8985/solr
> {code}
> causes this exception:
> {code}
> java.lang.NullPointerException
> at
> org.apache.solr.handler.component.FacetComponent.refineFacets(FacetComponent.java:330)
> at
> org.apache.solr.handler.component.FacetComponent.handleResponses(FacetComponent.java:231)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:266)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1325)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
> at
> org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
> at org.mortbay.jetty.Server.handle(Server.java:285)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
> at
> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
> at
> org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
> {code}
> which is at this area of the code in the aggregator, but not the root cause...
> {code}
> private void refineFacets(ResponseBuilder rb, ShardRequest sreq) {
> FacetInfo fi = rb._facetInfo;
> for (ShardResponse srsp: sreq.responses) {
> // int shardNum = rb.getShardNum(srsp.shard);
> NamedList facet_counts =
> (NamedList)srsp.getSolrResponse().getResponse().get("facet_counts");
> NamedList facet_fields = (NamedList)facet_counts.get("facet_fields");
>
> for (int i=0; i EXCEPTION LINE 330
> String key = facet_fields.getName(i);
> DistribFieldFacet dff = (DistribFieldFacet)fi.facets.get(key);
> if (dff == null) continue;
> NamedList shardCounts = (NamedList)facet_fields.getVal(i);
> for (int j=0; j String name = shardCounts.getName
[jira] Updated: (SOLR-951) Distributed query fails with tag / exclude pattern used for facets that are multi-valued.
[
https://issues.apache.org/jira/browse/SOLR-951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jayson Minard updated SOLR-951:
---
Comment: was deleted
> Distributed query fails with tag / exclude pattern used for facets that are
> multi-valued.
> -
>
> Key: SOLR-951
> URL: https://issues.apache.org/jira/browse/SOLR-951
> Project: Solr
> Issue Type: Bug
> Components: search
>Affects Versions: 1.3.1
>Reporter: Jayson Minard
>Priority: Critical
>
> Running this query:
> {code}
> http://testserver:8985/solr/select/?q=*%3A*&version=2.2&start=0&rows=10&indent=on&facet=true&;
> facet.field={!ex=t1}SubjectTerms_mfacet&fq={!tag=t1}SubjectTerms_mfacet:(testvalue)&
> shards=test1:8985/solr,test2:8985/solr
> {code}
> causes this exception:
> {code}
> java.lang.NullPointerException
> at
> org.apache.solr.handler.component.FacetComponent.refineFacets(FacetComponent.java:330)
> at
> org.apache.solr.handler.component.FacetComponent.handleResponses(FacetComponent.java:231)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:266)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1325)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
> at
> org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
> at org.mortbay.jetty.Server.handle(Server.java:285)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
> at
> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
> at
> org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
> {code}
> which is at this area of the code in the aggregator, but not the root cause...
> {code}
> private void refineFacets(ResponseBuilder rb, ShardRequest sreq) {
> FacetInfo fi = rb._facetInfo;
> for (ShardResponse srsp: sreq.responses) {
> // int shardNum = rb.getShardNum(srsp.shard);
> NamedList facet_counts =
> (NamedList)srsp.getSolrResponse().getResponse().get("facet_counts");
> NamedList facet_fields = (NamedList)facet_counts.get("facet_fields");
>
> for (int i=0; i EXCEPTION LINE 330
> String key = facet_fields.getName(i);
> DistribFieldFacet dff = (DistribFieldFacet)fi.facets.get(key);
> if (dff == null) continue;
> NamedList shardCounts = (NamedList)facet_fields.getVal(i);
> for (int j=0; j String name = shardCounts.getName(j);
> long count = ((Number)shardCounts.getVal(j)).longValue();
> ShardFacetCount sfc = dff.counts.get(name);
> sfc.count += count;
> }
> }
> }
> }
> {code}
> The shard actually has an error on it, which is:
> {code}
> Jan 9, 2009 6:15:31 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/select
> params={facet=true&f.SubjectTerms_mfacet.facet.limit=160&wt=javabin&rows=10&
> version=2.2&fl=id,score&start=0&q=*:*&
> facet.field={!ex%3Dt1}SubjectTerms_mfacet&isShard=true&fq={!tag%3Dt1}SubjectTerms_mfacet:(testvalue)&
> fsv=true} hits=0 status=0 QTime=265
> Jan 9, 2009 6:15:31 AM org.apache.solr.common.SolrException log
> SEVERE: Exception during facet
> counts:org.apache.lucene.queryParser.ParseException: Expected identifier at
> pos 37 str='{!terms=$SubjectTerms_mfacet__termsex=t1}SubjectTerms_mfacet'
> at
> org.apache.solr.search.QueryParsing$StrParser.getId(QueryParsing.j
[jira] Commented: (SOLR-951) Distributed query fails with tag / exclude pattern used for facets that are multi-valued.
[
https://issues.apache.org/jira/browse/SOLR-951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12662337#action_12662337
]
Jayson Minard commented on SOLR-951:
Ok, looking at QueryParsing.java and ...
parseLocalParams() finds the start of an ID and then calls getId() to retrieve
it, but getId() doesn't know when to end (i.e. on "=" or "}") so it eats until
it hits an unexpected java identifier character (in this case "=") and bombs
with the exception.
THere either needs to be a special getId() function that knows when to quit
nicely, or getId() changed to have valid end tokens where it stops politely
rather than as an error.
I'm testing the quick variant function just for the localParams to work around
this for now.
> Distributed query fails with tag / exclude pattern used for facets that are
> multi-valued.
> -
>
> Key: SOLR-951
> URL: https://issues.apache.org/jira/browse/SOLR-951
> Project: Solr
> Issue Type: Bug
> Components: search
>Affects Versions: 1.3.1
>Reporter: Jayson Minard
>Priority: Critical
>
> Running this query:
> {code}
> http://testserver:8985/solr/select/?q=*%3A*&version=2.2&start=0&rows=10&indent=on&facet=true&;
> facet.field={!ex=t1}SubjectTerms_mfacet&fq={!tag=t1}SubjectTerms_mfacet:(testvalue)&
> shards=test1:8985/solr,test2:8985/solr
> {code}
> causes this exception:
> {code}
> java.lang.NullPointerException
> at
> org.apache.solr.handler.component.FacetComponent.refineFacets(FacetComponent.java:330)
> at
> org.apache.solr.handler.component.FacetComponent.handleResponses(FacetComponent.java:231)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:266)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1325)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
> at
> org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
> at org.mortbay.jetty.Server.handle(Server.java:285)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
> at
> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
> at
> org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
> {code}
> which is at this area of the code in the aggregator, but not the root cause...
> {code}
> private void refineFacets(ResponseBuilder rb, ShardRequest sreq) {
> FacetInfo fi = rb._facetInfo;
> for (ShardResponse srsp: sreq.responses) {
> // int shardNum = rb.getShardNum(srsp.shard);
> NamedList facet_counts =
> (NamedList)srsp.getSolrResponse().getResponse().get("facet_counts");
> NamedList facet_fields = (NamedList)facet_counts.get("facet_fields");
>
> for (int i=0; i EXCEPTION LINE 330
> String key = facet_fields.getName(i);
> DistribFieldFacet dff = (DistribFieldFacet)fi.facets.get(key);
> if (dff == null) continue;
> NamedList shardCounts = (NamedList)facet_fields.getVal(i);
> for (int j=0; j String name = shardCounts.getName(j);
> long count = ((Number)shardCounts.getVal(j)).longValue();
> ShardFacetCount sfc = dff.counts.get(name);
> sfc.count += count;
> }
> }
> }
> }
> {code}
> The shard actually has an error on it, which is:
> {code}
> Jan 9, 2009 6:15:31 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path
[jira] Updated: (SOLR-951) Distributed query fails with tag / exclude pattern used for facets that are multi-valued.
[
https://issues.apache.org/jira/browse/SOLR-951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jayson Minard updated SOLR-951:
---
Description:
Running this query:
{code}
http://testserver:8985/solr/select/?q=*%3A*&version=2.2&start=0&rows=10&indent=on&facet=true&;
facet.field={!ex=t1}SubjectTerms_mfacet&fq={!tag=t1}SubjectTerms_mfacet:(testvalue)&
shards=test1:8985/solr,test2:8985/solr
{code}
causes this exception:
{code}
java.lang.NullPointerException
at
org.apache.solr.handler.component.FacetComponent.refineFacets(FacetComponent.java:330)
at
org.apache.solr.handler.component.FacetComponent.handleResponses(FacetComponent.java:231)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:266)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1325)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
at org.mortbay.jetty.Server.handle(Server.java:285)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
at
org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
{code}
which is at this area of the code in the aggregator, but not the root cause...
{code}
private void refineFacets(ResponseBuilder rb, ShardRequest sreq) {
FacetInfo fi = rb._facetInfo;
for (ShardResponse srsp: sreq.responses) {
// int shardNum = rb.getShardNum(srsp.shard);
NamedList facet_counts =
(NamedList)srsp.getSolrResponse().getResponse().get("facet_counts");
NamedList facet_fields = (NamedList)facet_counts.get("facet_fields");
for (int i=0; ihttp://testserver:8985/solr/select/?q=*%3A*&version=2.2&start=0&rows=10&indent=on&facet=true&facet.field={!ex=t1}SubjectTerms_mfacet&fq={!tag=t1}SubjectTerms_mfacet:(testvalue)&shards=test1:8985/solr,test2:8985/solr
{code}
causes this exception:
{code}
java.lang.NullPointerException
at
org.apache.solr.handler.component.FacetComponent.refineFacets(FacetComponent.java:330)
at
org.apache.solr.handler.component.FacetComponent.handleResponses(FacetComponent.java:231)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:266)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1325)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
at org.mortbay.jetty.Server.handle(Server.java:285)
at
org.mortbay.jetty.HttpConnection.handleReq
[jira] Created: (SOLR-951) Distributed query fails with tag / exclude pattern used for facets that are multi-valued.
Distributed query fails with tag / exclude pattern used for facets that are
multi-valued.
-
Key: SOLR-951
URL: https://issues.apache.org/jira/browse/SOLR-951
Project: Solr
Issue Type: Bug
Components: search
Affects Versions: 1.3.1
Reporter: Jayson Minard
Priority: Critical
Running this query:
{code}
http://testserver:8985/solr/select/?q=*%3A*&version=2.2&start=0&rows=10&indent=on&facet=true&facet.field={!ex=t1}SubjectTerms_mfacet&fq={!tag=t1}SubjectTerms_mfacet:(testvalue)&shards=test1:8985/solr,test2:8985/solr
{code}
causes this exception:
{code}
java.lang.NullPointerException
at
org.apache.solr.handler.component.FacetComponent.refineFacets(FacetComponent.java:330)
at
org.apache.solr.handler.component.FacetComponent.handleResponses(FacetComponent.java:231)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:266)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1325)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
at org.mortbay.jetty.Server.handle(Server.java:285)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
at
org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
{code}
which is at this area of the code in the aggregator, but not the root cause...
{code}
private void refineFacets(ResponseBuilder rb, ShardRequest sreq) {
FacetInfo fi = rb._facetInfo;
for (ShardResponse srsp: sreq.responses) {
// int shardNum = rb.getShardNum(srsp.shard);
NamedList facet_counts =
(NamedList)srsp.getSolrResponse().getResponse().get("facet_counts");
NamedList facet_fields = (NamedList)facet_counts.get("facet_fields");
for (int i=0; i
[jira] Updated: (SOLR-950) Solr can throw an exception on distributed search due to bad if statement in FacetComponent.java:99
[
https://issues.apache.org/jira/browse/SOLR-950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jayson Minard updated SOLR-950:
---
Affects Version/s: (was: 1.4)
1.3.1
> Solr can throw an exception on distributed search due to bad if statement in
> FacetComponent.java:99
> ---
>
> Key: SOLR-950
> URL: https://issues.apache.org/jira/browse/SOLR-950
> Project: Solr
> Issue Type: Bug
> Components: search
>Affects Versions: 1.3.1
>Reporter: Jayson Minard
>Priority: Critical
> Fix For: 1.4
>
>
> When issuing a distributed request, the aggregator node caused an exception
> due to the following code at line 99 in FacetComponent.java
> {code}
> if (refList == null | refList.size()==0) continue;
> {code}
> (!) note the single pipe for a bitwise OR rather than boolean OR
> Change code to:
> {code}
> if (refList == null || refList.size() == 0) continue;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Created: (SOLR-950) Solr can throw an exception on distributed search due to bad if statement in FacetComponent.java:99
Solr can throw an exception on distributed search due to bad if statement in
FacetComponent.java:99
---
Key: SOLR-950
URL: https://issues.apache.org/jira/browse/SOLR-950
Project: Solr
Issue Type: Bug
Components: search
Affects Versions: 1.4
Reporter: Jayson Minard
Priority: Critical
When issuing a distributed request, the aggregator node caused an exception due
to the following code at line 99 in FacetComponent.java
{code}
if (refList == null | refList.size()==0) continue;
{code}
(!) note the single pipe for a bitwise OR rather than boolean OR
Change code to:
{code}
if (refList == null || refList.size() == 0) continue;
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-936) Facet Results - REST vs SolrJ
[ https://issues.apache.org/jira/browse/SOLR-936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12658904#action_12658904 ] Jayson Minard commented on SOLR-936: erk. Guess I got caught up in the fun and games... I always thing of getFacetFields() as the list of facet fields passed in as the requested set of facets, not the response. Don't know where my brain was at. Guessing that solves Andrew's problem and this should be closed. Andrew? > Facet Results - REST vs SolrJ > - > > Key: SOLR-936 > URL: https://issues.apache.org/jira/browse/SOLR-936 > Project: Solr > Issue Type: Bug > Components: clients - java, search >Affects Versions: 1.4 >Reporter: Andrew Nagy >Priority: Minor > Fix For: 1.4 > > > There is a difference in the way Facet results are reported in SolrJ from the > REST interface. In REST, if you apply a facet via the fq param, no matter > what the count it is always reported back in the list of facets in the > responses. However, with SolrJ - it only reports back facets that don't > match the total number of documents. This is quite frustrating to deal with. > > The difference can be seen when ORing or ANDing in the fq param. When I or > to facet values together, they come back in SolrJ since their counts don't > match the total docs. But if I AND them together, they don't appear in the > list. So then I need to munge in the applied fq values. > Why the difference in behavior between REST and SolrJ? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-936) Facet Results - REST vs SolrJ
[ https://issues.apache.org/jira/browse/SOLR-936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12658874#action_12658874 ] Jayson Minard commented on SOLR-936: I see what Andrew is talking about. The difference is this: Solrj returns a list of further refinements, so it strips out anything that would not further reduce the set. It says, if the facet count == the total document count, don't return it in the list of further refinements. Whereas the raw response obviously has not yet stripped it. So Solrj could probably use a list of facets that are not further refinements if it is interesting for users of the API to have those. They don't make sense as possible new refinements but they do make sense in that they exist and some people care that they exist in the resulting data. > Facet Results - REST vs SolrJ > - > > Key: SOLR-936 > URL: https://issues.apache.org/jira/browse/SOLR-936 > Project: Solr > Issue Type: Bug > Components: clients - java, search >Affects Versions: 1.4 >Reporter: Andrew Nagy >Priority: Minor > Fix For: 1.4 > > > There is a difference in the way Facet results are reported in SolrJ from the > REST interface. In REST, if you apply a facet via the fq param, no matter > what the count it is always reported back in the list of facets in the > responses. However, with SolrJ - it only reports back facets that don't > match the total number of documents. This is quite frustrating to deal with. > > The difference can be seen when ORing or ANDing in the fq param. When I or > to facet values together, they come back in SolrJ since their counts don't > match the total docs. But if I AND them together, they don't appear in the > list. So then I need to munge in the applied fq values. > Why the difference in behavior between REST and SolrJ? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (SOLR-890) Solr distribution solr.war ends up with duplicate sfl4j jars in the WEB-INF lib directory
Solr distribution solr.war ends up with duplicate sfl4j jars in the WEB-INF lib directory - Key: SOLR-890 URL: https://issues.apache.org/jira/browse/SOLR-890 Project: Solr Issue Type: Bug Affects Versions: 1.4 Reporter: Jayson Minard Priority: Minor In the nightly builds, or in a local build, the solr.war file ends up with duplicate files in the WEB-INF lib directory. In most cases this isn't a notable problem, but some commands on the WAR file via jar util will cause failures. This is more of a cleaning issue, although it could have other consequences that I am not yet aware of to have duplicate directory listings in a zip file. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-802) Trivial bug which could cause null pointer exception
[ https://issues.apache.org/jira/browse/SOLR-802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12637595#action_12637595 ] Jayson Minard commented on SOLR-802: Thanks David! And Ryan, can the title be adjusted to indicate where and when this could happen so that release notes are more obvious? > Trivial bug which could cause null pointer exception > > > Key: SOLR-802 > URL: https://issues.apache.org/jira/browse/SOLR-802 > Project: Solr > Issue Type: Bug > Components: search >Affects Versions: 1.3 >Reporter: David Bowen >Assignee: Ryan McKinley >Priority: Minor > Attachments: patch > > > In FacetComponent there's an & which should be a &&. Patch to follow. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-565) Component to abstract shards from clients
[ https://issues.apache.org/jira/browse/SOLR-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12598506#action_12598506 ] Jayson Minard commented on SOLR-565: Selecting shards by sets is not overly fancy. You basically allow shards to be specified by location, then you allow shard sets to be specified including those sets. You reference the set (by default there is an "All" set) during the query and you are off to the races. Shard selection by sets covers a lot of ground in terms of bringing in more use cases without adding that much more complexity. Really, not much complexity, just a bit more code. > Component to abstract shards from clients > - > > Key: SOLR-565 > URL: https://issues.apache.org/jira/browse/SOLR-565 > Project: Solr > Issue Type: New Feature > Components: search >Affects Versions: 1.3 >Reporter: patrick o'leary >Priority: Minor > Attachments: distributor_component.patch > > > A component that will remove the need for calling clients to provide the > shards parameter for > a distributed search. > As systems grow, it's better to manage shards with in solr, rather than > managing each client. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-551) SOlr replication should include the schema also
[ https://issues.apache.org/jira/browse/SOLR-551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12598476#action_12598476 ] Jayson Minard commented on SOLR-551: Why is the schema stored outside of the index? Is another possible option to store it in a magic record within the index? That allows anyone to retrieve it that wants to see the schema, for example the UI might want to know the static fields quickly and can use the schema to determine that information. Basically, can some meta-data about the index be stored in the index which solves the replication problem, and makes it more easily accessible to the outside world? > SOlr replication should include the schema also > --- > > Key: SOLR-551 > URL: https://issues.apache.org/jira/browse/SOLR-551 > Project: Solr > Issue Type: Improvement > Components: replication >Affects Versions: 1.3 >Reporter: Noble Paul > > The current Solr replication just copy the data directory . So if the > schema changes and I do a re-index it will blissfully copy the index > and the slaves will fail because of incompatible schema. > So the steps we follow are > * Stop rsync on slaves > * Update the master with new schema > * re-index data > * forEach slave > ** Kill the slave > ** clean the data directory > ** install the new schema > ** restart > ** do a manual snappull > The amount of work the admin needs to do is quite significant > (depending on the no:of slaves). These are manual steps and very error > prone > The solution : > Make the replication mechanism handle the schema replication also. So > all I need to do is to just change the master and the slaves synch > automatically > What is a good way to implement this? > We have an idea along the following lines > This should involve changes to the snapshooter and snappuller scripts > and the snapinstaller components > Everytime the snapshooter takes a snapshot it must keep the timestamps > of schema.xml and elevate.xml (all the files which might affect the > runtime behavior in slaves) > For subsequent snapshots if the timestamps of any of them is changed > it must copy the all of them also for replication. > The snappuller copies the new directory as usual > The snapinstaller checks if these config files are present , > if yes, > * It can create a temporary core > * install the changed index and configuration > * load it completely and swap it out with the original core -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-565) Component to abstract shards from clients
[ https://issues.apache.org/jira/browse/SOLR-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12598473#action_12598473 ] Jayson Minard commented on SOLR-565: Another item to consider: Some times you want to control which shards participate in any given query. This is an important optimization for large scale deployments that need to quickly subset what is queried so that they do not waste CPU of irrelevant shards. > Component to abstract shards from clients > - > > Key: SOLR-565 > URL: https://issues.apache.org/jira/browse/SOLR-565 > Project: Solr > Issue Type: New Feature > Components: search >Affects Versions: 1.3 >Reporter: patrick o'leary >Priority: Minor > Attachments: distributor_component.patch > > > A component that will remove the need for calling clients to provide the > shards parameter for > a distributed search. > As systems grow, it's better to manage shards with in solr, rather than > managing each client. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-303) Distributed Search over HTTP
[ https://issues.apache.org/jira/browse/SOLR-303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12580403#action_12580403 ] Jayson Minard commented on SOLR-303: I'll see if I can work up a patch tonight on the extended response... > Distributed Search over HTTP > > > Key: SOLR-303 > URL: https://issues.apache.org/jira/browse/SOLR-303 > Project: Solr > Issue Type: New Feature > Components: search >Reporter: Sharad Agarwal >Assignee: Yonik Seeley > Attachments: distributed.patch, distributed.patch, distributed.patch, > distributed.patch, distributed.patch, distributed.patch, distributed.patch, > distributed.patch, distributed.patch, distributed.patch, distributed.patch, > distributed.patch, distributed_add_tests_for_intended_behavior.patch, > distributed_facet_count_bugfix.patch, distributed_pjaol.patch, > fedsearch.patch, fedsearch.patch, fedsearch.patch, fedsearch.patch, > fedsearch.patch, fedsearch.patch, fedsearch.patch, fedsearch.stu.patch, > fedsearch.stu.patch > > > Searching over multiple shards and aggregating results. > Motivated by http://wiki.apache.org/solr/DistributedSearch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-303) Distributed Search over HTTP
[ https://issues.apache.org/jira/browse/SOLR-303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12580188#action_12580188 ] Jayson Minard commented on SOLR-303: Would it be interesting to others to have an extended response format for distributed queries that would bring back the list of shards numbered, and then code each element of the response with the source list of shards that contributed to the element appearing in the results? For example, which shard was the source of a document? Or which shards had the facet value present? And so on. In really high shard counts it is more efficient if you can trim follow-on queries and pivots to only shards that matter. This information would help that effort. Regardless, it is useful for debugging. > Distributed Search over HTTP > > > Key: SOLR-303 > URL: https://issues.apache.org/jira/browse/SOLR-303 > Project: Solr > Issue Type: New Feature > Components: search >Reporter: Sharad Agarwal >Assignee: Yonik Seeley > Attachments: distributed.patch, distributed.patch, distributed.patch, > distributed.patch, distributed.patch, distributed.patch, distributed.patch, > distributed.patch, distributed.patch, distributed.patch, distributed.patch, > distributed.patch, distributed_add_tests_for_intended_behavior.patch, > distributed_facet_count_bugfix.patch, distributed_pjaol.patch, > fedsearch.patch, fedsearch.patch, fedsearch.patch, fedsearch.patch, > fedsearch.patch, fedsearch.patch, fedsearch.patch, fedsearch.stu.patch, > fedsearch.stu.patch > > > Searching over multiple shards and aggregating results. > Motivated by http://wiki.apache.org/solr/DistributedSearch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
