[jira] Commented: (SOLR-1144) replication hang
[
https://issues.apache.org/jira/browse/SOLR-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12884644#action_12884644
]
Toby Cole commented on SOLR-1144:
-
Just over a year since it was first spotted, I'm consistently getting the same
symptoms as this bug.
We've got a single master, with two slaves polling it, both slaves have stalled
at exactly the same point in the replication.
Here's the relevent section of the replication handler's 'details' response:
Node A
{code:xml}
str name=numFilesDownloaded18/str
str name=replicationStartTimeFri Jul 02 10:40:00 BST 2010/str
str name=timeElapsed6683s/str
str name=currentFile_9du.prx/str
str name=currentFileSize8.17 MB/str
str name=currentFileSizeDownloaded8.17 MB/str
str name=currentFileSizePercent100.0/str
str name=bytesDownloaded40.55 MB/str
str name=totalPercent0.0/str
str name=timeRemaining8290722s/str
str name=downloadSpeed6.21 KB/str
{code}
Node B
{code:xml}
str name=numFilesDownloaded18/str
str name=replicationStartTimeFri Jul 02 10:40:00 BST 2010/str
str name=timeElapsed6752s/str
str name=currentFile_9du.prx/str
str name=currentFileSize8.17 MB/str
str name=currentFileSizeDownloaded8.17 MB/str
str name=currentFileSizePercent100.0/str
str name=bytesDownloaded40.55 MB/str
str name=totalPercent0.0/str
str name=timeRemaining8376322s/str
str name=downloadSpeed6.15 KB/str
{code}
replication hang
Key: SOLR-1144
URL: https://issues.apache.org/jira/browse/SOLR-1144
Project: Solr
Issue Type: Bug
Reporter: Yonik Seeley
Assignee: Noble Paul
Fix For: 1.4
It seems that replication can sometimes hang.
http://www.lucidimagination.com/search/document/403305a3fda18599
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Updated: (SOLR-1144) replication hang
[ https://issues.apache.org/jira/browse/SOLR-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toby Cole updated SOLR-1144: Attachment: stacktrace-master.txt stacktrace-slave-1.txt stacktrace-slave-2.txt Adding stacktraces for both slave instances and the master instance. replication hang Key: SOLR-1144 URL: https://issues.apache.org/jira/browse/SOLR-1144 Project: Solr Issue Type: Bug Reporter: Yonik Seeley Assignee: Noble Paul Fix For: 1.4 Attachments: stacktrace-master.txt, stacktrace-slave-1.txt, stacktrace-slave-2.txt It seems that replication can sometimes hang. http://www.lucidimagination.com/search/document/403305a3fda18599 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Issue Comment Edited: (SOLR-1144) replication hang
[ https://issues.apache.org/jira/browse/SOLR-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12884672#action_12884672 ] Toby Cole edited comment on SOLR-1144 at 7/2/10 9:37 AM: - Adding stacktraces for both slave instances and the master instance. These stack traces are from a reproduction of the original problem, so the timestamps will not matchup with the XML from the replication-handler previously posted. was (Author: tub): Adding stacktraces for both slave instances and the master instance. replication hang Key: SOLR-1144 URL: https://issues.apache.org/jira/browse/SOLR-1144 Project: Solr Issue Type: Bug Reporter: Yonik Seeley Assignee: Noble Paul Fix For: 1.4 Attachments: stacktrace-master.txt, stacktrace-slave-1.txt, stacktrace-slave-2.txt It seems that replication can sometimes hang. http://www.lucidimagination.com/search/document/403305a3fda18599 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (SOLR-1144) replication hang
[ https://issues.apache.org/jira/browse/SOLR-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12884693#action_12884693 ] Toby Cole commented on SOLR-1144: - Oh yes, should have mentioned... we're already on Solr 1.4.1 in production as of yesterday (we don't hang about y'know ;) ). replication hang Key: SOLR-1144 URL: https://issues.apache.org/jira/browse/SOLR-1144 Project: Solr Issue Type: Bug Reporter: Yonik Seeley Assignee: Noble Paul Fix For: 1.4 Attachments: stacktrace-master.txt, stacktrace-slave-1.txt, stacktrace-slave-2.txt It seems that replication can sometimes hang. http://www.lucidimagination.com/search/document/403305a3fda18599 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (SOLR-1144) replication hang
[ https://issues.apache.org/jira/browse/SOLR-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12884719#action_12884719 ] Toby Cole commented on SOLR-1144: - I know exactly why the line numbers would be off. I just remembered we're using a custom war package so we can add our own plugins in (yes, I know we can use solr.home/lib, but we've not got round to that yet). The only classes we're overriding from solr are ConcurrentLRUCache and FastLRUCache. This was from pre solr 1.4, when the cache implementations were slowing faceting right down. I have a feeling if I remove those overridden classes and use the new (bug-free) ones, the hang may stop. I'll give it a go now, sorry in advance if it was my oversight that is causing this bug to re-appear. T replication hang Key: SOLR-1144 URL: https://issues.apache.org/jira/browse/SOLR-1144 Project: Solr Issue Type: Bug Reporter: Yonik Seeley Assignee: Noble Paul Fix For: 1.4 Attachments: stacktrace-master.txt, stacktrace-slave-1.txt, stacktrace-slave-2.txt It seems that replication can sometimes hang. http://www.lucidimagination.com/search/document/403305a3fda18599 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (SOLR-711) SimpleFacets: Performance Boost for Tokenized Fields for smaller DocSet using Term Vectors
[
https://issues.apache.org/jira/browse/SOLR-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12623950#action_12623950
]
Toby Cole commented on SOLR-711:
We've seen this problem with our dataset, we have around 10m small records and
were trying to facet on several multi-valued strings. Two of which had over 40k
unique values (around 10 values per record).
If we can come up with a plan I don't mind volunteering to implement it.
SimpleFacets: Performance Boost for Tokenized Fields for smaller DocSet using
Term Vectors
--
Key: SOLR-711
URL: https://issues.apache.org/jira/browse/SOLR-711
Project: Solr
Issue Type: Improvement
Components: search
Affects Versions: 1.3
Reporter: Fuad Efendi
Fix For: 1.4
Original Estimate: 1680h
Remaining Estimate: 1680h
From
[http://www.nabble.com/SimpleFacets%3A-Performance-Boost-for-Tokenized-Fields-td19033760.html]:
Scenario:
- 10,000,000 documents in the index;
- 5-10 terms per document;
- 200,000 unique terms for a tokenized field.
_Obviously calculating sizes of 200,000 intersections with FilterCache is 100
times slower than traversing 10 - 20,000 documents for smaller DocSets and
counting frequencies of Terms._
Not applicable if size of DocSet is close to total number of unique tokens
(200,000 in our scenario).
See SimpleFacets.java:
{code}
public NamedList getFacetTermEnumCounts(
SolrIndexSearcher searcher,
DocSet docs, ...
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
