[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421323#comment-13421323 ] Markus Jelsma commented on SOLR-1781: - One of the nodes ended up with two index directories today. Later some other nodes also didn't clean up after they got restarted. > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0, 5.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch, > SOLR-1781.patch, SOLR-1781.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421557#comment-13421557 ] Mark Miller commented on SOLR-1781: --- Odd - in the first case, you are sure the indexes were there over time? For brief periods, more than once can exist...it should just end up being cleaned up when no longer in use. I can try and dig in some more, but I'll have to think a little - I don't really know where to start. My test for this issue is a test that runs a lot of instances and randomly starts and stop them. I then monitor the index directories for these 6-12 instances - I run these tests for a long time and monitor that each keeps ending up with one index dir. At some points, there are two indexes - but it always drops to one shortly later. So there still may be some hole, but I don't know where or how. If you look in the logs, perhaps you will see a bunch of "Unable to delete directory : " entries? It might be that it's trying but cannot. It might make sense to start using deleteOnExit as a last resort if delete fails - I just looked and the del call in SnapPuller does not do this. > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0, 5.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch, > SOLR-1781.patch, SOLR-1781.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421565#comment-13421565 ] Markus Jelsma commented on SOLR-1781: - Hi, I've deleted both today and yesterday indexes of more than a few hours old. Some are being removed indeed and some persist. I just restarted all nodes (introduced new fieldTypes and one field) and at least one node has three index directories. Others had two, some just one. Not a single node has a `unable to delete` string in the logs. Thanks > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0, 5.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch, > SOLR-1781.patch, SOLR-1781.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421661#comment-13421661 ] Mark Miller commented on SOLR-1781: --- Could you tell me a little bit about the operations you are performing on your cluster - perhaps I am missing some activity that is needed to trigger the remaining issue(s)? > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0, 5.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch, > SOLR-1781.patch, SOLR-1781.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421675#comment-13421675 ] Markus Jelsma commented on SOLR-1781: - Besides search and index only the occasional restart when i change some config or deploy a new build. Sometimes i need to start ZK 3.4 again because it died for some reason. Restarting Tomcat a few times in a row may be a clue here. I'll check again tomorrow if whether it's consistent. > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0, 5.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch, > SOLR-1781.patch, SOLR-1781.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421707#comment-13421707 ] Mark Miller commented on SOLR-1781: --- bq. occasional restart when i change some config You can also do that by reloading all the cores in the cluster - the latest collections api work has a collection reload command. bq. Restarting Tomcat a few times in a row may be a clue here. Yeah, if any docs where added while a node was down, it will replicate and perhaps get a new index dir. I'm restarting jetty in my tests literally hundreds of times in a row and have not seen this yet. I also just added the search thread, but so far no luck. > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0, 5.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch, > SOLR-1781.patch, SOLR-1781.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[
https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422135#comment-13422135
]
Markus Jelsma commented on SOLR-1781:
-
Hi,
I'll restart one node with two cores.
{code}
#cat cores/openindex_b/data/index.properties
#index properties
#Wed Jul 25 09:58:26 UTC 2012
index=index.20120725095644707
{code}
{code}
# du -h cores/
4.0Kcores/lib
46M cores/openindex_b/data/index.20120725095644707
404Kcores/openindex_b/data/tlog
46M cores/openindex_b/data
46M cores/openindex_b
98M cores/openindex_a/data/index.20120725095843731
124Kcores/openindex_a/data/tlog
98M cores/openindex_a/data
98M cores/openindex_a
144Mcores/
{code}
2012-07-25 10:01:09,176 WARN [solr.core.SolrCore] - [main] - : New index
directory detected: old=null
new=/opt/solr/cores/openindex_b/data/index.20120725095644707
...
2012-07-25 10:01:17,303 WARN [solr.core.SolrCore] - [main] - : New index
directory detected: old=null
new=/opt/solr/cores/openindex_a/data/index.20120725095843731
...
2012-07-25 10:01:55,016 WARN [solr.core.SolrCore] - [RecoveryThread] - : New
index directory detected:
old=/opt/solr/cores/openindex_b/data/index.20120725095644707
new=/opt/solr/cores/openindex_b/data/index.20120725100120496
...
2012-07-25 10:03:35,236 WARN [solr.core.SolrCore] - [RecoveryThread] - : New
index directory detected:
old=/opt/solr/cores/openindex_a/data/index.20120725100220706
new=/opt/solr/cores/openindex_a/data/index.20120725100321897
{code}
# du -h cores/
4.0Kcores/lib
46M cores/openindex_b/data/index.20120725095644707
404Kcores/openindex_b/data/tlog
46M cores/openindex_b/data/index.20120725100120496
91M cores/openindex_b/data
91M cores/openindex_b
98M cores/openindex_a/data/index.20120725100321897
98M cores/openindex_a/data/index.20120725100220706
124Kcores/openindex_a/data/tlog
196Mcores/openindex_a/data
196Mcores/openindex_a
287Mcores/
{code}
A few minutes later we still have multiple index directories. No updates have
been sent to the cluster during this whole scenario. Each time another
directory appears it comes with a lot of I/O, on these RAM limited machines
it's almost trashing because of the additional directory. It does not create
another directory on each restart but sometimes does, it restarted the same
machine again and now i have three dirs for each core.
I'll turn on INFO logging for the node and restart it again without deleting
the surpluss dirs. The master and slave versions are still the same.
{code}
# du -h cores/
4.0Kcores/lib
46M cores/openindex_b/data/index.20120725100813961
42M cores/openindex_b/data/index.20120725101349376
46M cores/openindex_b/data/index.20120725095644707
46M cores/openindex_b/data/index.20120725101231289
404Kcores/openindex_b/data/tlog
46M cores/openindex_b/data/index.20120725100120496
223Mcores/openindex_b/data
223Mcores/openindex_b
98M cores/openindex_a/data/index.20120725101252920
98M cores/openindex_a/data/index.20120725100220706
124Kcores/openindex_a/data/tlog
196Mcores/openindex_a/data
196Mcores/openindex_a
418Mcores/
{code}
Maybe it cannot find the current index directory on start up (in my case).
2012-07-25 10:13:36,125 WARN [solr.core.SolrCore] - [main] - : New index
directory detected: old=null
new=/opt/solr/cores/openindex_b/data/index.20120725101231289
2012-07-25 10:13:45,840 WARN [solr.core.SolrCore] - [main] - : New index
directory detected: old=null
new=/opt/solr/cores/openindex_a/data/index.20120725101252920
2012-07-25 10:15:41,393 WARN [solr.core.SolrCore] - [RecoveryThread] - : New
index directory detected:
old=/opt/solr/cores/openindex_b/data/index.20120725101231289
new=/opt/solr/cores/openindex_b/data/index.20120725101349376
2012-07-25 10:15:46,895 WARN [solr.cloud.RecoveryStrategy] - [main-EventThread]
- : Stopping recovery for core openindex_b
zkNodeName=nl2.index.openindex.io:8080_solr_openindex_b
2012-07-25 10:15:46,952 WARN [solr.core.SolrCore] - [RecoveryThread] - :
[openindex_a] Error opening new searcher. exceeded limit of
maxWarmingSearchers=1, try again later.
2012-07-25 10:15:47,298 ERROR [solr.cloud.RecoveryStrategy] - [RecoveryThread]
- : Error while trying to recover.
org.apache.solr.common.SolrException: Error opening new searcher. exceeded
limit of maxWarmingSearchers=1, try again later.
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1365)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1157)
at
org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:560)
at
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:316)
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:210)
2012-07-25 10:15:47,299 ERROR [solr.cloud.RecoveryStrategy] - [RecoveryThre
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422257#comment-13422257 ] Mark Miller commented on SOLR-1781: --- Thanks for all the info - I think perhaps this is a different issue - I'll try and look into. > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0, 5.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch, > SOLR-1781.patch, SOLR-1781.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422298#comment-13422298 ] Mark Miller commented on SOLR-1781: --- Can I get more of that log? If you are not doing updates, I'm surprised it's going into replication at all on restart... > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0, 5.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch, > SOLR-1781.patch, SOLR-1781.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422308#comment-13422308 ] Markus Jelsma commented on SOLR-1781: - Ok, I purged the logs, enabled info and started a tomcat. New indexes are created shortly after : 2012-07-25 10:13:36,125 WARN [solr.core.SolrCore] - [main] - : New index directory detected: old=null new=/opt/solr/cores/openindex_b/data/index.20120725101231289 I'll send it right now. > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0, 5.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch, > SOLR-1781.patch, SOLR-1781.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422353#comment-13422353 ] Mark Miller commented on SOLR-1781: --- Thanks - there is some sort of race here around opening new searches and what index dir appears. I'm not sure why you see it and I don't in my tests, but it's fairly clear in your logs. I think I can address it with a simple change that ensures we are picking up the right directory to remove. I'll commit that change in a moment. > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0, 5.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch, > SOLR-1781.patch, SOLR-1781.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422526#comment-13422526 ] Markus Jelsma commented on SOLR-1781: - It seems this fixes the issue. I'll double check tomorrow! > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0, 5.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch, > SOLR-1781.patch, SOLR-1781.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13423022#comment-13423022 ] Markus Jelsma commented on SOLR-1781: - Yes, the problem no longer occurs! Great work! Thanks > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0, 5.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch, > SOLR-1781.patch, SOLR-1781.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13423779#comment-13423779 ] Markus Jelsma commented on SOLR-1781: - There's still a problem with old index directories not being cleaned up and strange replication on start up. I'll write to the ML for this, the problem is likely larger than just cleaning up. > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0, 5.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch, > SOLR-1781.patch, SOLR-1781.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13416005#comment-13416005 ] Markus Jelsma commented on SOLR-1781: - This happens almost daily on our SolrCloud (trunk) test cluster, we sometimes see four surpluss index directories created in a day. > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java) >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Noble Paul > Fix For: 4.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13416169#comment-13416169 ] Mark Miller commented on SOLR-1781: --- Markus: on windows? > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Noble Paul > Fix For: 4.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13416183#comment-13416183 ] Markus Jelsma commented on SOLR-1781: - We don't have that, i should have included it in my comment. All servers run Debian GNU/Linux 6.0 and the cloud test cluster always runs with a very recent build from trunk. > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Noble Paul > Fix For: 4.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13416231#comment-13416231 ] Mark Miller commented on SOLR-1781: --- I'll look into taking care of this. > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[
https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13416255#comment-13416255
]
Mark Miller commented on SOLR-1781:
---
I'm not sure who or when, but it looks like a form of the above patch has
already been committed, except instead of:
{code}
+else if (successfulInstall) {
+ delTree(indexDir);
+}
{code}
it's
{code}
+else {
+ delTree(indexDir);
+}
{code}
I think there is a problem with both. Anyway, I can duplicate this issue easy
enough - I'll figure out the right fix.
> Replication index directories not always cleaned up
> ---
>
> Key: SOLR-1781
> URL: https://issues.apache.org/jira/browse/SOLR-1781
> Project: Solr
> Issue Type: Bug
> Components: replication (java), SolrCloud
>Affects Versions: 1.4
> Environment: Windows Server 2003 R2, Java 6b18
>Reporter: Terje Sten Bjerkseth
>Assignee: Mark Miller
> Fix For: 4.0
>
> Attachments:
> 0001-Replication-does-not-always-clean-up-old-directories.patch
>
>
> We had the same problem as someone described in
> http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e.
> A partial copy of that message:
> We're using the new replication and it's working pretty well. There's
> one detail I'd like to get some more information about.
> As the replication works, it creates versions of the index in the data
> directory. Originally we had index/, but now there are dated versions
> such as index.20100127044500/, which are the replicated versions.
> Each copy is sized in the vicinity of 65G. With our current hard drive
> it's fine to have two around, but 3 gets a little dicey. Sometimes
> we're finding that the replication doesn't always clean up after
> itself. I would like to understand this better, or to not have this
> happen. It could be a configuration issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13416309#comment-13416309 ] Mark Miller commented on SOLR-1781: --- Hmm...so I think I have a fix...but it involves reloading the core even if you do not replicate conf files. The problem is, unless you reload the core, it's very difficult to know when searchers are done with the old index. > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13416318#comment-13416318 ] Markus Jelsma commented on SOLR-1781: - Perhaps they could be cleaned up on core start or after some time has passed? > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13416330#comment-13416330 ] Mark Miller commented on SOLR-1781: --- bq. after some time has passed? This was tempting for a moment - but it feels dirty. You could just say, hey, do it on the next core restart - but that may not come for a long time. You'd really like to be able to tell when all the indexreaders on the old dir are done, but that is very difficult it turns out. The only easy way to do it is to reload the core. If you replicate configuration files, you have to reload the core anyway. I'd say it's probably the way to go. It should not be that much more expensive than reloading a searcher in the larger scheme of replication. > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13417132#comment-13417132 ] Mark Miller commented on SOLR-1781: --- Spoke to yonik about this real quick and brought up the ref counting we have for Directories as an alternative to using reload. It took a couple new methods to our DirectoryFactory, but I think I have something working using that now. > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch, > SOLR-1781.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13419032#comment-13419032 ] Markus Jelsma commented on SOLR-1781: - Hi, is the core reloading still part of this? I get a lot of firstSearcher events on a test node now and it won't get online. Going back to July 18th (before this patch) build works fine. Other nodes won't come online with a build from the 19th (after this patch). > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0, 5.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch, > SOLR-1781.patch, SOLR-1781.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13419099#comment-13419099 ] Mark Miller commented on SOLR-1781: --- No, no reload. Can you please elaborate on what not going online means. Can you share logs? > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0, 5.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch, > SOLR-1781.patch, SOLR-1781.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13419114#comment-13419114 ] Markus Jelsma commented on SOLR-1781: - The node will never respond to HTTP requests, all ZK connections time out, very high resource consumption. I'll try provide a log snippet soon. I tried running today's build several times but one specific node refuses to `come online`. Another node did well and runs today's build. I cannot attach a file to a resolved issue. Send over mail? > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0, 5.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch, > SOLR-1781.patch, SOLR-1781.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13419127#comment-13419127 ] Markus Jelsma commented on SOLR-1781: - Log sent. This node has two shards on it and executed 2x 512 warmup queries which adds up. It won't talk to ZK (tail of the log). Restarting the node with an 18th's build works fine. Did it three times today. Thanks > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0, 5.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch, > SOLR-1781.patch, SOLR-1781.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13419380#comment-13419380 ] Mark Miller commented on SOLR-1781: --- Hmm...i can't replicate this issue so far. Another change around then was updating to ZooKeeper 3.3.5 (bug fix update). I wouldnt expect that to be an issue - but are you just upgrading one node and not all of them? > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0, 5.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch, > SOLR-1781.patch, SOLR-1781.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13419385#comment-13419385 ] Markus Jelsma commented on SOLR-1781: - Strange indeed. I can/could replicate it on one machine consistently and not on others. Machines weren't upgraded at the same time to prevent cluster downtime. I'll check back monday, there are two other machines left to upgrade plus the bad node. > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0, 5.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch, > SOLR-1781.patch, SOLR-1781.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13419427#comment-13419427 ] Mark Miller commented on SOLR-1781: --- bq. Machines weren't upgraded at the same time to prevent cluster downtime. Yeah, makes sense, just wasn't sure how you went about it. I'd expect a bugfix release of zookeeper to work no problem with the previous nodes, but it's the other variable I think. They recommend upgrading with rolling restarts, so it shouldn't be the problem... > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0, 5.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch, > SOLR-1781.patch, SOLR-1781.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13420495#comment-13420495 ] Markus Jelsma commented on SOLR-1781: - The problem is resolved. The `bad` node created several new index.[0-9] directories, even with this patch, and caused high I/O. I deleted the complete data directory so also the index.properties file. It loaded its index from the other nodes and no longer created many index dirs. Thanks > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0, 5.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch, > SOLR-1781.patch, SOLR-1781.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12971296#action_12971296 ] Shawn Heisey commented on SOLR-1781: I have been having this problem too, but most of the time it's fine. After a discussion today on the mailing list, I believe it is related to a replication initiated by swapping cores on the master. > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java) >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Noble Paul > Fix For: Next > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
[ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13586918#comment-13586918 ] zhuojunjian commented on SOLR-1781: --- hi sure this issue is fixed in solr4.0 ? we find the issue in solr4.0 in our enviroment . I have created a new JIRA "SOLR-4506". > Replication index directories not always cleaned up > --- > > Key: SOLR-1781 > URL: https://issues.apache.org/jira/browse/SOLR-1781 > Project: Solr > Issue Type: Bug > Components: replication (java), SolrCloud >Affects Versions: 1.4 > Environment: Windows Server 2003 R2, Java 6b18 >Reporter: Terje Sten Bjerkseth >Assignee: Mark Miller > Fix For: 4.0-BETA, 5.0 > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch, > SOLR-1781.patch, SOLR-1781.patch > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > We're using the new replication and it's working pretty well. There's > one detail I'd like to get some more information about. > As the replication works, it creates versions of the index in the data > directory. Originally we had index/, but now there are dated versions > such as index.20100127044500/, which are the replicated versions. > Each copy is sized in the vicinity of 65G. With our current hard drive > it's fine to have two around, but 3 gets a little dicey. Sometimes > we're finding that the replication doesn't always clean up after > itself. I would like to understand this better, or to not have this > happen. It could be a configuration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
My test is not doing much searching - mostly indexing. Add try adding a more constant search thread and see if that helps. On Tue, Jul 24, 2012 at 3:17 PM, Markus Jelsma (JIRA) wrote: > > [ > https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421675#comment-13421675] > > Markus Jelsma commented on SOLR-1781: > - > > Besides search and index only the occasional restart when i change some > config or deploy a new build. Sometimes i need to start ZK 3.4 again > because it died for some reason. Restarting Tomcat a few times in a row may > be a clue here. I'll check again tomorrow if whether it's consistent. > > > Replication index directories not always cleaned up > > --- > > > > Key: SOLR-1781 > > URL: https://issues.apache.org/jira/browse/SOLR-1781 > > Project: Solr > > Issue Type: Bug > > Components: replication (java), SolrCloud > >Affects Versions: 1.4 > > Environment: Windows Server 2003 R2, Java 6b18 > >Reporter: Terje Sten Bjerkseth > >Assignee: Mark Miller > > Fix For: 4.0, 5.0 > > > > Attachments: > 0001-Replication-does-not-always-clean-up-old-directories.patch, > SOLR-1781.patch, SOLR-1781.patch > > > > > > We had the same problem as someone described in > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222a518d-ddf5-4fc8-a02a-74d4f232b...@snooth.com%3e. > A partial copy of that message: > > We're using the new replication and it's working pretty well. There's > > one detail I'd like to get some more information about. > > As the replication works, it creates versions of the index in the data > > directory. Originally we had index/, but now there are dated versions > > such as index.20100127044500/, which are the replicated versions. > > Each copy is sized in the vicinity of 65G. With our current hard drive > > it's fine to have two around, but 3 gets a little dicey. Sometimes > > we're finding that the replication doesn't always clean up after > > itself. I would like to understand this better, or to not have this > > happen. It could be a configuration issue. > > -- > This message is automatically generated by JIRA. > If you think it was sent incorrectly, please contact your JIRA > administrators: > https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa > For more information on JIRA, see: http://www.atlassian.com/software/jira > > > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > > -- - Mark http://www.lucidimagination.com
