[jira] [Updated] (SOLR-2279) Add a MockDirectoryFactory (or similar) for Solr tests
[ https://issues.apache.org/jira/browse/SOLR-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir updated SOLR-2279: -- Fix Version/s: 3.3 > Add a MockDirectoryFactory (or similar) for Solr tests > -- > > Key: SOLR-2279 > URL: https://issues.apache.org/jira/browse/SOLR-2279 > Project: Solr > Issue Type: Test > Components: Build >Reporter: Robert Muir >Assignee: Robert Muir > Fix For: 3.3, 4.0 > > Attachments: SOLR-2279.patch, SOLR-2279.patch, SOLR-2279.patch > > > Currently, all Lucene tests open directories with newDirectory() [and > soon-to-be added newFSDirectory() which always ensures the directory returned > is an FSDir subclass, see LUCENE-2804 for this]. Additionally the directory > is wrapped with MockDirectoryWrapper. > This has a number of advantages: > * By default the directory implementation is random, but you can easily > specify a specific impl e.g. -Dtests.directory=MMapDirectory. When proposing > a change to one of our directory implementations, we can run all tests with > it this way... it would be good for Solr tests to respect this too. > * The test framework (LuceneTestCase before/afterclass) ensures that these > directories are properly closed, if not, it causes the test to fail with a > stacktrace of where you > first opened the directory. > * MockDirectoryWrapper.close() then ensures that there are no resource leaks > by default, when you open a file they save the stacktrace of where you opened > it from. If you try to close the directory without say, closing an > IndexReader, it fails with the stacktrace of where you opened the reader > from. This is helpful for tracking down resource leaks. Currently Solr warns > if it cannot delete its test temporary directory, but this is better since > you know exactly where the resource leak came from. This can be disabled with > an optional setter which we should probably expose for some tests that have > known leaks like SpellCheck. > * MockDirectoryWrapper enforce consistent test behavior on any operating > system, as it won't be dependent on the return value of FSDirectory.open > * MockDirectoryWrapper has a number of other checks and features, such as > simulating a crash, simulating disk full, emulating windows (where you can't > delete open files), etc. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-2279) Add a MockDirectoryFactory (or similar) for Solr tests
[
https://issues.apache.org/jira/browse/SOLR-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir updated SOLR-2279:
--
Attachment: SOLR-2279.patch
attached is a committable patch, changes from previous:
* added to uima contrib too
* disabled for the two problematic tests.
I think its ok for now that we disable this factory for two tests like such in
setUp():
{noformat}
// TODO: fix this test to use MockDirectoryFactory
System.clearProperty("solr.directoryFactory");
{noformat}
Hopefully we can fix these tests in the future, but for now this is a good
improvement in test coverage. I'll test on windows now, and commit this as a
first step if everything is ok. I'll keep the issue open because I think we
want to fix those 2 tests.
Also, I was surprised to find no problems with the spellchecker, but the reason
for this is that it doesnt use the DirectoryFactory in solr, instead just
FSDirectory.open! (I wonder if this should be improved separately?)
> Add a MockDirectoryFactory (or similar) for Solr tests
> --
>
> Key: SOLR-2279
> URL: https://issues.apache.org/jira/browse/SOLR-2279
> Project: Solr
> Issue Type: Test
> Components: Build
>Reporter: Robert Muir
>Assignee: Robert Muir
> Fix For: 4.0
>
> Attachments: SOLR-2279.patch, SOLR-2279.patch, SOLR-2279.patch
>
>
> Currently, all Lucene tests open directories with newDirectory() [and
> soon-to-be added newFSDirectory() which always ensures the directory returned
> is an FSDir subclass, see LUCENE-2804 for this]. Additionally the directory
> is wrapped with MockDirectoryWrapper.
> This has a number of advantages:
> * By default the directory implementation is random, but you can easily
> specify a specific impl e.g. -Dtests.directory=MMapDirectory. When proposing
> a change to one of our directory implementations, we can run all tests with
> it this way... it would be good for Solr tests to respect this too.
> * The test framework (LuceneTestCase before/afterclass) ensures that these
> directories are properly closed, if not, it causes the test to fail with a
> stacktrace of where you
> first opened the directory.
> * MockDirectoryWrapper.close() then ensures that there are no resource leaks
> by default, when you open a file they save the stacktrace of where you opened
> it from. If you try to close the directory without say, closing an
> IndexReader, it fails with the stacktrace of where you opened the reader
> from. This is helpful for tracking down resource leaks. Currently Solr warns
> if it cannot delete its test temporary directory, but this is better since
> you know exactly where the resource leak came from. This can be disabled with
> an optional setter which we should probably expose for some tests that have
> known leaks like SpellCheck.
> * MockDirectoryWrapper enforce consistent test behavior on any operating
> system, as it won't be dependent on the return value of FSDirectory.open
> * MockDirectoryWrapper has a number of other checks and features, such as
> simulating a crash, simulating disk full, emulating windows (where you can't
> delete open files), etc.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-2279) Add a MockDirectoryFactory (or similar) for Solr tests
[ https://issues.apache.org/jira/browse/SOLR-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir updated SOLR-2279: -- Attachment: SOLR-2279.patch updated patch: I really want to be able to track file handles in Solr's tests, so I added a hack to avoid the issue of Directory.close() never being called. A couple tests fail using MockDirectoryWrapperFactory, still trying to track down why this is. > Add a MockDirectoryFactory (or similar) for Solr tests > -- > > Key: SOLR-2279 > URL: https://issues.apache.org/jira/browse/SOLR-2279 > Project: Solr > Issue Type: Test > Components: Build >Reporter: Robert Muir > Fix For: 4.0 > > Attachments: SOLR-2279.patch, SOLR-2279.patch > > > Currently, all Lucene tests open directories with newDirectory() [and > soon-to-be added newFSDirectory() which always ensures the directory returned > is an FSDir subclass, see LUCENE-2804 for this]. Additionally the directory > is wrapped with MockDirectoryWrapper. > This has a number of advantages: > * By default the directory implementation is random, but you can easily > specify a specific impl e.g. -Dtests.directory=MMapDirectory. When proposing > a change to one of our directory implementations, we can run all tests with > it this way... it would be good for Solr tests to respect this too. > * The test framework (LuceneTestCase before/afterclass) ensures that these > directories are properly closed, if not, it causes the test to fail with a > stacktrace of where you > first opened the directory. > * MockDirectoryWrapper.close() then ensures that there are no resource leaks > by default, when you open a file they save the stacktrace of where you opened > it from. If you try to close the directory without say, closing an > IndexReader, it fails with the stacktrace of where you opened the reader > from. This is helpful for tracking down resource leaks. Currently Solr warns > if it cannot delete its test temporary directory, but this is better since > you know exactly where the resource leak came from. This can be disabled with > an optional setter which we should probably expose for some tests that have > known leaks like SpellCheck. > * MockDirectoryWrapper enforce consistent test behavior on any operating > system, as it won't be dependent on the return value of FSDirectory.open > * MockDirectoryWrapper has a number of other checks and features, such as > simulating a crash, simulating disk full, emulating windows (where you can't > delete open files), etc. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Updated: (SOLR-2279) Add a MockDirectoryFactory (or similar) for Solr tests
[ https://issues.apache.org/jira/browse/SOLR-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir updated SOLR-2279: -- Fix Version/s: (was: 3.1) moving out.. i don't see myself fixing this test issue very quickly. > Add a MockDirectoryFactory (or similar) for Solr tests > -- > > Key: SOLR-2279 > URL: https://issues.apache.org/jira/browse/SOLR-2279 > Project: Solr > Issue Type: Test > Components: Build >Reporter: Robert Muir > Fix For: 4.0 > > Attachments: SOLR-2279.patch > > > Currently, all Lucene tests open directories with newDirectory() [and > soon-to-be added newFSDirectory() which always ensures the directory returned > is an FSDir subclass, see LUCENE-2804 for this]. Additionally the directory > is wrapped with MockDirectoryWrapper. > This has a number of advantages: > * By default the directory implementation is random, but you can easily > specify a specific impl e.g. -Dtests.directory=MMapDirectory. When proposing > a change to one of our directory implementations, we can run all tests with > it this way... it would be good for Solr tests to respect this too. > * The test framework (LuceneTestCase before/afterclass) ensures that these > directories are properly closed, if not, it causes the test to fail with a > stacktrace of where you > first opened the directory. > * MockDirectoryWrapper.close() then ensures that there are no resource leaks > by default, when you open a file they save the stacktrace of where you opened > it from. If you try to close the directory without say, closing an > IndexReader, it fails with the stacktrace of where you opened the reader > from. This is helpful for tracking down resource leaks. Currently Solr warns > if it cannot delete its test temporary directory, but this is better since > you know exactly where the resource leak came from. This can be disabled with > an optional setter which we should probably expose for some tests that have > known leaks like SpellCheck. > * MockDirectoryWrapper enforce consistent test behavior on any operating > system, as it won't be dependent on the return value of FSDirectory.open > * MockDirectoryWrapper has a number of other checks and features, such as > simulating a crash, simulating disk full, emulating windows (where you can't > delete open files), etc. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Updated: (SOLR-2279) Add a MockDirectoryFactory (or similar) for Solr tests
[ https://issues.apache.org/jira/browse/SOLR-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir updated SOLR-2279: -- Attachment: SOLR-2279.patch here's a patch, but many solr tests fail because as Shai mentioned, they don't close() their Directory instances. I think this is a pretty serious bug. While it may be the case that this works with our supplied directories, i think for "DirectoryFactory" to be exposed as a public extension point, we should call .close(). > Add a MockDirectoryFactory (or similar) for Solr tests > -- > > Key: SOLR-2279 > URL: https://issues.apache.org/jira/browse/SOLR-2279 > Project: Solr > Issue Type: Test > Components: Build >Reporter: Robert Muir > Fix For: 3.1, 4.0 > > Attachments: SOLR-2279.patch > > > Currently, all Lucene tests open directories with newDirectory() [and > soon-to-be added newFSDirectory() which always ensures the directory returned > is an FSDir subclass, see LUCENE-2804 for this]. Additionally the directory > is wrapped with MockDirectoryWrapper. > This has a number of advantages: > * By default the directory implementation is random, but you can easily > specify a specific impl e.g. -Dtests.directory=MMapDirectory. When proposing > a change to one of our directory implementations, we can run all tests with > it this way... it would be good for Solr tests to respect this too. > * The test framework (LuceneTestCase before/afterclass) ensures that these > directories are properly closed, if not, it causes the test to fail with a > stacktrace of where you > first opened the directory. > * MockDirectoryWrapper.close() then ensures that there are no resource leaks > by default, when you open a file they save the stacktrace of where you opened > it from. If you try to close the directory without say, closing an > IndexReader, it fails with the stacktrace of where you opened the reader > from. This is helpful for tracking down resource leaks. Currently Solr warns > if it cannot delete its test temporary directory, but this is better since > you know exactly where the resource leak came from. This can be disabled with > an optional setter which we should probably expose for some tests that have > known leaks like SpellCheck. > * MockDirectoryWrapper enforce consistent test behavior on any operating > system, as it won't be dependent on the return value of FSDirectory.open > * MockDirectoryWrapper has a number of other checks and features, such as > simulating a crash, simulating disk full, emulating windows (where you can't > delete open files), etc. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
