[jira] Commented: (LUCENE-2598) allow tests to use different Directory impls
[ https://issues.apache.org/jira/browse/LUCENE-2598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12902433#action_12902433 ] Robert Muir commented on LUCENE-2598: - the fixes to NIOFS and MMap are committed in revision 989030. On windows all tests pass with all directory impls, but the default is still RAMDirectory until at least we verify macos and linux are ok with "random". > allow tests to use different Directory impls > > > Key: LUCENE-2598 > URL: https://issues.apache.org/jira/browse/LUCENE-2598 > Project: Lucene - Java > Issue Type: Test > Components: Build >Affects Versions: 3.1, 4.0 >Reporter: Robert Muir >Assignee: Robert Muir > Fix For: 3.1, 4.0 > > Attachments: LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch, > LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch, > LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch > > > Now that all tests use MockRAMDirectory instead of RAMDirectory, they are all > picky like windows and force our tests to > close readers etc before closing the directory. > I think we should do the following: > # change new MockRAMDIrectory() in tests to .newDirectory(random) > # LuceneTestCase[J4] tracks if all dirs are closed at tearDown and also > cleans up temp dirs like solr. > # factor out the Mockish stuff from MockRAMDirectory into MockDirectoryWrapper > # allow a -Dtests.directoryImpl or simpler to specify the default Directory > to use for tests: default being "random" > i think theres a chance we might find some bugs that havent yet surfaced > because they are easier to trigger with FSDir > Furthermore, this would be beneficial to Directory-implementors as they could > run the entire testsuite against their Directory impl, just like > codec-implementors can do now. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2598) allow tests to use different Directory impls
[ https://issues.apache.org/jira/browse/LUCENE-2598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12902121#action_12902121 ] Robert Muir commented on LUCENE-2598: - Hoss, I honestly think what you describe is too complex. we need to keep the tests simple. bq. There are simplifications that can be made - eliminating the marker interfaces and just having the tests pick concrete classes for example - but ultimately a decision still has to be made about what to do if "tests.directory" conflicts with the impl the actual test says it can/can't work with. Dude not all tests respect tests.directory as-is: they dont all use newDirectory(), some instantiate RAMDirectories themselves still. I only converted the ones that would be easy (and some i should have left alone, so i hardwired them back to ramdir) honestly i think if i hardwire TestIndexWriter.testThreadInterruptDeadlock to RAMDirectory, then all tests will pass with NIOFS and MMAP too, and we can randomly choose whichever we want. and we are no worse off than before, last week all these tests ran with RAMDirectory no matter what. the crazy picky ones can stay that way. > allow tests to use different Directory impls > > > Key: LUCENE-2598 > URL: https://issues.apache.org/jira/browse/LUCENE-2598 > Project: Lucene - Java > Issue Type: Test > Components: Build >Affects Versions: 3.1, 4.0 >Reporter: Robert Muir >Assignee: Robert Muir > Fix For: 3.1, 4.0 > > Attachments: LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch, > LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch > > > Now that all tests use MockRAMDirectory instead of RAMDirectory, they are all > picky like windows and force our tests to > close readers etc before closing the directory. > I think we should do the following: > # change new MockRAMDIrectory() in tests to .newDirectory(random) > # LuceneTestCase[J4] tracks if all dirs are closed at tearDown and also > cleans up temp dirs like solr. > # factor out the Mockish stuff from MockRAMDirectory into MockDirectoryWrapper > # allow a -Dtests.directoryImpl or simpler to specify the default Directory > to use for tests: default being "random" > i think theres a chance we might find some bugs that havent yet surfaced > because they are easier to trigger with FSDir > Furthermore, this would be beneficial to Directory-implementors as they could > run the entire testsuite against their Directory impl, just like > codec-implementors can do now. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2598) allow tests to use different Directory impls
[ https://issues.apache.org/jira/browse/LUCENE-2598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12902115#action_12902115 ] Hoss Man commented on LUCENE-2598: -- Of the top of my head... What if we add some marker interfaces with no methods that the various directories would implement, along the lines of "InMemoryBasedDirectory", "NTFSFriendlyDirectory", etc Then change the "newDirectory(Random)" method to "newDirectory(Random,Class...)" Tests would specify a list of valid Directory "Classes" (either concrete or interfaces) or nothing if they expect to work with any impl. newDirectory would use reflection to expand the class/interface list from the params to all known concrete types, and then likewise for the "tests.directory" system property, and then intersect those lists of concrete directory implementations, and pick randomly from it -- if the intersection is an empty set (ie: the test said it's only valid for "FSDiskDirectory" but the user specified -Dtests.directory=RAMDirectory) then newDirectory returns null and signals that the test should skip (or fail). (most tests would just keep calling newDirectory(random) and the default run prop becomes -Dtests.directory=Directory) There are simplifications that can be made -- eliminating the marker interfaces and just having the tests pick concrete classes for example -- but ultimately a decision still has to be made about what to do if "tests.directory" conflicts with the impl the actual test says it can/can't work with. (personally; i think marker interfaces would be useful outside of the tests, as a documentation hint to users about what a given Directory subclass is capable of - and having the test take advantage of them would help verify that they are accurate, assuming we get the tests running on various platforms/filesystems) > allow tests to use different Directory impls > > > Key: LUCENE-2598 > URL: https://issues.apache.org/jira/browse/LUCENE-2598 > Project: Lucene - Java > Issue Type: Test > Components: Build >Affects Versions: 3.1, 4.0 >Reporter: Robert Muir >Assignee: Robert Muir > Fix For: 3.1, 4.0 > > Attachments: LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch, > LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch > > > Now that all tests use MockRAMDirectory instead of RAMDirectory, they are all > picky like windows and force our tests to > close readers etc before closing the directory. > I think we should do the following: > # change new MockRAMDIrectory() in tests to .newDirectory(random) > # LuceneTestCase[J4] tracks if all dirs are closed at tearDown and also > cleans up temp dirs like solr. > # factor out the Mockish stuff from MockRAMDirectory into MockDirectoryWrapper > # allow a -Dtests.directoryImpl or simpler to specify the default Directory > to use for tests: default being "random" > i think theres a chance we might find some bugs that havent yet surfaced > because they are easier to trigger with FSDir > Furthermore, this would be beneficial to Directory-implementors as they could > run the entire testsuite against their Directory impl, just like > codec-implementors can do now. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2598) allow tests to use different Directory impls
[ https://issues.apache.org/jira/browse/LUCENE-2598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12902093#action_12902093 ] Robert Muir commented on LUCENE-2598: - at the moment, all tests are passing on windows now with -Dtests.directory=FSDirectory. So I think the next step would be to default to 'random'. and we will need a thing to force certain tests like the one triggered by LUCENE-2239 to not use directories that will always make it fail: it will always fail from NIOFS or Mmap. > allow tests to use different Directory impls > > > Key: LUCENE-2598 > URL: https://issues.apache.org/jira/browse/LUCENE-2598 > Project: Lucene - Java > Issue Type: Test > Components: Build >Affects Versions: 3.1, 4.0 >Reporter: Robert Muir >Assignee: Robert Muir > Fix For: 3.1, 4.0 > > Attachments: LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch, > LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch > > > Now that all tests use MockRAMDirectory instead of RAMDirectory, they are all > picky like windows and force our tests to > close readers etc before closing the directory. > I think we should do the following: > # change new MockRAMDIrectory() in tests to .newDirectory(random) > # LuceneTestCase[J4] tracks if all dirs are closed at tearDown and also > cleans up temp dirs like solr. > # factor out the Mockish stuff from MockRAMDirectory into MockDirectoryWrapper > # allow a -Dtests.directoryImpl or simpler to specify the default Directory > to use for tests: default being "random" > i think theres a chance we might find some bugs that havent yet surfaced > because they are easier to trigger with FSDir > Furthermore, this would be beneficial to Directory-implementors as they could > run the entire testsuite against their Directory impl, just like > codec-implementors can do now. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2598) allow tests to use different Directory impls
[ https://issues.apache.org/jira/browse/LUCENE-2598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12902012#action_12902012 ] Robert Muir commented on LUCENE-2598: - bq. but it does seem like it would be useful for tests where only the setUp method is expected to create the index, I'm not against the proposal, but i'm curious just how it would be useful? bq. (new Jira?) I agree [i don't really understand the suggestion and how it would help improve tests yet], as this issue is really about improving test coverage (and as a side effect, providing some way to do basic tests against an arbitrary directory impl). but this is no substitute for real unit tests against any directory impl someone has written. first i'd really like to see the tests actually work with -Dtests.directory=FSDirectory (on windows too), this is proving to be challenging enough :) > allow tests to use different Directory impls > > > Key: LUCENE-2598 > URL: https://issues.apache.org/jira/browse/LUCENE-2598 > Project: Lucene - Java > Issue Type: Test > Components: Build >Affects Versions: 3.1, 4.0 >Reporter: Robert Muir >Assignee: Robert Muir > Fix For: 3.1, 4.0 > > Attachments: LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch, > LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch > > > Now that all tests use MockRAMDirectory instead of RAMDirectory, they are all > picky like windows and force our tests to > close readers etc before closing the directory. > I think we should do the following: > # change new MockRAMDIrectory() in tests to .newDirectory(random) > # LuceneTestCase[J4] tracks if all dirs are closed at tearDown and also > cleans up temp dirs like solr. > # factor out the Mockish stuff from MockRAMDirectory into MockDirectoryWrapper > # allow a -Dtests.directoryImpl or simpler to specify the default Directory > to use for tests: default being "random" > i think theres a chance we might find some bugs that havent yet surfaced > because they are easier to trigger with FSDir > Furthermore, this would be beneficial to Directory-implementors as they could > run the entire testsuite against their Directory impl, just like > codec-implementors can do now. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2598) allow tests to use different Directory impls
[ https://issues.apache.org/jira/browse/LUCENE-2598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12902007#action_12902007 ] Hoss Man commented on LUCENE-2598: -- bq. because most of them create a new directory for writing in setUp() etc. but it does seem like it would be useful for tests where only the setUp method is expected to create the index, and the rest of the code tested is expected to be "read only" .. at a naive level the setUp method of tests for "read only" functionality could unconditionally wrap the underlying Directory in a ReadOnlyDirectory once it's all finished ... but that seems like it could be done (for the tests that want it) independently of this. (new Jira?) > allow tests to use different Directory impls > > > Key: LUCENE-2598 > URL: https://issues.apache.org/jira/browse/LUCENE-2598 > Project: Lucene - Java > Issue Type: Test > Components: Build >Affects Versions: 3.1, 4.0 >Reporter: Robert Muir >Assignee: Robert Muir > Fix For: 3.1, 4.0 > > Attachments: LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch, > LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch > > > Now that all tests use MockRAMDirectory instead of RAMDirectory, they are all > picky like windows and force our tests to > close readers etc before closing the directory. > I think we should do the following: > # change new MockRAMDIrectory() in tests to .newDirectory(random) > # LuceneTestCase[J4] tracks if all dirs are closed at tearDown and also > cleans up temp dirs like solr. > # factor out the Mockish stuff from MockRAMDirectory into MockDirectoryWrapper > # allow a -Dtests.directoryImpl or simpler to specify the default Directory > to use for tests: default being "random" > i think theres a chance we might find some bugs that havent yet surfaced > because they are easier to trigger with FSDir > Furthermore, this would be beneficial to Directory-implementors as they could > run the entire testsuite against their Directory impl, just like > codec-implementors can do now. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2598) allow tests to use different Directory impls
[ https://issues.apache.org/jira/browse/LUCENE-2598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12901995#action_12901995 ] Shai Erera commented on LUCENE-2598: Yeah I figured this is sort of 'out of Lucene' case, but thought that I'd better mention it, in case you know of a Lucene test that might benefit from it (to simulate a crash or something). > allow tests to use different Directory impls > > > Key: LUCENE-2598 > URL: https://issues.apache.org/jira/browse/LUCENE-2598 > Project: Lucene - Java > Issue Type: Test > Components: Build >Affects Versions: 3.1, 4.0 >Reporter: Robert Muir >Assignee: Robert Muir > Fix For: 3.1, 4.0 > > Attachments: LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch, > LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch > > > Now that all tests use MockRAMDirectory instead of RAMDirectory, they are all > picky like windows and force our tests to > close readers etc before closing the directory. > I think we should do the following: > # change new MockRAMDIrectory() in tests to .newDirectory(random) > # LuceneTestCase[J4] tracks if all dirs are closed at tearDown and also > cleans up temp dirs like solr. > # factor out the Mockish stuff from MockRAMDirectory into MockDirectoryWrapper > # allow a -Dtests.directoryImpl or simpler to specify the default Directory > to use for tests: default being "random" > i think theres a chance we might find some bugs that havent yet surfaced > because they are easier to trigger with FSDir > Furthermore, this would be beneficial to Directory-implementors as they could > run the entire testsuite against their Directory impl, just like > codec-implementors can do now. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2598) allow tests to use different Directory impls
[ https://issues.apache.org/jira/browse/LUCENE-2598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12901990#action_12901990 ] Robert Muir commented on LUCENE-2598: - Shai, not sure Lucene's tests will be helpful there (they would just all fail with exceptions), because most of them create a new directory for writing in setUp() etc. > allow tests to use different Directory impls > > > Key: LUCENE-2598 > URL: https://issues.apache.org/jira/browse/LUCENE-2598 > Project: Lucene - Java > Issue Type: Test > Components: Build >Affects Versions: 3.1, 4.0 >Reporter: Robert Muir >Assignee: Robert Muir > Fix For: 3.1, 4.0 > > Attachments: LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch, > LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch > > > Now that all tests use MockRAMDirectory instead of RAMDirectory, they are all > picky like windows and force our tests to > close readers etc before closing the directory. > I think we should do the following: > # change new MockRAMDIrectory() in tests to .newDirectory(random) > # LuceneTestCase[J4] tracks if all dirs are closed at tearDown and also > cleans up temp dirs like solr. > # factor out the Mockish stuff from MockRAMDirectory into MockDirectoryWrapper > # allow a -Dtests.directoryImpl or simpler to specify the default Directory > to use for tests: default being "random" > i think theres a chance we might find some bugs that havent yet surfaced > because they are easier to trigger with FSDir > Furthermore, this would be beneficial to Directory-implementors as they could > run the entire testsuite against their Directory impl, just like > codec-implementors can do now. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2598) allow tests to use different Directory impls
[ https://issues.apache.org/jira/browse/LUCENE-2598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12901966#action_12901966 ] Shai Erera commented on LUCENE-2598: Not sure if it's interesting and/or related to this issue, but we've run into a bug in our code (not Lucene's that is) when the index was located on read-only file system. To simulate the bug in a JUnit test we wrote a ReadOnlyDirectory which throws exceptions on its 'write' methods. Do you think there are tests that are worth running w/ such a Directory impl? It might be that it's meaningful in the context of our tests only ... > allow tests to use different Directory impls > > > Key: LUCENE-2598 > URL: https://issues.apache.org/jira/browse/LUCENE-2598 > Project: Lucene - Java > Issue Type: Test > Components: Build >Affects Versions: 3.1, 4.0 >Reporter: Robert Muir >Assignee: Robert Muir > Fix For: 3.1, 4.0 > > Attachments: LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch, > LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch > > > Now that all tests use MockRAMDirectory instead of RAMDirectory, they are all > picky like windows and force our tests to > close readers etc before closing the directory. > I think we should do the following: > # change new MockRAMDIrectory() in tests to .newDirectory(random) > # LuceneTestCase[J4] tracks if all dirs are closed at tearDown and also > cleans up temp dirs like solr. > # factor out the Mockish stuff from MockRAMDirectory into MockDirectoryWrapper > # allow a -Dtests.directoryImpl or simpler to specify the default Directory > to use for tests: default being "random" > i think theres a chance we might find some bugs that havent yet surfaced > because they are easier to trigger with FSDir > Furthermore, this would be beneficial to Directory-implementors as they could > run the entire testsuite against their Directory impl, just like > codec-implementors can do now. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2598) allow tests to use different Directory impls
[ https://issues.apache.org/jira/browse/LUCENE-2598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12901881#action_12901881 ] Robert Muir commented on LUCENE-2598: - Wow, some tests still fail (not any new ones with the patch though), but previously test-core -Dtests.directory=SimpleFSDirectory took 20 minutes. For example TestIW took 1000seconds alone. With the patch, test-core is significantly faster with fsdir (3 minutes, 16 seconds), and for examples TestIW takes only 93 seconds. > allow tests to use different Directory impls > > > Key: LUCENE-2598 > URL: https://issues.apache.org/jira/browse/LUCENE-2598 > Project: Lucene - Java > Issue Type: Test > Components: Build >Affects Versions: 3.1, 4.0 >Reporter: Robert Muir >Assignee: Robert Muir > Fix For: 3.1, 4.0 > > Attachments: LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch, > LUCENE-2598.patch, LUCENE-2598.patch > > > Now that all tests use MockRAMDirectory instead of RAMDirectory, they are all > picky like windows and force our tests to > close readers etc before closing the directory. > I think we should do the following: > # change new MockRAMDIrectory() in tests to .newDirectory(random) > # LuceneTestCase[J4] tracks if all dirs are closed at tearDown and also > cleans up temp dirs like solr. > # factor out the Mockish stuff from MockRAMDirectory into MockDirectoryWrapper > # allow a -Dtests.directoryImpl or simpler to specify the default Directory > to use for tests: default being "random" > i think theres a chance we might find some bugs that havent yet surfaced > because they are easier to trigger with FSDir > Furthermore, this would be beneficial to Directory-implementors as they could > run the entire testsuite against their Directory impl, just like > codec-implementors can do now. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2598) allow tests to use different Directory impls
[ https://issues.apache.org/jira/browse/LUCENE-2598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12901626#action_12901626 ] Robert Muir commented on LUCENE-2598: - Mike, looks great. I applied the patch and less tests fail on windows for me, too. Most remaining problems with FSDir look windows specific, like not being able to overwrite open files on crash() > allow tests to use different Directory impls > > > Key: LUCENE-2598 > URL: https://issues.apache.org/jira/browse/LUCENE-2598 > Project: Lucene - Java > Issue Type: Test > Components: Build >Affects Versions: 3.1, 4.0 >Reporter: Robert Muir >Assignee: Robert Muir > Fix For: 3.1, 4.0 > > Attachments: LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch, > LUCENE-2598.patch > > > Now that all tests use MockRAMDirectory instead of RAMDirectory, they are all > picky like windows and force our tests to > close readers etc before closing the directory. > I think we should do the following: > # change new MockRAMDIrectory() in tests to .newDirectory(random) > # LuceneTestCase[J4] tracks if all dirs are closed at tearDown and also > cleans up temp dirs like solr. > # factor out the Mockish stuff from MockRAMDirectory into MockDirectoryWrapper > # allow a -Dtests.directoryImpl or simpler to specify the default Directory > to use for tests: default being "random" > i think theres a chance we might find some bugs that havent yet surfaced > because they are easier to trigger with FSDir > Furthermore, this would be beneficial to Directory-implementors as they could > run the entire testsuite against their Directory impl, just like > codec-implementors can do now. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2598) allow tests to use different Directory impls
[ https://issues.apache.org/jira/browse/LUCENE-2598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12901486#action_12901486 ] Robert Muir commented on LUCENE-2598: - Committed to r988206 (trunk). > allow tests to use different Directory impls > > > Key: LUCENE-2598 > URL: https://issues.apache.org/jira/browse/LUCENE-2598 > Project: Lucene - Java > Issue Type: Test > Components: Build >Affects Versions: 3.1, 4.0 >Reporter: Robert Muir >Assignee: Robert Muir > Fix For: 3.1, 4.0 > > Attachments: LUCENE-2598.patch, LUCENE-2598.patch, LUCENE-2598.patch > > > Now that all tests use MockRAMDirectory instead of RAMDirectory, they are all > picky like windows and force our tests to > close readers etc before closing the directory. > I think we should do the following: > # change new MockRAMDIrectory() in tests to .newDirectory(random) > # LuceneTestCase[J4] tracks if all dirs are closed at tearDown and also > cleans up temp dirs like solr. > # factor out the Mockish stuff from MockRAMDirectory into MockDirectoryWrapper > # allow a -Dtests.directoryImpl or simpler to specify the default Directory > to use for tests: default being "random" > i think theres a chance we might find some bugs that havent yet surfaced > because they are easier to trigger with FSDir > Furthermore, this would be beneficial to Directory-implementors as they could > run the entire testsuite against their Directory impl, just like > codec-implementors can do now. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2598) allow tests to use different Directory impls
[ https://issues.apache.org/jira/browse/LUCENE-2598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12897950#action_12897950 ] Robert Muir commented on LUCENE-2598: - Committed revision 984968 to trunk. I'll backport to 3x and then look at making MockDirectoryWrapper from MockRAM so we can support other directories in tests. > allow tests to use different Directory impls > > > Key: LUCENE-2598 > URL: https://issues.apache.org/jira/browse/LUCENE-2598 > Project: Lucene - Java > Issue Type: Test > Components: Build >Affects Versions: 3.1, 4.0 >Reporter: Robert Muir >Assignee: Robert Muir > Fix For: 3.1, 4.0 > > Attachments: LUCENE-2598.patch > > > Now that all tests use MockRAMDirectory instead of RAMDirectory, they are all > picky like windows and force our tests to > close readers etc before closing the directory. > I think we should do the following: > # change new MockRAMDIrectory() in tests to .newDirectory(random) > # LuceneTestCase[J4] tracks if all dirs are closed at tearDown and also > cleans up temp dirs like solr. > # factor out the Mockish stuff from MockRAMDirectory into MockDirectoryWrapper > # allow a -Dtests.directoryImpl or simpler to specify the default Directory > to use for tests: default being "random" > i think theres a chance we might find some bugs that havent yet surfaced > because they are easier to trigger with FSDir > Furthermore, this would be beneficial to Directory-implementors as they could > run the entire testsuite against their Directory impl, just like > codec-implementors can do now. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2598) allow tests to use different Directory impls
[ https://issues.apache.org/jira/browse/LUCENE-2598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12897727#action_12897727 ] Robert Muir commented on LUCENE-2598: - Yeah i think its neat too, but i guess really we abuse junit :) the first thing I am working (#1 and #2) is to add newDirectory(Random), and for LuceneTestCase[j4] to ensure all MockRAMDirectories are actually closed in tests. This is good because MockRAM checks in its close if readers etc are closed: and its pickiness is os-independent (will fail on unix too) > allow tests to use different Directory impls > > > Key: LUCENE-2598 > URL: https://issues.apache.org/jira/browse/LUCENE-2598 > Project: Lucene - Java > Issue Type: Test > Components: Build >Affects Versions: 3.1, 4.0 >Reporter: Robert Muir >Assignee: Robert Muir > Fix For: 3.1, 4.0 > > > Now that all tests use MockRAMDirectory instead of RAMDirectory, they are all > picky like windows and force our tests to > close readers etc before closing the directory. > I think we should do the following: > # change new MockRAMDIrectory() in tests to .newDirectory(random) > # LuceneTestCase[J4] tracks if all dirs are closed at tearDown and also > cleans up temp dirs like solr. > # factor out the Mockish stuff from MockRAMDirectory into MockDirectoryWrapper > # allow a -Dtests.directoryImpl or simpler to specify the default Directory > to use for tests: default being "random" > i think theres a chance we might find some bugs that havent yet surfaced > because they are easier to trigger with FSDir > Furthermore, this would be beneficial to Directory-implementors as they could > run the entire testsuite against their Directory impl, just like > codec-implementors can do now. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2598) allow tests to use different Directory impls
[ https://issues.apache.org/jira/browse/LUCENE-2598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12897724#action_12897724 ] Shai Erera commented on LUCENE-2598: Ok, that makes sense. I forgot that we create different dir per test JVM. I read about it and thought it's neat - apparently it's too basic. > allow tests to use different Directory impls > > > Key: LUCENE-2598 > URL: https://issues.apache.org/jira/browse/LUCENE-2598 > Project: Lucene - Java > Issue Type: Test > Components: Build >Affects Versions: 3.1, 4.0 >Reporter: Robert Muir >Assignee: Robert Muir > Fix For: 3.1, 4.0 > > > Now that all tests use MockRAMDirectory instead of RAMDirectory, they are all > picky like windows and force our tests to > close readers etc before closing the directory. > I think we should do the following: > # change new MockRAMDIrectory() in tests to .newDirectory(random) > # LuceneTestCase[J4] tracks if all dirs are closed at tearDown and also > cleans up temp dirs like solr. > # factor out the Mockish stuff from MockRAMDirectory into MockDirectoryWrapper > # allow a -Dtests.directoryImpl or simpler to specify the default Directory > to use for tests: default being "random" > i think theres a chance we might find some bugs that havent yet surfaced > because they are easier to trigger with FSDir > Furthermore, this would be beneficial to Directory-implementors as they could > run the entire testsuite against their Directory impl, just like > codec-implementors can do now. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2598) allow tests to use different Directory impls
[ https://issues.apache.org/jira/browse/LUCENE-2598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12897699#action_12897699 ] Robert Muir commented on LUCENE-2598: - bq. Robert, how about if we use JUnit's TemporaryFolder (http://kentbeck.github.com/junit/javadoc/latest/org/junit/rules/TemporaryFolder.html)? Hi Shai, I looked at this, but I don't think we should use it. I think we should keep a consistent API between LuceneTestCase and LuceneTestCase4J. But this TemporaryFolder is too basic for our needs and does not support our usage of LuceneTestCase4J (where the index is created in @beforeClass and destroyed in @afterClass), it only supports removing the directory after each test METHOD. Furthermore, there is no way to tell junit where to put this TemporaryFolder. Right now ensure that tests don't stomp on each others feet by giving them unique temp directories per-jvm, i would like to continue ensuring that any temporary data is placed there, not somewhere randomly on the users system. This way all data created by 'ant test' is guaranteed to be removed in 'ant clean'. > allow tests to use different Directory impls > > > Key: LUCENE-2598 > URL: https://issues.apache.org/jira/browse/LUCENE-2598 > Project: Lucene - Java > Issue Type: Test > Components: Build >Affects Versions: 3.1, 4.0 >Reporter: Robert Muir >Assignee: Robert Muir > Fix For: 3.1, 4.0 > > > Now that all tests use MockRAMDirectory instead of RAMDirectory, they are all > picky like windows and force our tests to > close readers etc before closing the directory. > I think we should do the following: > # change new MockRAMDIrectory() in tests to .newDirectory(random) > # LuceneTestCase[J4] tracks if all dirs are closed at tearDown and also > cleans up temp dirs like solr. > # factor out the Mockish stuff from MockRAMDirectory into MockDirectoryWrapper > # allow a -Dtests.directoryImpl or simpler to specify the default Directory > to use for tests: default being "random" > i think theres a chance we might find some bugs that havent yet surfaced > because they are easier to trigger with FSDir > Furthermore, this would be beneficial to Directory-implementors as they could > run the entire testsuite against their Directory impl, just like > codec-implementors can do now. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (LUCENE-2598) allow tests to use different Directory impls
[ https://issues.apache.org/jira/browse/LUCENE-2598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12897583#action_12897583 ] Shai Erera commented on LUCENE-2598: Robert, how about if we use JUnit's TemporaryFolder (http://kentbeck.github.com/junit/javadoc/latest/org/junit/rules/TemporaryFolder.html)? We can put it in LTC(J4) and have tests that wish to create on-disk resources use it. JUnit guarantees the contents of this folder will be removed when the test is finished, and if the test did not close its resources properly, it will fail. It will save us some code, nothing we cannot develop on our own, as you suggest. > allow tests to use different Directory impls > > > Key: LUCENE-2598 > URL: https://issues.apache.org/jira/browse/LUCENE-2598 > Project: Lucene - Java > Issue Type: Test > Components: Build >Affects Versions: 3.1, 4.0 >Reporter: Robert Muir >Assignee: Robert Muir > Fix For: 3.1, 4.0 > > > Now that all tests use MockRAMDirectory instead of RAMDirectory, they are all > picky like windows and force our tests to > close readers etc before closing the directory. > I think we should do the following: > # change new MockRAMDIrectory() in tests to .newDirectory(random) > # LuceneTestCase[J4] tracks if all dirs are closed at tearDown and also > cleans up temp dirs like solr. > # factor out the Mockish stuff from MockRAMDirectory into MockDirectoryWrapper > # allow a -Dtests.directoryImpl or simpler to specify the default Directory > to use for tests: default being "random" > i think theres a chance we might find some bugs that havent yet surfaced > because they are easier to trigger with FSDir > Furthermore, this would be beneficial to Directory-implementors as they could > run the entire testsuite against their Directory impl, just like > codec-implementors can do now. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
