[jira] Commented: (LUCENE-475) RAMDirectory(Directory dir, boolean closeDir) constructor uses memory inefficiently.
[ http://issues.apache.org/jira/browse/LUCENE-475?page=comments#action_12359122 ] Alexey Panchenko commented on LUCENE-475: - byte[] buf = new byte[READ_BUFFER_SIZE]; should be moved before the for-loop, so the single buffer can be used to load all the files. > RAMDirectory(Directory dir, boolean closeDir) constructor uses memory > inefficiently. > -- > > Key: LUCENE-475 > URL: http://issues.apache.org/jira/browse/LUCENE-475 > Project: Lucene - Java > Type: Improvement > Components: Store > Reporter: Volodymyr Bychkoviak > Attachments: RamDirectory.diff > > recently I found that RAMDirectory(Directory dir, boolean closeDir) > constructor uses memory inefficiently. > files from source index are read entirely intro memory as single byte array > which is after all is thrown away. And if I want to load my 200M optimized, > compound format index to memory for faster search I should give JVM at least > 400Mb memory limit. For larger indexes this can be an issue. > I've attached patch how to solve this problem. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: class org.apache.lucene.index.TermInfosTest
On Dec 1, 2005, at 4:55 PM, Bernhard Messer wrote: just found a class "org.apache.lucene.index.TermInfosTest" within the src/test directory. It seems to be a relict from previous days. It doesn't run without a resource file "words.txt" and is no JUnit- Test. I would like to delete if there is nobody raising it's hand to stop me. There are several Test*.java files that are main() programs that don't get run from the build process. I'm not sure if anyone uses these at all anymore. They were, I believe, the early "test cases" before JUnit got adopted. I suspect that all of these files could be removed, actually. Erik - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: org.apache.lucene.index.store.FSDirectoryTest not runnning with "ant test"
On Dec 1, 2005, at 5:03 PM, Bernhard Messer wrote: do you have any idea why "org.apache.lucene.index.store.FSDirectoryTest" is not running when "ant test" is called. As far as i understand the common-build.xml, all testcases where the classname starts or ends with "Test" should run. Because we have several files named *Test.java that are not actually TestCase's, build.xml overrides the common-build.xml pattern to only use Test*.java rather than both. And unfortunately is too retarded to avoid running non-TestCase classes. I've just renamed FSDirectoryTest, as well as StringHelperTest. Thanks for alerting us! Erik - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Commented: (LUCENE-475) RAMDirectory(Directory dir, boolean closeDir) constructor uses memory inefficiently.
[ http://issues.apache.org/jira/browse/LUCENE-475?page=comments#action_12359098 ] Doug Cutting commented on LUCENE-475: - Why not just use BufferedIndexOutput.BUFFER_SIZE, a constant declared elsewhere and already used for all RAMDirectory buffers? That way you're not introducing any new constants and disturbing the constant averse... And I disagree that the default behaviour should be as before: there's no good reason to buffer the entire file. That was a bug that this patch fixes. > RAMDirectory(Directory dir, boolean closeDir) constructor uses memory > inefficiently. > -- > > Key: LUCENE-475 > URL: http://issues.apache.org/jira/browse/LUCENE-475 > Project: Lucene - Java > Type: Improvement > Components: Store > Reporter: Volodymyr Bychkoviak > Attachments: RamDirectory.diff > > recently I found that RAMDirectory(Directory dir, boolean closeDir) > constructor uses memory inefficiently. > files from source index are read entirely intro memory as single byte array > which is after all is thrown away. And if I want to load my 200M optimized, > compound format index to memory for faster search I should give JVM at least > 400Mb memory limit. For larger indexes this can be an issue. > I've attached patch how to solve this problem. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Commented: (LUCENE-475) RAMDirectory(Directory dir, boolean closeDir) constructor uses memory inefficiently.
[ http://issues.apache.org/jira/browse/LUCENE-475?page=comments#action_12359097 ] Hoss Man commented on LUCENE-475: - Using different buffer sizes may not make a big difference in your performacne tests -- but that doesn't mean it won't mke a differnece for other people running with different heap sizees, or on different hardware. As i understand it, the point of this patch is to give people the ability to use memory more efficiently, and I applaud that -- but why not go all out and let users control he buffer size so they can tune it to their hearts content? > RAMDirectory(Directory dir, boolean closeDir) constructor uses memory > inefficiently. > -- > > Key: LUCENE-475 > URL: http://issues.apache.org/jira/browse/LUCENE-475 > Project: Lucene - Java > Type: Improvement > Components: Store > Reporter: Volodymyr Bychkoviak > Attachments: RamDirectory.diff > > recently I found that RAMDirectory(Directory dir, boolean closeDir) > constructor uses memory inefficiently. > files from source index are read entirely intro memory as single byte array > which is after all is thrown away. And if I want to load my 200M optimized, > compound format index to memory for faster search I should give JVM at least > 400Mb memory limit. For larger indexes this can be an issue. > I've attached patch how to solve this problem. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
org.apache.lucene.index.store.FSDirectoryTest not runnning with "ant test"
Erik, do you have any idea why "org.apache.lucene.index.store.FSDirectoryTest" is not running when "ant test" is called. As far as i understand the common-build.xml, all testcases where the classname starts or ends with "Test" should run. thanks Bernhard - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
class org.apache.lucene.index.TermInfosTest
just found a class "org.apache.lucene.index.TermInfosTest" within the src/test directory. It seems to be a relict from previous days. It doesn't run without a resource file "words.txt" and is no JUnit-Test. I would like to delete if there is nobody raising it's hand to stop me. regards Bernhard - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Closed: (LUCENE-288) [patch] better support gcj compilation
[ http://issues.apache.org/jira/browse/LUCENE-288?page=all ] Erik Hatcher closed LUCENE-288: --- > [patch] better support gcj compilation > -- > > Key: LUCENE-288 > URL: http://issues.apache.org/jira/browse/LUCENE-288 > Project: Lucene - Java > Type: Bug > Components: Search > Versions: 1.4 > Environment: Operating System: All > Platform: Other > Reporter: Andi Vajda > Attachments: 15411.txt > > In order to workaround http://gcc.gnu.org/bugzilla/show_bug.cgi?id=15411 the > attached patch is necessary. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Resolved: (LUCENE-288) [patch] better support gcj compilation
[ http://issues.apache.org/jira/browse/LUCENE-288?page=all ] Erik Hatcher resolved LUCENE-288: - Resolution: Fixed Assign To: (was: Lucene Developers) applied with the recommended comments > [patch] better support gcj compilation > -- > > Key: LUCENE-288 > URL: http://issues.apache.org/jira/browse/LUCENE-288 > Project: Lucene - Java > Type: Bug > Components: Search > Versions: 1.4 > Environment: Operating System: All > Platform: Other > Reporter: Andi Vajda > Attachments: 15411.txt > > In order to workaround http://gcc.gnu.org/bugzilla/show_bug.cgi?id=15411 the > attached patch is necessary. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Commented: (LUCENE-475) RAMDirectory(Directory dir, boolean closeDir) constructor uses memory inefficiently.
[ http://issues.apache.org/jira/browse/LUCENE-475?page=comments#action_12359083 ] Bernhard Messer commented on LUCENE-475: I like the patch and find it very helpful if one tries to load larger indices into RAMDirectory. Hoss Man, why do you would like to have a new constructor to adjust the internal buffer size. I do not see any reason to make the buffersize configurable from outside. The tests i made with different sizes didn't show any difference on performace or disk usage. The new implementation would be similar to BufferedIndexOutput where the internal buffer size couldn't be changed either. Do i miss something ? > RAMDirectory(Directory dir, boolean closeDir) constructor uses memory > inefficiently. > -- > > Key: LUCENE-475 > URL: http://issues.apache.org/jira/browse/LUCENE-475 > Project: Lucene - Java > Type: Improvement > Components: Store > Reporter: Volodymyr Bychkoviak > Attachments: RamDirectory.diff > > recently I found that RAMDirectory(Directory dir, boolean closeDir) > constructor uses memory inefficiently. > files from source index are read entirely intro memory as single byte array > which is after all is thrown away. And if I want to load my 200M optimized, > compound format index to memory for faster search I should give JVM at least > 400Mb memory limit. For larger indexes this can be an issue. > I've attached patch how to solve this problem. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Commented: (LUCENE-475) RAMDirectory(Directory dir, boolean closeDir) constructor uses memory inefficiently.
[ http://issues.apache.org/jira/browse/LUCENE-475?page=comments#action_12359068 ] Hoss Man commented on LUCENE-475: - I'm no expert on File I/O, so i don't know if this is/isn't a good change to make, but i deplore constants: If this patch is a good idea, then i'd like to suggest that instead of a static constant for the buffer size, a new constructor arg be added specifying the buffer size, with the default (ie: no value, or "0" specified) being to use the old behavior (ie: a buffer the same size as the file) > RAMDirectory(Directory dir, boolean closeDir) constructor uses memory > inefficiently. > -- > > Key: LUCENE-475 > URL: http://issues.apache.org/jira/browse/LUCENE-475 > Project: Lucene - Java > Type: Improvement > Components: Store > Reporter: Volodymyr Bychkoviak > Attachments: RamDirectory.diff > > recently I found that RAMDirectory(Directory dir, boolean closeDir) > constructor uses memory inefficiently. > files from source index are read entirely intro memory as single byte array > which is after all is thrown away. And if I want to load my 200M optimized, > compound format index to memory for faster search I should give JVM at least > 400Mb memory limit. For larger indexes this can be an issue. > I've attached patch how to solve this problem. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: open source YourKit licence
I'll propose that to them. On 12/1/05, Doug Cutting <[EMAIL PROTECTED]> wrote: > Yonik Seeley wrote: > > a) do any other committers want a license, and > > Why not just include all committer names? > > > b) would we be willing to put their logo somewhere in exchange? > > Perhaps we should reserve that until we find that Lucene has been > significantly improved by YourKit. > > Doug -Yonik Now hiring -- http://forms.cnet.com/slink?231706 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: open source YourKit licence
Yonik Seeley wrote: a) do any other committers want a license, and Why not just include all committer names? b) would we be willing to put their logo somewhere in exchange? Perhaps we should reserve that until we find that Lucene has been significantly improved by YourKit. Doug - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Commented: (LUCENE-475) RAMDirectory(Directory dir, boolean closeDir) constructor uses memory inefficiently.
[ http://issues.apache.org/jira/browse/LUCENE-475?page=comments#action_12359036 ] Volodymyr Bychkoviak commented on LUCENE-475: - Oh, I didn't described my solution... Solution is simle: use 1k buffer to copy files from source index to ram files > RAMDirectory(Directory dir, boolean closeDir) constructor uses memory > inefficiently. > -- > > Key: LUCENE-475 > URL: http://issues.apache.org/jira/browse/LUCENE-475 > Project: Lucene - Java > Type: Improvement > Components: Store > Reporter: Volodymyr Bychkoviak > Attachments: RamDirectory.diff > > recently I found that RAMDirectory(Directory dir, boolean closeDir) > constructor uses memory inefficiently. > files from source index are read entirely intro memory as single byte array > which is after all is thrown away. And if I want to load my 200M optimized, > compound format index to memory for faster search I should give JVM at least > 400Mb memory limit. For larger indexes this can be an issue. > I've attached patch how to solve this problem. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Updated: (LUCENE-475) RAMDirectory(Directory dir, boolean closeDir) constructor uses memory inefficiently.
[ http://issues.apache.org/jira/browse/LUCENE-475?page=all ] Volodymyr Bychkoviak updated LUCENE-475: Attachment: RamDirectory.diff > RAMDirectory(Directory dir, boolean closeDir) constructor uses memory > inefficiently. > -- > > Key: LUCENE-475 > URL: http://issues.apache.org/jira/browse/LUCENE-475 > Project: Lucene - Java > Type: Improvement > Components: Store > Reporter: Volodymyr Bychkoviak > Attachments: RamDirectory.diff > > recently I found that RAMDirectory(Directory dir, boolean closeDir) > constructor uses memory inefficiently. > files from source index are read entirely intro memory as single byte array > which is after all is thrown away. And if I want to load my 200M optimized, > compound format index to memory for faster search I should give JVM at least > 400Mb memory limit. For larger indexes this can be an issue. > I've attached patch how to solve this problem. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Created: (LUCENE-475) RAMDirectory(Directory dir, boolean closeDir) constructor uses memory inefficiently.
RAMDirectory(Directory dir, boolean closeDir) constructor uses memory inefficiently. -- Key: LUCENE-475 URL: http://issues.apache.org/jira/browse/LUCENE-475 Project: Lucene - Java Type: Improvement Components: Store Reporter: Volodymyr Bychkoviak recently I found that RAMDirectory(Directory dir, boolean closeDir) constructor uses memory inefficiently. files from source index are read entirely intro memory as single byte array which is after all is thrown away. And if I want to load my 200M optimized, compound format index to memory for faster search I should give JVM at least 400Mb memory limit. For larger indexes this can be an issue. I've attached patch how to solve this problem. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
