Re: Java 1.5 was [jira] Updated: (LUCENE-600) ParallelWriter companion to ParallelReader
+1 Do you want to post it on the user list? It might also be good to put it up on the main website. Otis Gospodnetic wrote: Grant: how to poll users? How about this: http://www.quimble.com/poll/view/2156 ? If you think that's ok, we can send that to java-user tomorrow and see. Hey, how about some bets? I'll put a $10 for a beer on 1.5. Wow, $10 for a beer? That must be some pretty good beer. Either that or you live in New York City and that is a cheap beer! Anyway, I am betting it is 1.5 as well. Maybe we can get together at ApacheCon or something for one... Otis - Original Message From: Grant Ingersoll [EMAIL PROTECTED] To: java-dev@lucene.apache.org Sent: Tuesday, June 13, 2006 5:01:30 PM Subject: Re: Java 1.5 was [jira] Updated: (LUCENE-600) ParallelWriter companion to ParallelReader In addition to performance, productivity and functionality benefits, my main argument for 1.5 is that it is used by the vast majority of lucene community members. I am not so sure about this. Perhaps we should take a poll on the user list? Not even sure how that would be managed or counted, but... Everything I write is in 1.5 and I don't have time to backport. I have a significant body of code from which to extract and contribute patches that others would likely find useful. How many others are in a similar position? I definitely would prefer to make future contributions in 1.5 (even the patch we just contributed (issue 545) could have been better given 1.5, but it is fine with 1.4 as well). I tend to think if people don't want the new functionality or if it breaks their app. then they need not upgrade, or they can contribute patches against the branches for prior releases and we can support that as needed. To me, this is what major releases are about. I know that when a major release comes out that I should expect library changes that may break my code. If I don't want that pain, then I don't upgrade. On the side, not leaving valued community members behind is important. I think the pmc / committers just need to make a decision which will impact one group or the other. Chuck Grant Ingersoll wrote on 06/13/2006 03:35 AM: Well, we have our first Java 1.5 patch... Now that we have had a week or two to digest the comments, do we want to reopen the discussion? Chuck Williams (JIRA) wrote: [ http://issues.apache.org/jira/browse/LUCENE-600?page=all ] Chuck Williams updated LUCENE-600: -- Attachment: ParallelWriter.patch Patch to create and integrate ParallelWriter, Writable and TestParallelWriter -- also modifies build to use java 1.5. ParallelWriter companion to ParallelReader -- Key: LUCENE-600 URL: http://issues.apache.org/jira/browse/LUCENE-600 Project: Lucene - Java Type: Improvement Components: Index Versions: 2.1 Reporter: Chuck Williams Attachments: ParallelWriter.patch A new class ParallelWriter is provided that serves as a companion to ParallelReader. ParallelWriter meets all of the doc-id synchronization requirements of ParallelReader, subject to: 1. ParallelWriter.addDocument() is synchronized, which might have an adverse effect on performance. The writes to the sub-indexes are, however, done in parallel. 2. The application must ensure that the ParallelReader is never reopened inside ParallelWriter.addDocument(), else it might find the sub-indexes out of sync. 3. The application must deal with recovery from ParallelWriter.addDocument() exceptions. Recovery must restore the synchronization of doc-ids, e.g. by deleting any trailing document(s) in one sub-index that were not successfully added to all sub-indexes, and then optimizing all sub-indexes. A new interface, Writable, is provided to abstract IndexWriter and ParallelWriter. This is in the same spirit as the existing Searchable and Fieldable classes. This implementation uses java 1.5. The patch applies against today's svn head. All tests pass, including the new TestParallelWriter. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Grant Ingersoll Sr. Software Engineer Center for Natural Language Processing Syracuse University School of Information Studies 335 Hinds Hall Syracuse, NY 13244 http://www.cnlp.org Voice: 315-443-5484 Fax: 315-443-6886 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Java 1.5 was [jira] Updated: (LUCENE-600) ParallelWriter companion to ParallelReader
I'll just send it to java-user in a bit in order to get the answers only from Lucene users (and not peeps just passing by lucene.apache.org). Otis - Original Message From: Grant Ingersoll [EMAIL PROTECTED] To: java-dev@lucene.apache.org Sent: Friday, June 16, 2006 6:53:57 AM Subject: Re: Java 1.5 was [jira] Updated: (LUCENE-600) ParallelWriter companion to ParallelReader +1 Do you want to post it on the user list? It might also be good to put it up on the main website. Otis Gospodnetic wrote: Grant: how to poll users? How about this: http://www.quimble.com/poll/view/2156 ? If you think that's ok, we can send that to java-user tomorrow and see. Hey, how about some bets? I'll put a $10 for a beer on 1.5. Wow, $10 for a beer? That must be some pretty good beer. Either that or you live in New York City and that is a cheap beer! Anyway, I am betting it is 1.5 as well. Maybe we can get together at ApacheCon or something for one... Otis - Original Message From: Grant Ingersoll [EMAIL PROTECTED] To: java-dev@lucene.apache.org Sent: Tuesday, June 13, 2006 5:01:30 PM Subject: Re: Java 1.5 was [jira] Updated: (LUCENE-600) ParallelWriter companion to ParallelReader In addition to performance, productivity and functionality benefits, my main argument for 1.5 is that it is used by the vast majority of lucene community members. I am not so sure about this. Perhaps we should take a poll on the user list? Not even sure how that would be managed or counted, but... Everything I write is in 1.5 and I don't have time to backport. I have a significant body of code from which to extract and contribute patches that others would likely find useful. How many others are in a similar position? I definitely would prefer to make future contributions in 1.5 (even the patch we just contributed (issue 545) could have been better given 1.5, but it is fine with 1.4 as well). I tend to think if people don't want the new functionality or if it breaks their app. then they need not upgrade, or they can contribute patches against the branches for prior releases and we can support that as needed. To me, this is what major releases are about. I know that when a major release comes out that I should expect library changes that may break my code. If I don't want that pain, then I don't upgrade. On the side, not leaving valued community members behind is important. I think the pmc / committers just need to make a decision which will impact one group or the other. Chuck Grant Ingersoll wrote on 06/13/2006 03:35 AM: Well, we have our first Java 1.5 patch... Now that we have had a week or two to digest the comments, do we want to reopen the discussion? Chuck Williams (JIRA) wrote: [ http://issues.apache.org/jira/browse/LUCENE-600?page=all ] Chuck Williams updated LUCENE-600: -- Attachment: ParallelWriter.patch Patch to create and integrate ParallelWriter, Writable and TestParallelWriter -- also modifies build to use java 1.5. ParallelWriter companion to ParallelReader -- Key: LUCENE-600 URL: http://issues.apache.org/jira/browse/LUCENE-600 Project: Lucene - Java Type: Improvement Components: Index Versions: 2.1 Reporter: Chuck Williams Attachments: ParallelWriter.patch A new class ParallelWriter is provided that serves as a companion to ParallelReader. ParallelWriter meets all of the doc-id synchronization requirements of ParallelReader, subject to: 1. ParallelWriter.addDocument() is synchronized, which might have an adverse effect on performance. The writes to the sub-indexes are, however, done in parallel. 2. The application must ensure that the ParallelReader is never reopened inside ParallelWriter.addDocument(), else it might find the sub-indexes out of sync. 3. The application must deal with recovery from ParallelWriter.addDocument() exceptions. Recovery must restore the synchronization of doc-ids, e.g. by deleting any trailing document(s) in one sub-index that were not successfully added to all sub-indexes, and then optimizing all sub-indexes. A new interface, Writable, is provided to abstract IndexWriter and ParallelWriter. This is in the same spirit as the existing Searchable and Fieldable classes. This implementation uses java 1.5. The patch applies against today's svn head. All tests pass, including the new TestParallelWriter. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Grant Ingersoll Sr. Software Engineer Center for Natural
Re: Java 1.5 was [jira] Updated: (LUCENE-600) ParallelWriter companion to ParallelReader
I agree and completely understand Chuck. I'm waiting for my employer to sign and fax the CCLA for some search benchmarking code I wrote, and it uses Java 1.5 stuff. It would only be a contrib piece, not core, so it's less of a problem, but... Grant: how to poll users? How about this: http://www.quimble.com/poll/view/2156 ? If you think that's ok, we can send that to java-user tomorrow and see. Hey, how about some bets? I'll put a $10 for a beer on 1.5. Otis - Original Message From: Grant Ingersoll [EMAIL PROTECTED] To: java-dev@lucene.apache.org Sent: Tuesday, June 13, 2006 5:01:30 PM Subject: Re: Java 1.5 was [jira] Updated: (LUCENE-600) ParallelWriter companion to ParallelReader In addition to performance, productivity and functionality benefits, my main argument for 1.5 is that it is used by the vast majority of lucene community members. I am not so sure about this. Perhaps we should take a poll on the user list? Not even sure how that would be managed or counted, but... Everything I write is in 1.5 and I don't have time to backport. I have a significant body of code from which to extract and contribute patches that others would likely find useful. How many others are in a similar position? I definitely would prefer to make future contributions in 1.5 (even the patch we just contributed (issue 545) could have been better given 1.5, but it is fine with 1.4 as well). I tend to think if people don't want the new functionality or if it breaks their app. then they need not upgrade, or they can contribute patches against the branches for prior releases and we can support that as needed. To me, this is what major releases are about. I know that when a major release comes out that I should expect library changes that may break my code. If I don't want that pain, then I don't upgrade. On the side, not leaving valued community members behind is important. I think the pmc / committers just need to make a decision which will impact one group or the other. Chuck Grant Ingersoll wrote on 06/13/2006 03:35 AM: Well, we have our first Java 1.5 patch... Now that we have had a week or two to digest the comments, do we want to reopen the discussion? Chuck Williams (JIRA) wrote: [ http://issues.apache.org/jira/browse/LUCENE-600?page=all ] Chuck Williams updated LUCENE-600: -- Attachment: ParallelWriter.patch Patch to create and integrate ParallelWriter, Writable and TestParallelWriter -- also modifies build to use java 1.5. ParallelWriter companion to ParallelReader -- Key: LUCENE-600 URL: http://issues.apache.org/jira/browse/LUCENE-600 Project: Lucene - Java Type: Improvement Components: Index Versions: 2.1 Reporter: Chuck Williams Attachments: ParallelWriter.patch A new class ParallelWriter is provided that serves as a companion to ParallelReader. ParallelWriter meets all of the doc-id synchronization requirements of ParallelReader, subject to: 1. ParallelWriter.addDocument() is synchronized, which might have an adverse effect on performance. The writes to the sub-indexes are, however, done in parallel. 2. The application must ensure that the ParallelReader is never reopened inside ParallelWriter.addDocument(), else it might find the sub-indexes out of sync. 3. The application must deal with recovery from ParallelWriter.addDocument() exceptions. Recovery must restore the synchronization of doc-ids, e.g. by deleting any trailing document(s) in one sub-index that were not successfully added to all sub-indexes, and then optimizing all sub-indexes. A new interface, Writable, is provided to abstract IndexWriter and ParallelWriter. This is in the same spirit as the existing Searchable and Fieldable classes. This implementation uses java 1.5. The patch applies against today's svn head. All tests pass, including the new TestParallelWriter. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Grant Ingersoll Sr. Software Engineer Center for Natural Language Processing Syracuse University School of Information Studies 335 Hinds Hall Syracuse, NY 13244 http://www.cnlp.org Voice: 315-443-5484 Fax: 315-443-6886 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Java 1.5 was [jira] Updated: (LUCENE-600) ParallelWriter companion to ParallelReader
Well, we have our first Java 1.5 patch... Now that we have had a week or two to digest the comments, do we want to reopen the discussion? Chuck Williams (JIRA) wrote: [ http://issues.apache.org/jira/browse/LUCENE-600?page=all ] Chuck Williams updated LUCENE-600: -- Attachment: ParallelWriter.patch Patch to create and integrate ParallelWriter, Writable and TestParallelWriter -- also modifies build to use java 1.5. ParallelWriter companion to ParallelReader -- Key: LUCENE-600 URL: http://issues.apache.org/jira/browse/LUCENE-600 Project: Lucene - Java Type: Improvement Components: Index Versions: 2.1 Reporter: Chuck Williams Attachments: ParallelWriter.patch A new class ParallelWriter is provided that serves as a companion to ParallelReader. ParallelWriter meets all of the doc-id synchronization requirements of ParallelReader, subject to: 1. ParallelWriter.addDocument() is synchronized, which might have an adverse effect on performance. The writes to the sub-indexes are, however, done in parallel. 2. The application must ensure that the ParallelReader is never reopened inside ParallelWriter.addDocument(), else it might find the sub-indexes out of sync. 3. The application must deal with recovery from ParallelWriter.addDocument() exceptions. Recovery must restore the synchronization of doc-ids, e.g. by deleting any trailing document(s) in one sub-index that were not successfully added to all sub-indexes, and then optimizing all sub-indexes. A new interface, Writable, is provided to abstract IndexWriter and ParallelWriter. This is in the same spirit as the existing Searchable and Fieldable classes. This implementation uses java 1.5. The patch applies against today's svn head. All tests pass, including the new TestParallelWriter. -- Grant Ingersoll Sr. Software Engineer Center for Natural Language Processing Syracuse University School of Information Studies 335 Hinds Hall Syracuse, NY 13244 http://www.cnlp.org Voice: 315-443-5484 Fax: 315-443-6886 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Java 1.5 was [jira] Updated: (LUCENE-600) ParallelWriter companion to ParallelReader
I think the last discussion ended with the main counter-argument being lack of support by gjc. Current top of GJC News: *June 6, 2006* RMS approved the plan to use the Eclipse compiler as the new gcj front end. Work is being done on the |gcj-eclipse| branch; it can already build libgcj. This project will allow us to ship a 1.5 compiler in the relatively near future. The old |gcjx| branch and project is now dead. In addition to performance, productivity and functionality benefits, my main argument for 1.5 is that it is used by the vast majority of lucene community members. Everything I write is in 1.5 and I don't have time to backport. I have a significant body of code from which to extract and contribute patches that others would likely find useful. How many others are in a similar position? On the side, not leaving valued community members behind is important. I think the pmc / committers just need to make a decision which will impact one group or the other. Chuck Grant Ingersoll wrote on 06/13/2006 03:35 AM: Well, we have our first Java 1.5 patch... Now that we have had a week or two to digest the comments, do we want to reopen the discussion? Chuck Williams (JIRA) wrote: [ http://issues.apache.org/jira/browse/LUCENE-600?page=all ] Chuck Williams updated LUCENE-600: -- Attachment: ParallelWriter.patch Patch to create and integrate ParallelWriter, Writable and TestParallelWriter -- also modifies build to use java 1.5. ParallelWriter companion to ParallelReader -- Key: LUCENE-600 URL: http://issues.apache.org/jira/browse/LUCENE-600 Project: Lucene - Java Type: Improvement Components: Index Versions: 2.1 Reporter: Chuck Williams Attachments: ParallelWriter.patch A new class ParallelWriter is provided that serves as a companion to ParallelReader. ParallelWriter meets all of the doc-id synchronization requirements of ParallelReader, subject to: 1. ParallelWriter.addDocument() is synchronized, which might have an adverse effect on performance. The writes to the sub-indexes are, however, done in parallel. 2. The application must ensure that the ParallelReader is never reopened inside ParallelWriter.addDocument(), else it might find the sub-indexes out of sync. 3. The application must deal with recovery from ParallelWriter.addDocument() exceptions. Recovery must restore the synchronization of doc-ids, e.g. by deleting any trailing document(s) in one sub-index that were not successfully added to all sub-indexes, and then optimizing all sub-indexes. A new interface, Writable, is provided to abstract IndexWriter and ParallelWriter. This is in the same spirit as the existing Searchable and Fieldable classes. This implementation uses java 1.5. The patch applies against today's svn head. All tests pass, including the new TestParallelWriter. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
