New LuSolr trunk (was: RE: (LUCENE-2297) IndexWriter should let you optionally enable reader pooling)
Hi all, the discussion where to do the development after the merge, now gets actual: Currently a lusolr test-trunk is done as a branch inside solr (https://svn.apache.org/repos/asf/lucene/solr/branches/newtrunk). The question is, where to put the main development and how to switch, so non-developers that have checkouts of solr and/or lucene will see the change and do not send us outdated patches. I propose to do the following: - Start a new top-level project folder inside /lucene root svn folder: https://svn.apache.org/repos/asf/lucene/lusolr (please see "lusolr" as a placeholder name) and add branches, tags subfolders to it. Do not create trunk and do this together with the next step. - Move the branch from https://svn.apache.org/repos/asf/lucene/solr/branches/newtrunk to this new directory as "trunk" - For lucene flexible indexing, create a corresponding flex branch there and svn copy it from current new trunk. Merge the lucene flex changes into it. Alternatively, land flex now. Or simply do svn copy of current flex branch instead of merging (may be less work). - Do the same for possible solr branches in development - Create a tag in the lucene tags folder and in the solr tags folder with the current state of each trunk. After that delete all contents from old trunk in solr and lucene and place a readme file pointing developers to the new merged trunk folder (for both old trunks). This last step is important, else people who checkout the old trunk will soon see a very outdated view and may send us outdated patches in JIRA. When the contents of old-trunk disappear it's obvious to them what happened. If they had already some changes in their checkout, the svn client will keep the changed files as unversioned (after upgrade). The history keeps available, so it's also possible to checkout an older version from trunk using @rev or -r rev. I did a similar step with some backwards compatibility changes in lucene (add a README). Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Michael McCandless [mailto:luc...@mikemccandless.com] > Sent: Monday, March 22, 2010 11:37 AM > To: java-dev@lucene.apache.org > Subject: Re: (LUCENE-2297) IndexWriter should let you optionally enable > reader pooling > > I think we should. > > It (newtrunk) was created to test Hoss's side-by-sdie proposal, and > that approach looks to be working very well. > > Up until now we've been committing to the old trunk and then > systematically merging over to newtrunk. I think we should now flip > that, ie, commit to newtrunk and only merge back to the old trunk if > for some strange reason it's needed. > > Mike > > On Mon, Mar 22, 2010 at 6:32 AM, Uwe Schindler wrote: > > Are we now only working on newtrunk? > > > > - > > Uwe Schindler > > H.-H.-Meier-Allee 63, D-28213 Bremen > > http://www.thetaphi.de > > eMail: u...@thetaphi.de > > > >> -Original Message- > >> From: Michael McCandless (JIRA) [mailto:j...@apache.org] > >> Sent: Monday, March 22, 2010 11:22 AM > >> To: java-dev@lucene.apache.org > >> Subject: [jira] Resolved: (LUCENE-2297) IndexWriter should let you > >> optionally enable reader pooling > >> > >> > >> [ https://issues.apache.org/jira/browse/LUCENE- > >> 2297?page=com.atlassian.jira.plugin.system.issuetabpanels:all- > tabpanel > >> ] > >> > >> Michael McCandless resolved LUCENE-2297. > >> > >> > >> Resolution: Fixed > >> > >> Fixed on newtrunk. > >> > >> > IndexWriter should let you optionally enable reader pooling > >> > --- > >> > > >> > Key: LUCENE-2297 > >> > URL: https://issues.apache.org/jira/browse/LUCENE- > >> 2297 > >> > Project: Lucene - Java > >> > Issue Type: Improvement > >> >Reporter: Michael McCandless > >> >Priority: Minor > >> > Fix For: 3.1 > >> > > >> > Attachments: LUCENE-2297.patch > >> > > >> > > >> > For apps using a large index and frequently need to commit and > >> resolve deletes, the cost of opening the SegmentReaders on demand > for > >> every commit can be prohibitive. > >> > We an already pool readers (NRT does so), but, we only turn it on > if > >> NRT readers are in use. > >> > We should allow separate control. > >> > We should do this after LUCENE-2294. > >> > >> -- > >> This message is automatically generated by JIRA. > >> - > >> You can reply to this email to add a comment to the issue online. > >> > >> > >> > - > >> To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org > >> For additional commands, e-mail: java-dev-h...@lucene.apache.org > > > > > > > > - > > To unsubscribe, e-mail: java-dev-unsubscr...@lucen
Re: New LuSolr trunk (was: RE: (LUCENE-2297) IndexWriter should let you optionally enable reader pooling)
On Mar 22, 2010, at 8:27 AM, Uwe Schindler wrote: > Hi all, > > the discussion where to do the development after the merge, now gets actual: > > Currently a lusolr test-trunk is done as a branch inside solr > (https://svn.apache.org/repos/asf/lucene/solr/branches/newtrunk). The > question is, where to put the main development and how to switch, so > non-developers that have checkouts of solr and/or lucene will see the change > and do not send us outdated patches. > > I propose to do the following: > > - Start a new top-level project folder inside /lucene root svn folder: > https://svn.apache.org/repos/asf/lucene/lusolr (please see "lusolr" as a > placeholder name) and add branches, tags subfolders to it. Do not create > trunk and do this together with the next step. > - Move the branch from > https://svn.apache.org/repos/asf/lucene/solr/branches/newtrunk to this new > directory as "trunk" OK, makes sense. Frankly, I think we could just keep the name "java" for "lusolr", but "search" would work too or even something as simple as dev. > - For lucene flexible indexing, create a corresponding flex branch there and > svn copy it from current new trunk. Merge the lucene flex changes into it. > Alternatively, land flex now. Or simply do svn copy of current flex branch > instead of merging (may be less work). > - Do the same for possible solr branches in development > - Create a tag in the lucene tags folder and in the solr tags folder with the > current state of each trunk. After that delete all contents from old trunk in > solr and lucene and place a readme file pointing developers to the new merged > trunk folder (for both old trunks). This last step is important, else people > who checkout the old trunk will soon see a very outdated view and may send us > outdated patches in JIRA. When the contents of old-trunk disappear it's > obvious to them what happened. If they had already some changes in their > checkout, the svn client will keep the changed files as unversioned (after > upgrade). The history keeps available, so it's also possible to checkout an > older version from trunk using @rev or -r rev. I did a similar step with some > backwards compatibility changes in lucene (add a README). Makes sense. We can always move things again if we need to. This isn't CVS after all. - To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
Re: New LuSolr trunk (was: RE: (LUCENE-2297) IndexWriter should let you optionally enable reader pooling)
On Mar 22, 2010, at 8:27 AM, Uwe Schindler wrote: > Hi all, > > the discussion where to do the development after the merge, now gets actual: > > Currently a lusolr test-trunk is done as a branch inside solr > (https://svn.apache.org/repos/asf/lucene/solr/branches/newtrunk). The > question is, where to put the main development and how to switch, so > non-developers that have checkouts of solr and/or lucene will see the change > and do not send us outdated patches. > > I propose to do the following: > > - Start a new top-level project folder inside /lucene root svn folder: > https://svn.apache.org/repos/asf/lucene/lusolr (please see "lusolr" as a > placeholder name) and add branches, tags subfolders to it. Do not create > trunk and do this together with the next step. OK, I created https://svn.apache.org/repos/asf/lucene/dev/ and given appropriate rights. Uwe, you can now do the rest of the move. Once you've done it, let me know and I can make sure to add back the contrib rights. > - Move the branch from > https://svn.apache.org/repos/asf/lucene/solr/branches/newtrunk to this new > directory as "trunk" > - For lucene flexible indexing, create a corresponding flex branch there and > svn copy it from current new trunk. Merge the lucene flex changes into it. > Alternatively, land flex now. Or simply do svn copy of current flex branch > instead of merging (may be less work). > - Do the same for possible solr branches in development > - Create a tag in the lucene tags folder and in the solr tags folder with the > current state of each trunk. After that delete all contents from old trunk in > solr and lucene and place a readme file pointing developers to the new merged > trunk folder (for both old trunks). This last step is important, else people > who checkout the old trunk will soon see a very outdated view and may send us > outdated patches in JIRA. When the contents of old-trunk disappear it's > obvious to them what happened. If they had already some changes in their > checkout, the svn client will keep the changed files as unversioned (after > upgrade). The history keeps available, so it's also possible to checkout an > older version from trunk using @rev or -r rev. I did a similar step with some > backwards compatibility changes in lucene (add a README). > > Uwe > > - > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > >> -Original Message- >> From: Michael McCandless [mailto:luc...@mikemccandless.com] >> Sent: Monday, March 22, 2010 11:37 AM >> To: java-dev@lucene.apache.org >> Subject: Re: (LUCENE-2297) IndexWriter should let you optionally enable >> reader pooling >> >> I think we should. >> >> It (newtrunk) was created to test Hoss's side-by-sdie proposal, and >> that approach looks to be working very well. >> >> Up until now we've been committing to the old trunk and then >> systematically merging over to newtrunk. I think we should now flip >> that, ie, commit to newtrunk and only merge back to the old trunk if >> for some strange reason it's needed. >> >> Mike >> >> On Mon, Mar 22, 2010 at 6:32 AM, Uwe Schindler wrote: >>> Are we now only working on newtrunk? >>> >>> - >>> Uwe Schindler >>> H.-H.-Meier-Allee 63, D-28213 Bremen >>> http://www.thetaphi.de >>> eMail: u...@thetaphi.de >>> -Original Message- From: Michael McCandless (JIRA) [mailto:j...@apache.org] Sent: Monday, March 22, 2010 11:22 AM To: java-dev@lucene.apache.org Subject: [jira] Resolved: (LUCENE-2297) IndexWriter should let you optionally enable reader pooling [ https://issues.apache.org/jira/browse/LUCENE- 2297?page=com.atlassian.jira.plugin.system.issuetabpanels:all- >> tabpanel ] Michael McCandless resolved LUCENE-2297. Resolution: Fixed Fixed on newtrunk. > IndexWriter should let you optionally enable reader pooling > --- > >Key: LUCENE-2297 >URL: https://issues.apache.org/jira/browse/LUCENE- 2297 >Project: Lucene - Java > Issue Type: Improvement > Reporter: Michael McCandless > Priority: Minor >Fix For: 3.1 > >Attachments: LUCENE-2297.patch > > > For apps using a large index and frequently need to commit and resolve deletes, the cost of opening the SegmentReaders on demand >> for every commit can be prohibitive. > We an already pool readers (NRT does so), but, we only turn it on >> if NRT readers are in use. > We should allow separate control. > We should do this after LUCENE-2294. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue on
Re: New LuSolr trunk (was: RE: (LUCENE-2297) IndexWriter should let you optionally enable reader pooling)
For Solr, we can just move the current trunk to a 15 branch. -Yonik On Tue, Mar 23, 2010 at 9:39 AM, Grant Ingersoll wrote: > > On Mar 22, 2010, at 8:27 AM, Uwe Schindler wrote: > >> Hi all, >> >> the discussion where to do the development after the merge, now gets actual: >> >> Currently a lusolr test-trunk is done as a branch inside solr >> (https://svn.apache.org/repos/asf/lucene/solr/branches/newtrunk). The >> question is, where to put the main development and how to switch, so >> non-developers that have checkouts of solr and/or lucene will see the change >> and do not send us outdated patches. >> >> I propose to do the following: >> >> - Start a new top-level project folder inside /lucene root svn folder: >> https://svn.apache.org/repos/asf/lucene/lusolr (please see "lusolr" as a >> placeholder name) and add branches, tags subfolders to it. Do not create >> trunk and do this together with the next step. > > OK, I created https://svn.apache.org/repos/asf/lucene/dev/ and given > appropriate rights. Uwe, you can now do the rest of the move. Once you've > done it, let me know and I can make sure to add back the contrib rights. > >> - Move the branch from >> https://svn.apache.org/repos/asf/lucene/solr/branches/newtrunk to this new >> directory as "trunk" >> - For lucene flexible indexing, create a corresponding flex branch there and >> svn copy it from current new trunk. Merge the lucene flex changes into it. >> Alternatively, land flex now. Or simply do svn copy of current flex branch >> instead of merging (may be less work). >> - Do the same for possible solr branches in development >> - Create a tag in the lucene tags folder and in the solr tags folder with >> the current state of each trunk. After that delete all contents from old >> trunk in solr and lucene and place a readme file pointing developers to the >> new merged trunk folder (for both old trunks). This last step is important, >> else people who checkout the old trunk will soon see a very outdated view >> and may send us outdated patches in JIRA. When the contents of old-trunk >> disappear it's obvious to them what happened. If they had already some >> changes in their checkout, the svn client will keep the changed files as >> unversioned (after upgrade). The history keeps available, so it's also >> possible to checkout an older version from trunk using @rev or -r rev. I did >> a similar step with some backwards compatibility changes in lucene (add a >> README). >> >> Uwe >> >> - >> Uwe Schindler >> H.-H.-Meier-Allee 63, D-28213 Bremen >> http://www.thetaphi.de >> eMail: u...@thetaphi.de >> >> >>> -Original Message- >>> From: Michael McCandless [mailto:luc...@mikemccandless.com] >>> Sent: Monday, March 22, 2010 11:37 AM >>> To: java-dev@lucene.apache.org >>> Subject: Re: (LUCENE-2297) IndexWriter should let you optionally enable >>> reader pooling >>> >>> I think we should. >>> >>> It (newtrunk) was created to test Hoss's side-by-sdie proposal, and >>> that approach looks to be working very well. >>> >>> Up until now we've been committing to the old trunk and then >>> systematically merging over to newtrunk. I think we should now flip >>> that, ie, commit to newtrunk and only merge back to the old trunk if >>> for some strange reason it's needed. >>> >>> Mike >>> >>> On Mon, Mar 22, 2010 at 6:32 AM, Uwe Schindler wrote: Are we now only working on newtrunk? - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Michael McCandless (JIRA) [mailto:j...@apache.org] > Sent: Monday, March 22, 2010 11:22 AM > To: java-dev@lucene.apache.org > Subject: [jira] Resolved: (LUCENE-2297) IndexWriter should let you > optionally enable reader pooling > > > [ https://issues.apache.org/jira/browse/LUCENE- > 2297?page=com.atlassian.jira.plugin.system.issuetabpanels:all- >>> tabpanel > ] > > Michael McCandless resolved LUCENE-2297. > > > Resolution: Fixed > > Fixed on newtrunk. > >> IndexWriter should let you optionally enable reader pooling >> --- >> >> Key: LUCENE-2297 >> URL: https://issues.apache.org/jira/browse/LUCENE- > 2297 >> Project: Lucene - Java >> Issue Type: Improvement >> Reporter: Michael McCandless >> Priority: Minor >> Fix For: 3.1 >> >> Attachments: LUCENE-2297.patch >> >> >> For apps using a large index and frequently need to commit and > resolve deletes, the cost of opening the SegmentReaders on demand >>> for > every commit can be prohibitive. >> We an already pool readers (NRT does so), but, we only turn it on >>> if > NRT readers ar
