Re: [sage-devel] Basing new development versions of Sage on the previous development version
On Thu, Mar 15, 2012 at 3:34 AM, Dima Pasechnik wrote: > On Thursday, 15 March 2012 17:55:31 UTC+8, Jeroen Demeyer wrote: >> >> Okay, I'll think about your suggestion and changing the merger >> procedure. But I'll be honest that this is not too high on my list of >> priorities. > > > well, I think that Keshav's approach is very important if we want to > decrease the huge rate of bitrot we have now > with patches that did not make it into a release quickly. > As the current scheme of things destroys the history of development, it gets > hard to recreate the state of source when > the now bit-rotten patch has been working. +1. Note that this is rather orthogonal from the git vs. hg discussion, right now we're using hg as a glorified diff and patch (and periodically-constructed changelog). > By the way, is http://hg.sagemath.org/ now officially dead? It didn't change > a bit since January or so... In the current release model, there's no "history" to push to the the central repository until a final release is cut, as it keeps getting re-written and tossed. Part of this proposal would be that hg.sagemath.org (or its equivalent) would be the actual current development head(s). - Robert -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Basing new development versions of Sage on the previous development version
On Thursday, 15 March 2012 17:55:31 UTC+8, Jeroen Demeyer wrote: > > Okay, I'll think about your suggestion and changing the merger > procedure. But I'll be honest that this is not too high on my list of > priorities. > well, I think that Keshav's approach is very important if we want to decrease the huge rate of bitrot we have now with patches that did not make it into a release quickly. As the current scheme of things destroys the history of development, it gets hard to recreate the state of source when the now bit-rotten patch has been working. By the way, is http://hg.sagemath.org/ now officially dead? It didn't change a bit since January or so... Dima -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Basing new development versions of Sage on the previous development version
Okay, I'll think about your suggestion and changing the merger procedure. But I'll be honest that this is not too high on my list of priorities. -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Basing new development versions of Sage on the previous development version
Le mardi 06 mars, Jeroen Demeyer a écrit: > On 2012-03-06 10:29, Julien Puydt wrote: > > Will you take it bad if I notice a good chunk of those problems > > exist just because sage has an organization where each spkg is a > > separate versioned project hidden in a binary archive? > Of course, I agree that the organization of spkgs could be a lot > better, but that's not the point of this thread. I think it is : if sage had an (unversioned?) upstream/ directory containing the raw archives of upstream sources, and all the rest correctly versioned in a single tree, then : - each development version would naturally be based on the previous one ; - there would be no point checking for "uncommitted changes" in a hundred places ; - there would be no "missing files" (they would show up in "whatever-dvcs status") ; - etc. Snark on #sagemath -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Basing new development versions of Sage on the previous development version
On 2012-03-06 10:29, Julien Puydt wrote: > Will you take it bad if I notice a good chunk of those problems exist > just because sage has an organization where each spkg is a separate > versioned project hidden in a binary archive? Of course, I agree that the organization of spkgs could be a lot better, but that's not the point of this thread. -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Basing new development versions of Sage on the previous development version
On 2012-03-06 10:29, Julien Puydt wrote: > Will you take it bad if I notice a good chunk of those problems exist > just because sage has an organization where each spkg is a separate > versioned project hidden in a binary archive? To be fair, this is true of only *one* of the problems I mentioned. -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Basing new development versions of Sage on the previous development version
Le mardi 06 mars, Jeroen Demeyer a écrit: > On 2012-03-06 05:49, Keshav Kini wrote: > > Hi Jeroen, > > > > If I understand correctly, your reasoning for basing new development > > releases on the previous stable release rather than the previous > > development release is that sometimes a ticket may be merged into a > > development release but then need to be removed from a subsequent > > development release due to problems that were found later, or an > > SPKG that was upgraded needs to be downgraded again. > > Another important reason is that I sometimes work on more than one > Sage version at the same time. While putting together > sage-5.0.beta7, I was already merging stuff in sage-5.0.beta8. This > gets harder with what you propose. > > It's also harder to deal with "exceptional" cases. Like the problem > we had with sage-5.0.beta6 where two files were missing from the > distribution. It's easier to go back and do it the right way, as > opposed to fixing it afterwards. > > Another example: #11235. While checking all spkgs for uncommitted > changes, I discovered the spkg there contained some garbage. I simply > removed the garbage and put up a new spkg with the same version > number. With your proposal, this would probably have involved > creating a new ticket instead of making the trivial adjustment in a > subsequent Sage version. > > Obviously, reverting a patch would become more difficult, I don't even > know the "right" way to do it with Mercurial. > > Why doesn't git/hg consider the version sage-5.0.beta0 inside > sage-5.0.beta6 the same as version sage-5.0.beta0 inside > sage-5.0.beta7? They should be identical (apart from timestamps). Will you take it bad if I notice a good chunk of those problems exist just because sage has an organization where each spkg is a separate versioned project hidden in a binary archive? Snark on #sagemath -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Basing new development versions of Sage on the previous development version
On 2012-03-06 05:49, Keshav Kini wrote: > Hi Jeroen, > > If I understand correctly, your reasoning for basing new development > releases on the previous stable release rather than the previous > development release is that sometimes a ticket may be merged into a > development release but then need to be removed from a subsequent > development release due to problems that were found later, or an SPKG > that was upgraded needs to be downgraded again. Another important reason is that I sometimes work on more than one Sage version at the same time. While putting together sage-5.0.beta7, I was already merging stuff in sage-5.0.beta8. This gets harder with what you propose. It's also harder to deal with "exceptional" cases. Like the problem we had with sage-5.0.beta6 where two files were missing from the distribution. It's easier to go back and do it the right way, as opposed to fixing it afterwards. Another example: #11235. While checking all spkgs for uncommitted changes, I discovered the spkg there contained some garbage. I simply removed the garbage and put up a new spkg with the same version number. With your proposal, this would probably have involved creating a new ticket instead of making the trivial adjustment in a subsequent Sage version. Obviously, reverting a patch would become more difficult, I don't even know the "right" way to do it with Mercurial. Why doesn't git/hg consider the version sage-5.0.beta0 inside sage-5.0.beta6 the same as version sage-5.0.beta0 inside sage-5.0.beta7? They should be identical (apart from timestamps). -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Basing new development versions of Sage on the previous development version
> Until SPKGs are possible to uninstall or > downgrade effectively, that is impractical. Not really. For Sage purposes, downgrading a spkg is exactly the same as upgrading a spkg. It's not like we sort package versions or anything, we only check for equality. -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Basing new development versions of Sage on the previous development version
Hi Jeroen, If I understand correctly, your reasoning for basing new development releases on the previous stable release rather than the previous development release is that sometimes a ticket may be merged into a development release but then need to be removed from a subsequent development release due to problems that were found later, or an SPKG that was upgraded needs to be downgraded again. However, leaving SPKG versions aside, isn't it possible to at least keep the history of the Sage repositories linear? If you want to unmerge a ticket, you can just commit the reverse of the relevant patches. `hg backout` is a command you could use to do this. If you do keep the history of the Sage repositories linear, it would prevent problems such as I will now describe. Please take a look at https://github.com/sagemath/sagelib/network (you can click and drag to scroll the graph). Apart from the large number of abandoned branches in the "sagemath" row (i.e. which come from your merger script), you may also notice that William has committed some changes on top of Sage 5.0.beta4. In order to create a patch based on the latest development version (currently 5.0.beta6), as you have requested all Sage developers do, he must rebase his "lfun" branch onto 5.0.beta6. This involves moving his branch down from 5.0.beta4 all the way to 4.8, and then up again to 5.0.beta6 along the new branch you have created for 5.0.beta6. fs@boone ~/src/sagelib $ git rev-list upstream/prerelease-5.0.beta6..upstream/prerelease-5.0.beta4 | wc -l 143 fs@boone ~/src/sagelib $ git rev-list upstream/prerelease-5.0.beta4..upstream/prerelease-5.0.beta6 | wc -l 201 fs@boone ~/src/sagelib $ git rev-list 5.0.beta4..upstream/prerelease-5.0.beta6 | wc -l 60 In other words, William must first rebase his commits down by 143 commits to 4.8, and then up 201 commits to 5.0.beta6. The third command shows the distance between "5.0.beta4" the *tag* (i.e. the 5.0.beta4 tag which is published in the repository provided with 5.0.beta6) and the tip of 5.0.beta6. In other words, if you based development releases on the previous development release, instead of throwing away each development release before making the next one, then William would only have to rebase upwards, and only by 60 commits. You might question why this is important. The more commits you have to rebase over, the more likely the rebase will fail and need to be done manually, increasing developer workload. Also, a developer who is not well-versed in version control systems may not understand what is going on if the rebase fails on the way downward, since he expects the rebase to be going upward from an older development version to a newer one. If his code is heavily based on another patch that was merged in a previous development release, the rebase is almost certain to fail on the way down, which is entirely unexpected from the developer's point of view - he's just trying to make his code catch up with the latest development release, so why should he have to worry about previous development releases disappearing from under him? And in any case, developers should not even be forced to rebase except in unusual circumstances (or if they personally feel like it). See my reasoning about why merging is better than rebasing here: https://groups.google.com/d/msg/sage-devel/DmjL8hHJYI8/o-R2FYVUczQJ Merging is not even possible anymore when you don't base development releases on the previous development release, because this practice /causes published history to be rescinded/, and so developers must scramble to get rid of commits that have disappeared from the upstream source, which requires rebasing. In summary, can you please base the Mercurial repositories of new Sage development releases on the Mercurial repositories of the previous Sage development release? I'm not asking you to allow `sage -upgrade` for development versions. Until SPKGs are possible to uninstall or downgrade effectively, that is impractical. But can you please try not to discard commits from repositories that are published and that people are encouraged to do development work on? I am CCing this mail to sage-devel to solicit comments. Thanks for your time, Keshav -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
