Re: [sage-combinat-devel] sage-combinat move to git
Hi! Here is an update from Sage Days 54 from today. We discussed the new workflow and the move of the sage-combinat queue. Mathieu and I wrote a first version of the new workflow including branch naming conventions, where to put your branches on trac (in your personal space u/yourname/yourbranch versus in the public space public/combinat/yourbranch). Please read the details http://wiki.sagemath.org/TentativeConventions and send comments if you have any! Volker Braun and Andrew Ohanah said that the move to git will happen before the end of the year. At that point the sage-combinat queue will need to move to trac/git. We have a section on how to do that for your personal patches on the sage-combinat queue and we started to move some patches. Currently we mark them as kschur-as.patch # git:u/aschilling/combinat/kschur to say that the patch was already moved and say to which branch. At some point, every patch that does not have this information should be moved by the script and the combinat-queue should be frozen at that point. Best, Anne -- You received this message because you are subscribed to the Google Groups sage-combinat-devel group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-combinat-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-combinat-devel@googlegroups.com. Visit this group at http://groups.google.com/group/sage-combinat-devel. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sage-combinat-devel] sage-combinat move to git
On Mon, Nov 04, 2013 at 11:30:53PM -0800, Anne Schilling wrote: With that in mind, I would aim for just using a bunch of branches in the sage git repository, with a way to group those in collections that would model our subcommunities: combinat / cluster_algebras / categories / tableaux / crystals / dynamics / semigroups / ... As Simon points out, some tags mechanism would be great for this. Talking to Andrew, tags are apparently not the right thing to do and are only used by release managers. On github we can just have various subdirectories for the various categories. Let me raise a possible ambiguity: what I meant by ``tags'' is *some* mechanism by which a single ticket/branch can be in several categories at once; call it keywords if you prefer. I wasn't being specific about whether we should be using git's tags for this or any other mechanism. In any cases, subdirectories do not allow this feature (unless we can have symbolic links???) As Simon pointed out, this will be easy. However whether it's practical or not will depend on how much time we will have to wait for recompilation in average. Part of the answer will be in the compilation cache[1] infrastructure that Andrew co have been exploring, but I don't know how far this project progressed. From the sage-git version you can just do sage -f ccache Excellent; thanks! Feedback about how it behaves in practice (speed, space usage) is welcome! Cheers, Nicolas -- Nicolas M. Thiéry Isil nthi...@users.sf.net http://Nicolas.Thiery.name/ -- You received this message because you are subscribed to the Google Groups sage-combinat-devel group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-combinat-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-combinat-devel@googlegroups.com. Visit this group at http://groups.google.com/group/sage-combinat-devel. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sage-combinat-devel] sage-combinat move to git
On Tuesday, November 5, 2013 5:03:56 AM UTC-8, Nicolas M. Thiery wrote: sage -f ccache Excellent; thanks! Feedback about how it behaves in practice (speed, Fedora has had it installed by default for quite some time by now. I don't understand why anybody would not use it. space usage) is welcome! Here's a nickel, go buy yourself a larger hard disk ;-) -- You received this message because you are subscribed to the Google Groups sage-combinat-devel group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-combinat-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-combinat-devel@googlegroups.com. Visit this group at http://groups.google.com/group/sage-combinat-devel. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sage-combinat-devel] sage-combinat move to git
Hi! Here is an update from Sage Days 54 of today! - Development model: = There are two ways to share code between collaborators on the trac server. (a) If users vivienne and anne work on some feature, both could have their own branches on trac in /u/vivienne/combinat/partition/V-branch /u/anne/combinat/partition/A-branch respectively. If V makes a change, A would need to pull from V's branch to get the changes, but would push her changes to A's branch and vice versa. (b) Users V and A could work on the same branch in /public/combinat/partition/our-branch In this case anybody with trac access can in principle make changes to this branch. If there are lots of collaborators, (a) might be cumbersome since a given developer needs to pull from the branches of all developers since it is not necessarily clear who made the last changes. But (a) is better if each developer wants to save their changes before merging. - Tornado branches: = If developers want to share functionalities with collaborators who are not Sage developers which are in various branches, it is possible to merge those and point the collaborators to that branch and put it on /public/combinat/... - Naming conventions for searching: It would be a good idea for searching purposes to name branches using keywords. For example, if there is a branch related to combinatorics, it is a good idea to put /combinat in the name. If it further is about partitions and symmetric functions, one could put partitions-symmetric_functions into the name, so that this is searchable for other developers. It is a good idea to label work-in-progress by name of branch-wip so that other developers do not base anything off this branch. It is a good idea to label tornado branches as name of branch-tornado so that other developers do not base anything off this branch. - We started playing with the trac-git functionalities by playing with ticket 15300. You can look at the log on the ticket there. We will probably import some more patches from the sage-combinat queue tomorrow to see how it works and how to play with the new development flow. Best, Anne -- You received this message because you are subscribed to the Google Groups sage-combinat-devel group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-combinat-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-combinat-devel@googlegroups.com. Visit this group at http://groups.google.com/group/sage-combinat-devel. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sage-combinat-devel] sage-combinat move to git
On Tue, Nov 05, 2013 at 06:04:05PM -0800, Anne Schilling wrote: Here is an update from Sage Days 54 of today! Thanks for the report! Just a silly question: why the name Tornado ? Cheers, Nicolas -- Nicolas M. Thiéry Isil nthi...@users.sf.net http://Nicolas.Thiery.name/ -- You received this message because you are subscribed to the Google Groups sage-combinat-devel group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-combinat-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-combinat-devel@googlegroups.com. Visit this group at http://groups.google.com/group/sage-combinat-devel. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sage-combinat-devel] sage-combinat move to git
- Should we use a fork for sage-combinat? Or branches for the various patches or projects? Technically, every clone is a fork. :-) * If we do use a fork, how to sync the combinat fork with the main sage fork? In particular, once branches are merged in the main fork, can they merge in the fork as well? I'd suggest that sage-combinat is built on top of the sage repository and that the master branch in that sage-combinat repository is identical with the master branch of sage. sage-combinat additions should be developed in a separate branch, call it combinat-master or so. As far as I understand, sage-combinat will have many branches anyway. Technically, that would just mean that sage-combinat is a (or many) branches of the official sage repository. That would certainly ease the way to keep in sync with the sage repository since a simple git fetch brings in all the commits from the sage repository. Is there an automatic way to remove a sage-combinat branch once it has been merged into main sage? To automate this is probably easy since one would only have to do git branch -D branch-to-be-removed git push origin :branch-to-be-removed (where I assume that origin points to the official sage-combinat repository). - Will it be easy to switch between branches (like experimental code, branches to be reviewed etc)? Just do a git checkout branch-name. Ralf -- You received this message because you are subscribed to the Google Groups sage-combinat-devel group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-combinat-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-combinat-devel@googlegroups.com. Visit this group at http://groups.google.com/group/sage-combinat-devel. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sage-combinat-devel] sage-combinat move to git
Hi! Thanks Anne and Travis for starting the discussion! On Sun, Nov 03, 2013 at 05:48:19PM -0800, Anne Schilling wrote: Here is an initial list of discussion points regarding the Sage-combinat queue: - Should we use a fork for sage-combinat? Or branches for the various patches or projects? * If we do use a fork, how to sync the combinat fork with the main sage fork? In particular, once branches are merged in the main fork, can they merge in the fork as well? * If we don't fork, then is there some way we can tie all of the combinat branches together? Specifically can we list all branches which don't have ticket numbers. At the beginning we were a relatively small community and there were lots of interactions across the whole combinat code base, because we had to build the foundations. Hence there was a relatively clear notion of combinat patch (which somehow included categories) and it was good to have a relatively centralized workflow. Now the core is stabilizing and the community has grown quite much both in terms of number of contributors and of breadth of topics. It's thus naturally evolving from a tightly knit community into a continuum of overlapping subcommunities of Sage developers that gather together at the occasion of specific projects. And that's great! Now, we want the workflow to reflect that. The dichotomy in Sage-Combinat or not that the queue enforces has become a bother, especially for borderline people like Simon. With that in mind, I would aim for just using a bunch of branches in the sage git repository, with a way to group those in collections that would model our subcommunities: combinat / cluster_algebras / categories / tableaux / crystals / dynamics / semigroups / ... As Simon points out, some tags mechanism would be great for this. Here is what feels like the largest hurdles we might face; I might be completely wrong though, and we probably can only know for sure through experimenting at a large scale with the new workflow: - Switching from one branch to the other. As Simon pointed out, this will be easy. However whether it's practical or not will depend on how much time we will have to wait for recompilation in average. Part of the answer will be in the compilation cache[1] infrastructure that Andrew co have been exploring, but I don't know how far this project progressed. - Mitigation of the divergence between parallel development branches. An important feature of the workflow is to be able to request things like ``pull on my local Sage all the experimental features about tableaux, categories, and semigroups''. Typical applications: I want to run a particular calculation for which I need a bunch of different in-development features. Or I want to share with a non-programmer colleague the latest semigroup code. That's what sage -combinat install was about; granted it very was far from 100% robust, but it worked relatively often. In theory, this is easy: it's just about creating a temporary merge point for a bunch of branches. But will this be robust in practice? That is will we be able to maintain the state of our branches in a consistent enough state that the most-often-needed combinations of branches will actually merge without conflicts? Of course the new workflow, and the fact that the development is progressively moving from the core to the periphery, is likely to reduce the life span of branches, which in turn will reduce the need for the above. - Dependency handling - How do we transform the queue into git branches, how do we handle dependencies? Back in March, I wrote a script which was based on the sage development tools and did just this (attached; see the import_mercurial_queue function). With it, I was able to import most of the Sage-Combinat queue [2] as a bunch of branches with dependencies (see the graph attached to [2]). So, in principle that should be easy technically. However: - The sage dev tools have evolved quite much since then, and I have not played with the script since then. So it certainly needs to be rebased on the latest dev tools. And it also deserves some cleanup. - A good part of the work was to annotate the series file with explicit dependencies information. Part of this information is on trac, but in practice it's often outdated or incomplete, especially for the more experimental patches. Thus that required quite some manual trial-and-error work (oh, it does not apply! hmm, let's see why ...). And now this information needs to be updated again w.r.t. all the changes in the queue in the past six months. Note that this is not wasted time, as we need to update this dependency information on trac anyway at some point. - We need to decide whether we do the migration all at once, or instead we let the patch developers migrate progressively their own patches. It's probably easier to do it all at once, if we can afford
Re: [sage-combinat-devel] sage-combinat move to git
Hi! With that in mind, I would aim for just using a bunch of branches in the sage git repository, with a way to group those in collections that would model our subcommunities: combinat / cluster_algebras / categories / tableaux / crystals / dynamics / semigroups / ... As Simon points out, some tags mechanism would be great for this. Talking to Andrew, tags are apparently not the right thing to do and are only used by release managers. On github we can just have various subdirectories for the various categories. As Simon pointed out, this will be easy. However whether it's practical or not will depend on how much time we will have to wait for recompilation in average. Part of the answer will be in the compilation cache[1] infrastructure that Andrew co have been exploring, but I don't know how far this project progressed. From the sage-git version you can just do sage -f ccache Best, Anne -- You received this message because you are subscribed to the Google Groups sage-combinat-devel group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-combinat-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-combinat-devel@googlegroups.com. Visit this group at http://groups.google.com/group/sage-combinat-devel. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sage-combinat-devel] sage-combinat move to git
Hi! Here is an initial list of discussion points regarding the Sage-combinat queue: - Should we use a fork for sage-combinat? Or branches for the various patches or projects? * If we do use a fork, how to sync the combinat fork with the main sage fork? In particular, once branches are merged in the main fork, can they merge in the fork as well? * If we don't fork, then is there some way we can tie all of the combinat branches together? Specifically can we list all branches which don't have ticket numbers. Is there an automatic way to remove a sage-combinat branch once it has been merged into main sage? - How do we transform the queue into git branches, how do we handle dependencies? - When will the change occur? * Will we freeze the hg queue at that point? * Where will we host the new combinat branches? - Will it be easy to switch between branches (like experimental code, branches to be reviewed etc)? Let us know if you have anything else to add. Anne and Travis On 11/3/13 8:27 AM, Anne Schilling wrote: Dear Sage-combinat developers, As most of you are probably aware of, the Sage development will soon change from hg to git. At the same time, the sage-combinat queue will need to be transformed into git branches. Most likely, this will mean a more shallow structure (rather than a linear queue) with parallel projects. We are going to discuss this change on Tuesday during Sage Days 54 in Davis. If you have any comments or suggestions, let us know! Best, Anne -- You received this message because you are subscribed to the Google Groups sage-combinat-devel group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-combinat-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-combinat-devel@googlegroups.com. Visit this group at http://groups.google.com/group/sage-combinat-devel. For more options, visit https://groups.google.com/groups/opt_out.
Fwd: Re: [sage-combinat-devel] sage-combinat move to git
From Dan Bump: Hi Anne, I'm not cc:ing the google group since I don't think I'm authorized to post from this email address. But I'm cc:ing Andrew and Travis. I am in Davis now. Greg Musicker was arriving on the shuttle as I got back from dinner. Here is a link to my slides: http://sporadic.stanford.edu/bump/Gittalk.pdf - Should we use a fork for sage-combinat? Or branches for the various patches or projects? It seems to me that a fork would be a bad idea if it can possibly be avoided. Maybe what would work would be to have a second combinat repository. This would have to be rebased this regularly with the main sage repository. Maybe the master branches would be identical. In other words, someone would rebase the master branch on the combinat repository with the master branch on the main repository. This would not be difficult, maybe it could be automated. But the combinat repo would have many branches, and rebasing these would be the responsibility of their owners. So there would be two main repositories: sage-main -- sage-combinat (with many branches) and the master branch would have to be frequently rebased. Patches in the current combinat mercurial queue would correspond to branches on the combinat repository, and every branch would have to be rebased regularly to keep up with changes on the sage-main master branch. But this would be the responsibility of the patch owner, not the release manager, so the work would be distributed. And because the patches would no longer be deeply nested it would be less difficult. * If we don't fork, then is there some way we can tie all of the combinat branches together? Specifically can we list all branches which don't have ticket numbers. Is there an automatic way to remove a sage-combinat branch once it has been merged into main sage? This should not be a problem. Dan -- You received this message because you are subscribed to the Google Groups sage-combinat-devel group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-combinat-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-combinat-devel@googlegroups.com. Visit this group at http://groups.google.com/group/sage-combinat-devel. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sage-combinat-devel] sage-combinat move to git
Hi Dan, Thank you for your comments. We will discuss this more on Tuesday. It seems to me that a fork would be a bad idea if it can possibly be avoided. I agree, especially if the aim is to merge the branches eventually with sage-main rather than having a separate project. Travis brought up the idea of forks. Maybe what would work would be to have a second combinat repository. This would have to be rebased this regularly with the main sage repository. Maybe the master branches would be identical. In other words, someone would rebase the master branch on the combinat repository with the master branch on the main repository. This would not be difficult, maybe it could be automated. But the combinat repo would have many branches, and rebasing these would be the responsibility of their owners. If there is a branch on partitions and a branch on tableaux, someone might be interested in merging them. So I guess we could also keep merged branches in the combinat repository. I discussed some of these issues already tonight with Mike and Mathieu and we seem mostly in agreement on what to do. Best, Anne -- You received this message because you are subscribed to the Google Groups sage-combinat-devel group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-combinat-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-combinat-devel@googlegroups.com. Visit this group at http://groups.google.com/group/sage-combinat-devel. For more options, visit https://groups.google.com/groups/opt_out.
