Re: [O] Sync up the org in emacs master to org maint branch?
Lars Ingebrigtsen writes: > John Wiegley writes: > >> We're moving toward a future where Emacs.git will represent "core >> Emacs", and only contain what core needs (plus a few historical bits, >> I'm sure). There should be no argument for keeping a project in core >> just to gain auxiliary benefits. > > I'm massively unenthusiastic about this future. Things in ELPA has to > be backwards-and-forwards compatible with a wide Emacs version range, no, they don't. That is one possible policy, but each package decides for itself whether to follow it. You can have ;; package-requires: ((emacs "25.2")) if you want. If you want to maintain two versions, one for older emacs, one for newer, you'll need two different package names, similar to gtk2, gtk3. It does appear we need a "next release" branch in ELPA git similar to "master" in emacs git, so the released ELPA package is still available, but those working on emacs master can access the leading edge ELPA packages. > Emacs doesn't seem to have a massive surfeit of developers, so I wonder > where this plan comes from. Several developers prefer the decoupled development cycle of ELPA packages. It is primarily up to the package developers whether a package is in core or not, at least until it is clear that there is no advantage to being in core. -- -- Stephe
Re: [O] Sync up the org in emacs master to org maint branch?
>> - Suppose that Emacs 22.0 is the current release and Emacs 22.1 is then >>released; CEDET is at >> - we update a registry somewhere indicating that Emacs 22.0 works with >> and 22.1 works with >> >> - When we make updates to CEDET we mark 22.1 as working with >> but we don't change that reference >>for 22.0 (or any older versions) >> - When someone complains that there's a bug in CEDET for 22.0 we >>indicate that it's no longer supported and that they should update >>Emacs to receive updates >> >> This process would almost be the same as what you get just by bundling >> CEDET with Emacs except that: >> >> - You can get the latest CEDET *if* you have the latest Emacs > > No. We have two branches: emacs-25 and master. The CEDET from master > will usually not work on any 25.x version. In the process which I proposed we would be developing against the most recently released Emacs. I now see that there's a trade off. If we were to go with my scheme then users would have access to the latest version, but we would: 1. miss out on the new features being developed on the Emacs master branch (which CEDET stands to gain a lot from); 2. have to endure a difficult and error prone release process every time Emacs is released because that's not what all the testing is done against; If we go ahead and merge it in then we would be able to benefit from all the new Emacs features and the release would be less painful. Of course our users would still have to wait for the latest features, but at least it would be worth the wait because we would be able to consider using features like threading. >> - The version of CEDET for any particular version of Emacs is as far as >>CEDET got before the next release of Emacs came out >> >> If this is what you were thinking of then please could you elaborate on >> what ended up being the problem which added more work. > > First off, CEDET is currently *not* a package, although that notion gets > thrown around a lot. It is scattered across the Emacs code base: > lisp/cedet, admin/grammars, etc/srecode, documentation, and test > suite. All this needs to be packaged in a way so that it can be > integrated into Emacs during a normal checkout. It needs to build and > test in such a normal checkout, but also separately when installed from > ELPA, including grammar compilation. And you need this twice: one for > emacs-25, one for master, with the possiblity to merge between the two. Yes, this would be a pain. I'm in favour of Phillip's approach in which the packages are self contained somewhere in the Emacs source tree. This would eliminate the need for a complicated copying process and eliminate the problem of where things go and what files should be updated. No other folder would be touched because a package contains it's own source, documentation etc. Unfortunately the idea isn't gaining a lot of traction. I'm not satisfied that the alternative would make things easier for anyone because, as you say, it would need to describe a complex copying process which intertwines CEDET with various parts of Emacs. > Then there's this "registry". No one has said how that should > work. "Submodules/Subtrees" are *not* a sufficient answer, they are just > tools. People will need to say how the *workflow* should be, including > such things like merges from stable, ChangeLog generation, AUTHORS, > NEWS, creating release tarballs, and so on. Once someone has written > this down *in detail*, we can discuss again if this indeed will make > things simpler and reduce our workload. I haven't heard the registry mentioned before. I mentioned it as a detail required by my process for supporting multiple versions, but I'm beginning to think that it's the wrong line of thought to pursue. >>> Well, we cannot really discuss this since there's no real plan how this >>> all should work. I can only speak from experience. >> >> We can still put ideas forward though. Haven't come up with anything >> myself yet though. > > Yes, you can, but it has a cost. Once again, the CEDET merge is stalled, > and we spend our time writing mails. I find this situation incredibly > frustrating. In lieu of any input from others the best we have is Phillip's idea of using Elpa. That solves how the files are copied in and compiled. >>> How does CEDET, Gnus and Org affect the rest of Emacs? They strongly >>> depend on Emacs "core" (whatever exactly that is), not the other way >>> round. >> >> I believe that one of the intentions of the move is to enforce this so >> we can't build bad dependencies -- am I wrong? > > I think other modes should use Semantic. Agreed. I mentioned this in my response to Stefan's email. Shouldn't we then consider moving it out of CEDET in Emacs? >> Perhaps I'm wrong, but to my mind the package approach would afford us >> with more testing before we get to the point of another release of >> Emacs. > > If you develop on Emacs 'master', you can u
Re: [O] Sync up the org in emacs master to org maint branch?
>>> they are extremely dependend on >>> many obscure Emacs internals (not sure about Org). >> >> Shouldn't we be trying to avoid this situation? It's sure to come back >> and bite us in the future. If we continue to develop a package >> (wherever it ends up being developed) which is so tightly coupled that >> any kind of re factoring in core becomes a nightmare, because we have to >> consider the umpteen ways in which it'll break the package, then surely >> that's a bad methodology to continue? > > I don't know what you have in mind. All I can say is: CEDET couldn't do > what it does if we'd restrict ourselves to some subset of Emacs. In particular I was worried by the phrasing "extremely dependent". I agree that in order to make a system like CEDET without re-inventing the wheel and without running into performance problems it's necessary to depend on more primitive parts of Emacs. Perhaps we can improve the relationship(?) Perhaps this is a discussion to be had when all of this is done though. >> I feel like this problem isn't intractable. > > I didn't say it's intractable. I just said it means more work for me. > >> We can mark some state of CEDET as being stable under the various >> versions of Emacs (because it was at the time) and then only support >> the current release for the latest package. This would most likely >> require changes to package to ensure that you get an appropriate >> version of the package when you download it. > > As I said: we did that. It was a huge amount of work. Please understand > where I'm coming from: if you look through the CEDET history, you will > see that in the past few years I almost exclusively did infrastructure > work and no real coding. All I can say is: *I* won't do this anymore, > and I don't want to be part in something which will increase grunt > work. We did not make this decision lightly. But with the amount of > developers we have, I'm convinced it's the right thing to do. Fair enough. I don't have the experience here. It just seems like we could meet both goals without increasing the work load in this regard. To be clear I agree that, whatever decision this discussion arrives at, we definitely don't want to we waste the time of any developer with grunt work. Continuing this line of thought. I'm not sure that we're thinking of the same process here. What I'm suggesting is as follows: - Suppose that Emacs 22.0 is the current release and Emacs 22.1 is then released; CEDET is at - we update a registry somewhere indicating that Emacs 22.0 works with and 22.1 works with - When we make updates to CEDET we mark 22.1 as working with but we don't change that reference for 22.0 (or any older versions) - When someone complains that there's a bug in CEDET for 22.0 we indicate that it's no longer supported and that they should update Emacs to receive updates This process would almost be the same as what you get just by bundling CEDET with Emacs except that: - You can get the latest CEDET *if* you have the latest Emacs - The version of CEDET for any particular version of Emacs is as far as CEDET got before the next release of Emacs came out If this is what you were thinking of then please could you elaborate on what ended up being the problem which added more work. Also, would this be a problem for our users? i.e. would we be inundated with emails requesting continued support on older versions or would users generally accept this kind of change. I mostly work on back end systems so I don't have a good feeling for how this would go down with users (I would find this reasonable as a user). > Bug fixes can go with point release, which we should have every > year. But yes, if you want the latest, greatest and buggiest, you'll > have to use Emacs master. But that goes for a lot of Emacs features, so > I don't see why it's particularly bad for CEDET. I feel like there are aspects of CEDET which are still under development. Perhaps I'm just unlucky in my particular usage patterns of upstream and the way things are going this will be fine (with the un-merged parts going into packages.) >> I'm interested in exploring more with regards to how the subtree >> approach would work. In particular how it would impact the Makefiles >> and build process. I don't think that introducing features like this >> necessarily increases the burden of maintaining our tooling. If we get >> it right it could reduce it. > > Well, we cannot really discuss this since there's no real plan how this > all should work. I can only speak from experience. We can still put ideas forward though. Haven't come up with anything myself yet though. >> I think that it's a worthwhile goal to make core smaller. It may not be >> a gigantic enterprise system with tens of thousands of source files, >> like some of us are accustomed to working on, but I think that it >> becomes easier to reason about the features and behaviour of core when >> it's smaller.
Re: [O] Sync up the org in emacs master to org maint branch?
> > DE> This is a misunderstanding. I said I wanted to move support for certain > DE> languages and project types into ELPA, not CEDET core. I'm still of the > DE> opinion that moving it completely to ELPA is a mistake. > > It would only be a mistake if other parts of core need to use it. If only > users make use of it, then having it ELPA will be invisible to them. Apologies in advance for the significant verbage. I just want to provide context. * Context regarding CEDET: (apologies: I know this thread is more concerned with org mode and, if you'll bare with me, I think that this is relevant.) I started the CEDET merge process a few months ago. There was talk on the CEDET mailing list regarding the difficulty of getting going with CEDET as a user and/or a developer. One of the ideas put forward was that there ought to be a merge into Emacs so that one wouldn't have to clone the repo, build the code and install it with some ELisp config. hacking. I got in touch with Eric directly and bounced some ideas passed him. Subsequently I volunteered to help with the merge. I got in touch with JohnW and (not being very familiar with Emacs development) asked some basic questions about how to go about accomplishing the merge. What resulted instead was the idea that we should try to look at streamlining how CEDET get's included with the Emacs tarball rather than having it live in two repositories. I agreed with the idea and got to work on getting a version of CEDET together which would work with 26. I merged in the Emacs -> CEDET direction and ended up with a version of CEDET which is a WIP and works with Emacs 26. Some time passed between then and the start of the discussions here. I think that the approach has evolved past what I was originally planning on working towards and is now something along the lines of: do a final merge of core CEDET components and make the rest into a series of ELPA packages. * What I think that we shouldn't lose sight of (if I may suggest it): is that packaging CEDET, Org Mode and other packages like them in a process which integrates them only when producing the tarball would serve to simplify things for everyone. Emacs core wouldn't be able to depend on packages which are more relevant to the users (and package developers) of Emacs, but these packages would still there like they always have been when one downloads binary Emacs. I'm new around here and I know that I lack the context and experience in this project to make such swooping suggestions or judge the validity of these points, but I thought that they would be worth raising. Kind regards, Edward Steere
Re: [O] Sync up the org in emacs master to org maint branch?
> It's not like packages communicate with Emacs over a well > defined RESTful interface. In other words: CEDET and Gnus are not > loosely coupled, quite the opposite: they are extremely dependend on > many obscure Emacs internals (not sure about Org). Shouldn't we be trying to avoid this situation? It's sure to come back and bite us in the future. If we continue to develop a package (wherever it ends up being developed) which is so tightly coupled that any kind of re factoring in core becomes a nightmare, because we have to consider the umpteen ways in which it'll break the package, then surely that's a bad methodology to continue? (I'm not suggesting that we rewrite it to make it more loosely coupled, just that it seems like a bad idea to continue allowing this going forward.) > As a consequence, we > and also the Gnus guys decided to not do separate releases anymore. We > used to provide CEDET for different Emacs versions, and it was a *huge* > amount of work. I had a buildbot running with 7 or 8 Emacs versions to > run the test suite, and things broke pretty regularly. > Now you might say: fine, only release a package for current master. But > let's say we put CEDET into ELPA. Emacs 26 gets released, and work on > Emacs 27 starts. Now there are changes happening in Emacs 27 that > require changes in CEDET, which make it incompatible with Emacs 26. So > you have to create two packages: one for 26, one for 27? Not only is > this confusing, but it most definitely increases my workload. I feel like this problem isn't intractable. We can mark some state of CEDET as being stable under the various versions of Emacs (because it was at the time) and then only support the current release for the latest package. This would most likely require changes to package to ensure that you get an appropriate version of the package when you download it. Consider the problem which our users currently face. The built in CEDET is miles behind and missing very important bug fixes and features. The upstream CEDET can be a real pain to setup, but it has the latest features. This is not a good place to be. If we merge CEDET in and only release it with the realeses of Emacs then we are heading for a state where you only get the new features and bug fixes every time Emacs is realesed, i.e. our users might have to wait up to two years to have something fixed. This is also a bad place to be. >> We can arrange things so that a Git clone of Emacs includes pulling the >> submodules (or trees, or ELPA.git, or what not) that are considered part of >> "main Emacs development", including some of those batteries. I see this all >> as >> a process issue, and that "living in one Git repository" has just been an >> implementation strategy to satisfy that process. > > Obviously, I'm very skeptical towards such an approach. Our tooling > around Emacs development is already very intricate and only works > because a few people work quietly behind the scenes to keep this all > running. Introducing even more complexity through > submodules/subtrees/whatever will do the opposite of what you want to > achieve: it creates more work for those few people who manage the Emacs > infrastructure. But I'd love to hear what for instance Glenn and Paul > think about this. I'm interested in exploring more with regards to how the subtree approach would work. In particular how it would impact the Makefiles and build process. I don't think that introducing features like this necessarily increases the burden of maintaining our tooling. If we get it right it could reduce it. For example we could establish some sort of convention for building submodules/subtrees which allows us to simplify the related Makefiles. >> Why do the split at all? Core becomes smaller, > > First off, the Emacs repo isn't that big w.r.t. the number of > files. Secondly, while there surely is an inverse correlation between > repo size and maintainability, I would argue that as long as the Big > Three are well maintained, they are not the problem. When did CEDET, > Gnus or Org ever significantly delay an Emacs release? When was an > Emacs core developer ever forced to fix a critical thing in those > packages he did not break? These are the questions we should be > asking. From watching this list over the past years, I don't get the > feeling that the inclusion of these packages has been a significant > burden, but I may be wrong. I think that it's a worthwhile goal to make core smaller. It may not be a gigantic enterprise system with tens of thousands of source files, like some of us are accustomed to working on, but I think that it becomes easier to reason about the features and behaviour of core when it's smaller. >> updating packages within it is no longer a major issue, and (I hope) >> it will be clearer when something is a core issue vs. a package issue. > > I find this whole core vs package argument strange. If you ship Emacs > with Org, Gnus and CEDET, they are pa
Re: [O] Sync up the org in emacs master to org maint branch?
> AFAIU, the main motivation for the "drive to ELPA" comes from > developers of individual packages, not from those working on the core Not sure what you mean exactly by "drive to ELPA". If you mean "drive to put packages in GNU ELPA rather than in core", then this drive is linked to various aspects: - we can put packages in elpa.git without worrying too much about whether or not they are "good/important enough", contrary to packages in core. - packages in elpa.git don't come with the same kind of implicit guarantee of compatibility, availability, and maintenance as those in core. - sync'ing with an external upstream is easier in elpa.git since there's no need to pay attention to Emacs release schedule. - sync'ing with an external upstream can be made technically easier because we can use a separate branch for the package. Stefan
Re: [O] Sync up the org in emacs master to org maint branch?
> From: John Wiegley > Date: Sat, 11 Feb 2017 18:46:09 -0800 > Cc: Bastien Guerry , Emacs developers , > Phillip Lord , > emacs-org list , > Kaushal Modi > > I hear your other points, so I'm curious now as to what more people think > about this who work on Emacs core: Do you want more modularity with regard > ELPA, or does the monolithic model work better for you? AFAIU, the main motivation for the "drive to ELPA" comes from developers of individual packages, not from those working on the core (even though some package developers also work on core).
Re: [O] Sync up the org in emacs master to org maint branch?
> "DE" == David Engster writes: DE> I find this whole core vs package argument strange. If you ship Emacs with DE> Org, Gnus and CEDET, they are part of Emacs, so it's in the interest of DE> all Emacs developers that they work well, whether they use them or not. DE> The users won't care if they originate from a separate repo and are DE> considered a "package". So if Paul is determined to fix all occurences of DE> "compatilibity" in the doc-strings, why would he only do that for core? If pulling Emacs.git also pulls ELPA.git (as it should, since it's within our purview as well), then global search&replace should still affect all the files. I hear your other points, so I'm curious now as to what more people think about this who work on Emacs core: Do you want more modularity with regard ELPA, or does the monolithic model work better for you? -- John Wiegley GPG fingerprint = 4710 CF98 AF9B 327B B80F http://newartisans.com 60E1 46C4 BD1A 7AC1 4BA2
Re: [O] Sync up the org in emacs master to org maint branch?
> "PL" == Phillip Lord writes: PL> The alternative proposal is, essentially, to copy files into the Emacs PL> core build structure and move from there. PL> Reading the discussion reinforces my feeling that the latter is the wrong PL> approach, because it reinforces a distinction between packages on ELPA and PL> packages in core above and beyond the location that they are stored and PL> versioned. I can't see the advantage of doing this. PL> I will probably try to work a little further on my package.el solution, PL> although there seems little enthusiasm for this as the way forward. Correct, I don't want to move forward with the package.el option yet. Just copying files into the core build structure should be very lightweight, and it lets us focus on the ELPA separation first. I believe the package.el coupling is something we can examine at a later date. -- John Wiegley GPG fingerprint = 4710 CF98 AF9B 327B B80F http://newartisans.com 60E1 46C4 BD1A 7AC1 4BA2
Re: [O] Sync up the org in emacs master to org maint branch?
Edward John Steere writes: > - Suppose that Emacs 22.0 is the current release and Emacs 22.1 is then >released; CEDET is at > - we update a registry somewhere indicating that Emacs 22.0 works with > and 22.1 works with > > - When we make updates to CEDET we mark 22.1 as working with > but we don't change that reference >for 22.0 (or any older versions) > - When someone complains that there's a bug in CEDET for 22.0 we >indicate that it's no longer supported and that they should update >Emacs to receive updates > > This process would almost be the same as what you get just by bundling > CEDET with Emacs except that: > > - You can get the latest CEDET *if* you have the latest Emacs No. We have two branches: emacs-25 and master. The CEDET from master will usually not work on any 25.x version. > - The version of CEDET for any particular version of Emacs is as far as >CEDET got before the next release of Emacs came out > > If this is what you were thinking of then please could you elaborate on > what ended up being the problem which added more work. First off, CEDET is currently *not* a package, although that notion gets thrown around a lot. It is scattered across the Emacs code base: lisp/cedet, admin/grammars, etc/srecode, documentation, and test suite. All this needs to be packaged in a way so that it can be integrated into Emacs during a normal checkout. It needs to build and test in such a normal checkout, but also separately when installed from ELPA, including grammar compilation. And you need this twice: one for emacs-25, one for master, with the possiblity to merge between the two. Then there's this "registry". No one has said how that should work. "Submodules/Subtrees" are *not* a sufficient answer, they are just tools. People will need to say how the *workflow* should be, including such things like merges from stable, ChangeLog generation, AUTHORS, NEWS, creating release tarballs, and so on. Once someone has written this down *in detail*, we can discuss again if this indeed will make things simpler and reduce our workload. > I feel like there are aspects of CEDET which are still under > development. I hope so. >> Well, we cannot really discuss this since there's no real plan how this >> all should work. I can only speak from experience. > > We can still put ideas forward though. Haven't come up with anything > myself yet though. Yes, you can, but it has a cost. Once again, the CEDET merge is stalled, and we spend our time writing mails. I find this situation incredibly frustrating. >> How does CEDET, Gnus and Org affect the rest of Emacs? They strongly >> depend on Emacs "core" (whatever exactly that is), not the other way >> round. > > I believe that one of the intentions of the move is to enforce this so > we can't build bad dependencies -- am I wrong? I think other modes should use Semantic. > Perhaps I'm wrong, but to my mind the package approach would afford us > with more testing before we get to the point of another release of > Emacs. If you develop on Emacs 'master', you can use all the new features (like threading and FFI), but you loose testers on 25.x. A package won't change that. -David
Re: [O] Sync up the org in emacs master to org maint branch?
John Wiegley writes: >> "KM" == Kaushal Modi writes: > > KM> If we are able the release the new packaging method in emacs 26.x, then we > KM> can remove org from emacs master completely, but if not, then at least as > KM> backup we have a newer org version to go out with that release. > > For Emacs 26, I intend the new ELPA process to be in place, whereby "default" > packages can be developed separately, and declare a way to get slip-streamed > into the release tarball so users are unaware of the separate nature of their > development. > > The CEDET developers have agreed to support this, and it sounds like you are > willing to as well. If Lars is game, I'd like for Gnus to be third major > package we do this for initially. That will reduce considerably the number of > external files we track in Emacs.git. > > The precise technical details have yet to be worked out, but it shouldn't be > too difficult. Phillip Lord has already began advance work on alternatives, > and I've received offers of help from others to work on this new process. > > I think now is a good time to begin. The first step is to solidify what is > meant by "tarball EPLA", and the means of slip-streaming a package's contents. > This will require at least two bits: > > - Some form of declaration to indicate how external files should appear in > the tarball. In order for the first version of this scheme to be as low > impact as possible, this should probably be done with a sexp in a data > file, to be checked in alongside the EPLA.git import of the project. > > - changes to "make dist" to integrate these files, and setup autoloading so > their inclusion is transparent to end users. > > Please comment with your recommendations for the first, and supporting changes > for the second, if anyone has ideas. Phillip, how is your work on these coming > along? At the moment it isn't. My original plan, if you remember, to have emacs core load ELPA packages with package.el. This requires some minor re-working of package.el (so that the -Q doesn't block them), some Makefile fiddling and introducing some standards to test file location in ELPA, so that it's possible to run ELPA tests from within core. The alternative proposal is, essentially, to copy files into the Emacs core build structure and move from there. Reading the discussion reinforces my feeling that the latter is the wrong approach, because it reinforces a distinction between packages on ELPA and packages in core above and beyond the location that they are stored and versioned. I can't see the advantage of doing this. I will probably try to work a little further on my package.el solution, although there seems little enthusiasm for this as the way forward. Phil
Re: [O] Sync up the org in emacs master to org maint branch?
Edward John Steere writes: >> It's not like packages communicate with Emacs over a well >> defined RESTful interface. In other words: CEDET and Gnus are not >> loosely coupled, quite the opposite: they are extremely dependend on >> many obscure Emacs internals (not sure about Org). > > Shouldn't we be trying to avoid this situation? It's sure to come back > and bite us in the future. If we continue to develop a package > (wherever it ends up being developed) which is so tightly coupled that > any kind of re factoring in core becomes a nightmare, because we have to > consider the umpteen ways in which it'll break the package, then surely > that's a bad methodology to continue? I don't know what you have in mind. All I can say is: CEDET couldn't do what it does if we'd restrict ourselves to some subset of Emacs. >> As a consequence, we and also the Gnus guys decided to not do >> separate releases anymore. We used to provide CEDET for different >> Emacs versions, and it was a *huge* amount of work. I had a buildbot >> running with 7 or 8 Emacs versions to run the test suite, and things >> broke pretty regularly. Now you might say: fine, only release a >> package for current master. But let's say we put CEDET into >> ELPA. Emacs 26 gets released, and work on Emacs 27 starts. Now there >> are changes happening in Emacs 27 that require changes in CEDET, >> which make it incompatible with Emacs 26. So you have to create two >> packages: one for 26, one for 27? Not only is this confusing, but it >> most definitely increases my workload. > > I feel like this problem isn't intractable. I didn't say it's intractable. I just said it means more work for me. > We can mark some state of CEDET as being stable under the various > versions of Emacs (because it was at the time) and then only support > the current release for the latest package. This would most likely > require changes to package to ensure that you get an appropriate > version of the package when you download it. As I said: we did that. It was a huge amount of work. Please understand where I'm coming from: if you look through the CEDET history, you will see that in the past few years I almost exclusively did infrastructure work and no real coding. All I can say is: *I* won't do this anymore, and I don't want to be part in something which will increase grunt work. We did not make this decision lightly. But with the amount of developers we have, I'm convinced it's the right thing to do. > Consider the problem which our users currently face. The built in CEDET > is miles behind and missing very important bug fixes and features. The > upstream CEDET can be a real pain to setup, but it has the latest > features. This is not a good place to be. If we merge CEDET in and > only release it with the realeses of Emacs then we are heading for a > state where you only get the new features and bug fixes every time Emacs > is realesed, i.e. our users might have to wait up to two years to have > something fixed. This is also a bad place to be. Bug fixes can go with point release, which we should have every year. But yes, if you want the latest, greatest and buggiest, you'll have to use Emacs master. But that goes for a lot of Emacs features, so I don't see why it's particularly bad for CEDET. >>> We can arrange things so that a Git clone of Emacs includes pulling the >>> submodules (or trees, or ELPA.git, or what not) that are considered part of >>> "main Emacs development", including some of those batteries. I see this all >>> as >>> a process issue, and that "living in one Git repository" has just been an >>> implementation strategy to satisfy that process. >> >> Obviously, I'm very skeptical towards such an approach. Our tooling >> around Emacs development is already very intricate and only works >> because a few people work quietly behind the scenes to keep this all >> running. Introducing even more complexity through >> submodules/subtrees/whatever will do the opposite of what you want to >> achieve: it creates more work for those few people who manage the Emacs >> infrastructure. But I'd love to hear what for instance Glenn and Paul >> think about this. > > I'm interested in exploring more with regards to how the subtree > approach would work. In particular how it would impact the Makefiles > and build process. I don't think that introducing features like this > necessarily increases the burden of maintaining our tooling. If we get > it right it could reduce it. Well, we cannot really discuss this since there's no real plan how this all should work. I can only speak from experience. > I think that it's a worthwhile goal to make core smaller. It may not be > a gigantic enterprise system with tens of thousands of source files, > like some of us are accustomed to working on, but I think that it > becomes easier to reason about the features and behaviour of core when > it's smaller. How does CEDET, Gnus and Org affect the rest of Emacs? They stron
Re: [O] Sync up the org in emacs master to org maint branch?
> From: Edward John Steere > Date: Sun, 05 Feb 2017 11:03:31 +0200 > Cc: Bastien Guerry , Emacs developers , > Phillip Lord , > emacs-org list , > Kaushal Modi > > > It's not like packages communicate with Emacs over a well > > defined RESTful interface. In other words: CEDET and Gnus are not > > loosely coupled, quite the opposite: they are extremely dependend on > > many obscure Emacs internals (not sure about Org). > > Shouldn't we be trying to avoid this situation? It's sure to come back > and bite us in the future. If we continue to develop a package > (wherever it ends up being developed) which is so tightly coupled that > any kind of re factoring in core becomes a nightmare, because we have to > consider the umpteen ways in which it'll break the package, then surely > that's a bad methodology to continue? (I'm not suggesting that we > rewrite it to make it more loosely coupled, just that it seems like a > bad idea to continue allowing this going forward.) How would you go about not allowing this to go forward? I don't think I see any practical way to do that; do you? IMO, this is up to the package developers: if they want to depend less on the Emacs internals, and thus be more loosely coupled with a particular Emacs version, they will need to invest the extra effort for that, e.g., by providing some compatibility shims. IOW, this isn't an issue that can be solved once and for all by the way we develop the core or the packages, or by the way we integrate packages with core, the solution is on a different level. > > As a consequence, we > > and also the Gnus guys decided to not do separate releases anymore. We > > used to provide CEDET for different Emacs versions, and it was a *huge* > > amount of work. I had a buildbot running with 7 or 8 Emacs versions to > > run the test suite, and things broke pretty regularly. > > Now you might say: fine, only release a package for current master. But > > let's say we put CEDET into ELPA. Emacs 26 gets released, and work on > > Emacs 27 starts. Now there are changes happening in Emacs 27 that > > require changes in CEDET, which make it incompatible with Emacs 26. So > > you have to create two packages: one for 26, one for 27? Not only is > > this confusing, but it most definitely increases my workload. > > I feel like this problem isn't intractable. We can mark some state of > CEDET as being stable under the various versions of Emacs (because it > was at the time) and then only support the current release for the > latest package. This would most likely require changes to package to > ensure that you get an appropriate version of the package when you > download it. IF (and its a big "if") package developers want to be able to target more than the single Emacs version on a single branch of the Emacs repo, then they will need to provide at least 3 branches: . "development" branch that tracks the Emacs master branch and introduces exciting new features . "bugfix" branch that tracks the Emacs release branch without adding any new features . "stable" branch that is compatible with the Emacs release branch, but also introduces some new features, the ones that don't need core developments available only on the Emacs master branch I envision that some packages will want the above (or maybe even a more elaborate scheme), because they can afford that, and because their users expect that. These are mostly those packages whose developers welcome the move to put more of Emacs on ELPA. Other packages, and I guess CEDET is one of them, will not want to do this, because it adds too much work to an already complicated and time-consuming job. IOW, once again the solution is not part of the way we develop the core or integrate non-core packages, it's elsewhere. Bottom line, I think people are bothered by aspects of the "move to ELPA" process that are not supposed to be affected by that move in any way. They are aspects that need to be resolved on entirely different levels, and the resolution is up to the package maintainers. That includes a possible decision of the developers that some package will not move to ELPA; I don't think that if a package developers say they want to stay in emacs.git, they will be told to get out regardless.
Re: [O] Sync up the org in emacs master to org maint branch?
John Wiegley writes: > Many software projects have package ecosystems surrounding them that deal with > similar issues, and I can't, off-hand, think of cases where the answer was > "let's move some of those packages into core development to ease > ___". Just from the software I worked with over the years, I remember KDevelop, Typo3, Drupal and MediaWiki moving plugins to core. It's really not uncommon. I still use Drupal, and moving important plugins to core has made updates *much* less brittle. I just wish Jenkins would do the same already... -David
Re: [O] Sync up the org in emacs master to org maint branch?
John Wiegley writes: >> "DE" == David Engster writes: > > DE> So if you don't get convinced, we'll just move again, right? No big deal. > > I suppose I'm asking that of you, yes. Sorry, but I rather wait. > DE> You are insinuating that my motivation is to delegate CEDET development to > DE> the core Emacs developers. This is simply not true, and I don't see how my > DE> original mail could be interpreted like that. > > I didn't mean to insinuate anything; it seems we may have gotten off on the > wrong foot, my intention is to make your life easier, not harder. If all this > would do is make more work for people, it's not worth it. This will most definitely make things harder for me. > DE> So let me try again: What I find completely misguided is to move packages > DE> out of core *but still putting them into the release*. In other words, in > DE> my opinion there are really just two options that make sense: you either [+] > DE> keep a package in core, or you kick it out and don't ship it with the > DE> release. > > Why is this so? Right now I see the Emacs release as more than just releasing > Emacs core; it's more of a "batteries included" release, combining the editor > with lots of other default packages. It makes sense to me to move these > batteries outside the core repository, than to put them all together in the > same Git repository. For package developers, keeping up with Emacs has become much harder in recent years, as its development has accelerated (which is a good thing). It's not like packages communicate with Emacs over a well defined RESTful interface. In other words: CEDET and Gnus are not loosely coupled, quite the opposite: they are extremely dependend on many obscure Emacs internals (not sure about Org). As a consequence, we and also the Gnus guys decided to not do separate releases anymore. We used to provide CEDET for different Emacs versions, and it was a *huge* amount of work. I had a buildbot running with 7 or 8 Emacs versions to run the test suite, and things broke pretty regularly. Now you might say: fine, only release a package for current master. But let's say we put CEDET into ELPA. Emacs 26 gets released, and work on Emacs 27 starts. Now there are changes happening in Emacs 27 that require changes in CEDET, which make it incompatible with Emacs 26. So you have to create two packages: one for 26, one for 27? Not only is this confusing, but it most definitely increases my workload. > We can arrange things so that a Git clone of Emacs includes pulling the > submodules (or trees, or ELPA.git, or what not) that are considered part of > "main Emacs development", including some of those batteries. I see this all as > a process issue, and that "living in one Git repository" has just been an > implementation strategy to satisfy that process. Obviously, I'm very skeptical towards such an approach. Our tooling around Emacs development is already very intricate and only works because a few people work quietly behind the scenes to keep this all running. Introducing even more complexity through submodules/subtrees/whatever will do the opposite of what you want to achieve: it creates more work for those few people who manage the Emacs infrastructure. But I'd love to hear what for instance Glenn and Paul think about this. > Why do the split at all? Core becomes smaller, First off, the Emacs repo isn't that big w.r.t. the number of files. Secondly, while there surely is an inverse correlation between repo size and maintainability, I would argue that as long as the Big Three are well maintained, they are not the problem. When did CEDET, Gnus or Org ever significantly delay an Emacs release? When was an Emacs core developer ever forced to fix a critical thing in those packages he did not break? These are the questions we should be asking. From watching this list over the past years, I don't get the feeling that the inclusion of these packages has been a significant burden, but I may be wrong. > its future history less cluttered, That's actually a bit funny, since we gave up an uncluttered history when we switched to git's spaghetti DAG. > updating packages within it is no longer a major issue, and (I hope) > it will be clearer when something is a core issue vs. a package issue. I find this whole core vs package argument strange. If you ship Emacs with Org, Gnus and CEDET, they are part of Emacs, so it's in the interest of all Emacs developers that they work well, whether they use them or not. The users won't care if they originate from a separate repo and are considered a "package". So if Paul is determined to fix all occurences of "compatilibity" in the doc-strings, why would he only do that for core? People won't care if it's a CEDET doc-string, they'd just say "Teh Emacs people cant spell!1!!". It's no big deal for him anyway if everything is in one repo. Likewise when Stefan does some refactoring, like renaming 'defmethod' to 'cl-defmethod'. Doing this for all files
Re: [O] Sync up the org in emacs master to org maint branch?
> "DE" == David Engster writes: DE> So if you don't get convinced, we'll just move again, right? No big deal. I suppose I'm asking that of you, yes. DE> You are insinuating that my motivation is to delegate CEDET development to DE> the core Emacs developers. This is simply not true, and I don't see how my DE> original mail could be interpreted like that. I didn't mean to insinuate anything; it seems we may have gotten off on the wrong foot, my intention is to make your life easier, not harder. If all this would do is make more work for people, it's not worth it. DE> So let me try again: What I find completely misguided is to move packages DE> out of core *but still putting them into the release*. In other words, in DE> my opinion there are really just two options that make sense: you either DE> keep a package in core, or you kick it out and don't ship it with the DE> release. Why is this so? Right now I see the Emacs release as more than just releasing Emacs core; it's more of a "batteries included" release, combining the editor with lots of other default packages. It makes sense to me to move these batteries outside the core repository, than to put them all together in the same Git repository. We can arrange things so that a Git clone of Emacs includes pulling the submodules (or trees, or ELPA.git, or what not) that are considered part of "main Emacs development", including some of those batteries. I see this all as a process issue, and that "living in one Git repository" has just been an implementation strategy to satisfy that process. Why do the split at all? Core becomes smaller, its future history less cluttered, updating packages within it is no longer a major issue, and (I hope) it will be clearer when something is a core issue vs. a package issue. Also, people wanting to contribute new code to Emacs will not feel we're consigning them to disuse by saying it will go in ELPA. I've seen a few arguments already for things going into core, just to ensure more people would use it. DE> Say the Python developers would decide: Hey, many people like Django, so DE> let's just put their latest git master into our release and ship it. Would DE> you think that is a good approach? Some companies have actually done this. I remember when ActivePython bundled quite a few things, making it an attractive alternate to installing core Python (back when package management was still very poor in Python world). -- John Wiegley GPG fingerprint = 4710 CF98 AF9B 327B B80F http://newartisans.com 60E1 46C4 BD1A 7AC1 4BA2
Re: [O] Sync up the org in emacs master to org maint branch?
> "EJS" == Edward John Steere writes: EJS> What I think that we shouldn't lose sight of (if I may suggest it): is EJS> that packaging CEDET, Org Mode and other packages like them in a process EJS> which integrates them only when producing the tarball would serve to EJS> simplify things for everyone. Emacs core wouldn't be able to depend on EJS> packages which are more relevant to the users (and package developers) of EJS> Emacs, but these packages would still there like they always have been EJS> when one downloads binary Emacs. This is the very point I was hoping to make, thanks Edward. If it can simplify things for everyone, great; if it really will only complicate things, not so great. Many software projects have package ecosystems surrounding them that deal with similar issues, and I can't, off-hand, think of cases where the answer was "let's move some of those packages into core development to ease ___". This is why I find it a bit surprising that some feel so strongly about keeping these large, external packages in core. -- John Wiegley GPG fingerprint = 4710 CF98 AF9B 327B B80F http://newartisans.com 60E1 46C4 BD1A 7AC1 4BA2
Re: [O] Sync up the org in emacs master to org maint branch?
> From: David Engster > Cc: emacs-de...@gnu.org, b...@gnu.org, emacs-orgmode@gnu.org, > kaushal.m...@gmail.com, la...@gnus.org, phillip.l...@russet.org.uk > Date: Thu, 02 Feb 2017 21:57:02 +0100 > > > Ask the package maintainers, they see significant advantages in being > > able to release interim versions independent of Emacs releases. > > But we are discussing moving CEDET, Gnus and Org out of Emacs core, and > at least the former two do not plan to provide updates between Emacs > releases. That's fine with me, I have nothing against leaving some packages in emacs.git if their maintainers so wish. I was replying to a single aspect of this discussion: the expectation that packages which will be moved to ELPA will necessarily and inevitably suffer in terms of QA and the developers' attention they receive. I'm saying that AFAIU this is not supposed to happen, for the reasons I presented.
Re: [O] Sync up the org in emacs master to org maint branch?
Eli Zaretskii writes: >> From: David Engster >> Cc: Lars Ingebrigtsen , b...@gnu.org, >> emacs-de...@gnu.org, phillip.l...@russet.org.uk, > >> which implies that they are not supposed to be present at a "normal" >> checkout. > > I don't see how it implies that. Release tarballs are prepared > specially for a release, so their build procedures don't have to be > identical to building from Git. > >> Otherwise, what difference would it make? > > Ask the package maintainers, they see significant advantages in being > able to release interim versions independent of Emacs releases. But we are discussing moving CEDET, Gnus and Org out of Emacs core, and at least the former two do not plan to provide updates between Emacs releases. We already had this discussion in 2015, where John was pretty determined to remove CEDET from core: https://lists.gnu.org/archive/html/emacs-devel/2015-11/msg00877.html -David
Re: [O] Sync up the org in emacs master to org maint branch?
> From: David Engster > Cc: Lars Ingebrigtsen , b...@gnu.org, emacs-de...@gnu.org, > phillip.l...@russet.org.uk, emacs-orgmode@gnu.org, kaushal.m...@gmail.com > Date: Thu, 02 Feb 2017 18:47:49 +0100 > > > I believe the intent is to make it so that checking out and building > > Emacs also checks out and builds all the packages that are intended to > > be part of a release tarball. If we indeed do that this way, there > > will be no difference, QA-wise, between core packages and ELPA > > packages that are logically part of an Emacs release. > > That's not how I understood it. I hope you have misunderstood, and not me. > It was always said that Emacs must not depend on those ELPA packages > that are shipped with the release I'm talking about building Emacs from Git, not from a release tarball. For the latter, you are right, and we are in agreement. But that's not relevant for the issue at hand, which appears to be the attention which such ELPA packages will get from Emacs developers. For that, building a fresh checkout should also build the latest versions from ELPA, from a branch that the package maintainers will designate as the equivalent of the Emacs master branch. > which implies that they are not supposed to be present at a "normal" > checkout. I don't see how it implies that. Release tarballs are prepared specially for a release, so their build procedures don't have to be identical to building from Git. > Otherwise, what difference would it make? Ask the package maintainers, they see significant advantages in being able to release interim versions independent of Emacs releases. But we are not talking about that aspect, we are talking about the parallel development of core and the packages.
Re: [O] Sync up the org in emacs master to org maint branch?
> So let me try again: What I find completely misguided is to move > packages out of core *but still putting them into the release*. In other > words, in my opinion there are really just two options that make sense: > you either keep a package in core, or you kick it out and don't ship it > with the release. AFAIK I am the one who started with the idea of including ELPA packages in the tarball, so I feel like I have to answer here: The idea was not to push packages out to elpa.git only to then include them in the tarball. Instead, the idea was just to allow some packages into the tarball without having to move them from elpa.git to emacs.git. Their inclusion in the tarball is simply a way to make that package more widely available. E.g., I'd like to include company-mode in Emacs's tarball since this kind of completion is very popular (and also because I think Emacs as a whole suffers from the dilution of effort between company-mode and auto-complete, and I have the delusion that integrating one of the two into the tarball would help solve this problem). We could do it by moving company-mode to emacs.git, but IIRC the maintainer of company-mode would prefer to keep it in elpa.git, so the other option is to bundle that ELPA package into the tarball. I never thought of this as a way to move stuff out of emacs.git. Yet, there is one case where I think it would be beneficial to move a package out of emacs.git only to re-integrate it back into the tarball: Org mode. Org mode is really managed as a separate package (which sadly isn't even present in elpa.git) that happens to be bundled in the tarball: almost no code in the rest of Emacs cares about Org, and the Org code is not tightly dependent on the specific Emacs version with which it is shipped. So IMO it would make more sense to keep Org outside of emacs.git (and save us the trouble of syncing), tho we'd still want to distribute it in Emacs's tarball, since it's an extremely popular package. Stefan
Re: [O] Sync up the org in emacs master to org maint branch?
Eli Zaretskii writes: >> From: Lars Ingebrigtsen >> Date: Thu, 02 Feb 2017 13:10:07 +0100 >> Cc: Bastien Guerry , Emacs developers , > >> Phillip Lord , >> emacs-org list , >> Kaushal Modi >> >> If Django had traditionally always been distributed along with Python, >> and maintained in the Python repo, and the suggestion now would be to >> move Django to a part of the Python repo that very few developers look >> at, but Django would continue to be distributed with Python, and all >> Django bug reports would continue to go to the Python bug repository, >> and Python developers would continue to be responsible for QA and bug >> fixing of Django. > > I believe the intent is to make it so that checking out and building > Emacs also checks out and builds all the packages that are intended to > be part of a release tarball. If we indeed do that this way, there > will be no difference, QA-wise, between core packages and ELPA > packages that are logically part of an Emacs release. That's not how I understood it. It was always said that Emacs must not depend on those ELPA packages that are shipped with the release, which implies that they are not supposed to be present at a "normal" checkout. Otherwise, what difference would it make? (Besides requiring more cumbersome tooling to make this all work.) -David
Re: [O] Sync up the org in emacs master to org maint branch?
> From: Lars Ingebrigtsen > Date: Thu, 02 Feb 2017 13:10:07 +0100 > Cc: Bastien Guerry , Emacs developers , > Phillip Lord , > emacs-org list , > Kaushal Modi > > If Django had traditionally always been distributed along with Python, > and maintained in the Python repo, and the suggestion now would be to > move Django to a part of the Python repo that very few developers look > at, but Django would continue to be distributed with Python, and all > Django bug reports would continue to go to the Python bug repository, > and Python developers would continue to be responsible for QA and bug > fixing of Django. I believe the intent is to make it so that checking out and building Emacs also checks out and builds all the packages that are intended to be part of a release tarball. If we indeed do that this way, there will be no difference, QA-wise, between core packages and ELPA packages that are logically part of an Emacs release.
Re: [O] Sync up the org in emacs master to org maint branch?
John Wiegley writes: >> "DE" == David Engster writes: > > DE> Also, I currently have no idea how to continue with CEDET, as the future > DE> where development should happen is unclear, and I get the feeling we're > DE> just waisting our time with the ongoing merge. > > Until the dust has settled, please proceed, assuming nothing has > changed. Move your primary development into Emacs.git. > > The changes I'm proposing don't have to happen tomorrow, and I can still be > convinced they're unnecessary. So if you don't get convinced, we'll just move again, right? No big deal. > My gut tells me, however, that we're supporting an unnecessarily > monolithic development model for no better reason than "we're used to > it". > > In fact, what we're doing feels like if Python included Django in its main > repository, just to solve Django's problems of compatibility, testing, and > making its bugs known to the main Python developers. You are insinuating that my motivation is to delegate CEDET development to the core Emacs developers. This is simply not true, and I don't see how my original mail could be interpreted like that. So let me try again: What I find completely misguided is to move packages out of core *but still putting them into the release*. In other words, in my opinion there are really just two options that make sense: you either keep a package in core, or you kick it out and don't ship it with the release. Say the Python developers would decide: Hey, many people like Django, so let's just put their latest git master into our release and ship it. Would you think that is a good approach? -David
Re: [O] Sync up the org in emacs master to org maint branch?
John Wiegley writes: > OK, to continue the analogy, what is the right answer? Technically it > doesn't seem as though Django belongs there, even if culturally it > sounds hard to separate. Should it stay indefinitely, or should the > development model change? If somebody genuinely offered to take over, say, rmail, and maintain it separately, and handle bug reports, and, like, be the maintainer, that would be a help. Great, go ahead, and put the resulting thing in ELPA. But nobody has made that offer? Or have they, and I just missed it? I fail to see how just shuffling rmail from Emacs to ELPA helps us in any way with the maintainership. Instead it creates an extra burden on us, since we (in addition to all the normal maintainership) will also have to consider Emacs version compatibility. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no
Re: [O] Sync up the org in emacs master to org maint branch?
On 02.02.2017 14:10, Lars Ingebrigtsen wrote: If Django had traditionally always been distributed along with Python, and maintained in the Python repo, I'm pretty sure the first versions of Emacs came without Gnus. Later, it got bundled. Some time after that, Org and CEDET joined too. All of that was before we got ELPA and with it, a very easy way for users to install third-party packages. and the suggestion now would be to move Django to a part of the Python repo that very few developers look at, I've never looked at the Gnus source code either, even inside the Emacs repository. but Django would continue to be distributed with Python, and all Django bug reports would continue to go to the Python bug repository, and Python developers would continue to be responsible for QA and bug fixing of Django. You could introduce a separate bug tracker, if that helps.
Re: [O] Sync up the org in emacs master to org maint branch?
> "LI" == Lars Ingebrigtsen writes: LI> If Django had traditionally always been distributed along with Python, and LI> maintained in the Python repo, and the suggestion now would be to move LI> Django to a part of the Python repo that very few developers look at, but LI> Django would continue to be distributed with Python, and all Django bug LI> reports would continue to go to the Python bug repository, and Python LI> developers would continue to be responsible for QA and bug fixing of LI> Django. OK, to continue the analogy, what is the right answer? Technically it doesn't seem as though Django belongs there, even if culturally it sounds hard to separate. Should it stay indefinitely, or should the development model change? -- John Wiegley GPG fingerprint = 4710 CF98 AF9B 327B B80F http://newartisans.com 60E1 46C4 BD1A 7AC1 4BA2
Re: [O] Sync up the org in emacs master to org maint branch?
John Wiegley writes: > In fact, what we're doing feels like if Python included Django in its main > repository, just to solve Django's problems of compatibility, testing, and > making its bugs known to the main Python developers. I guess that would be a fair comparison. If Django had traditionally always been distributed along with Python, and maintained in the Python repo, and the suggestion now would be to move Django to a part of the Python repo that very few developers look at, but Django would continue to be distributed with Python, and all Django bug reports would continue to go to the Python bug repository, and Python developers would continue to be responsible for QA and bug fixing of Django. See? It's totally the same thing. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no
Re: [O] Sync up the org in emacs master to org maint branch?
> "DE" == David Engster writes: DE> Also, I currently have no idea how to continue with CEDET, as the future DE> where development should happen is unclear, and I get the feeling we're DE> just waisting our time with the ongoing merge. Until the dust has settled, please proceed, assuming nothing has changed. Move your primary development into Emacs.git. The changes I'm proposing don't have to happen tomorrow, and I can still be convinced they're unnecessary. My gut tells me, however, that we're supporting an unnecessarily monolithic development model for no better reason than "we're used to it". In fact, what we're doing feels like if Python included Django in its main repository, just to solve Django's problems of compatibility, testing, and making its bugs known to the main Python developers. -- John Wiegley GPG fingerprint = 4710 CF98 AF9B 327B B80F http://newartisans.com 60E1 46C4 BD1A 7AC1 4BA2
Re: [O] Sync up the org in emacs master to org maint branch?
On Wed, February 1, 2017 6:51 pm, Lars Ingebrigtsen wrote: > Aaron Ecay writes: > > >> If ELPA made available (on the server for downloading, and in the >> client for installing) old versions of packages, then users could always >> be offered the latest compatible version, but not later incompatible >> ones. > > I don't think having Emacs developers fixing bugs in a number of > different versions of a package sounds like a good way to spend time, > either. Well, they do that anyway. Org-mode, in Emacs core is quite a way behind org-mode latest. That's the start point of this discussion. And, for packages which are maintained only in core (not synced to an external repo like org), well, we currently have Emacs-25 and Emacs-26 in active development. Currently development of Emacs core with slow releases contributes to this, because the old versions of org remain around and in active use for a long period of time. In the case of org, this was exacerbated when it changed the features it provides, meaning that upgrading to ELPA worked imperfectly. There is even a reasonable possibility that with a smaller core, and faster releases, fewer changes would happen in core, so supporting multiple versions might well become easier because the differences would be smaller. These are complex questions, and it's hard to get evidence one way or another. But many other systems do support numerous small packages, building up into a greater whole: consider npm, CPAN, CRAN, CTAN or, indeed, debian. And, yes, sometimes we do end up in version hell, but mostly we don't. Phil
Re: [O] Sync up the org in emacs master to org maint branch?
Aaron Ecay writes: > If ELPA made available (on the server for downloading, and in the > client for installing) old versions of packages, then users could > always be offered the latest compatible version, but not later > incompatible ones. I don't think having Emacs developers fixing bugs in a number of different versions of a package sounds like a good way to spend time, either. We're not infinite monkeys, I'm sad to say. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no
Re: [O] Sync up the org in emacs master to org maint branch?
Stephen Leake writes: >> I'm massively unenthusiastic about this future. Things in ELPA has to >> be backwards-and-forwards compatible with a wide Emacs version range, > > no, they don't. > > That is one possible policy, but each package decides for itself whether > to follow it. You can have > > ;; package-requires: ((emacs "25.2")) > > if you want. That's technically correct. (The best kind of correct, as they say.) But for a package archive to be useful, it should support a range of Emacs releases, otherwise users will be sad. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no
Re: [O] Sync up the org in emacs master to org maint branch?
Hi Lars, 2017ko urtarrilak 31an, Lars Ingebrigtsen-ek idatzi zuen: > > John Wiegley writes: > >> We're moving toward a future where Emacs.git will represent "core >> Emacs", and only contain what core needs (plus a few historical bits, >> I'm sure). There should be no argument for keeping a project in core >> just to gain auxiliary benefits. > > I'm massively unenthusiastic about this future. Things in ELPA has to > be backwards-and-forwards compatible with a wide Emacs version range, This seems like a technical limitation of ELPAʼs current implementation, rather than a conceptual impossibility. If ELPA made available (on the server for downloading, and in the client for installing) old versions of packages, then users could always be offered the latest compatible version, but not later incompatible ones. Developers would have to be a little more diligent about declaring their packagesʼ dependencies on emacs major versions (or on other packages, if they depend on parts of core that have migrated to ELPA), but this would be a small hurdle. Aaron PS Speaking of dependency management, Iʼd be more worried that this kind of approach will accelerate the advent of dependency hell with ELPA packages...but I think all package repos have to confront that problem eventually. So Iʼd file that thought under “inevitable growing pains” rather than “arguments against”. -- Aaron Ecay
Re: [O] Sync up the org in emacs master to org maint branch?
John Wiegley writes: >> "DE" == David Engster writes: > > DE> It is a mistake because you are creating more moving targets and bring > DE> them together very late in the release process. This reduces the amount of > DE> testing that is done for those packages, so bugs will be noticed later and > DE> the quality of the relases suffer. It moves even more work into the > DE> RC-phase, which is already crowded and where people who can fix those bugs > DE> might not be readily available. It removes those packages from Emacs CI, > DE> so that breakages due to changes in core are not immediately noticed, and > DE> often times they have to be fixed not by those who created the breakage, > DE> but by those who notice them. > > These are issues to be fixed in the way ELPA integrates with our development > process. I recognize today's ELPA may have these drawbacks, but I believe they > can be fixed. This really has not much to do with how ELPA works. My points above are about the underlying concept you are proposing: moving packages out of Emacs core and hence removing them from current Emacs development, but still bundling them with the release. It's a have-and-eat-cake concept. > We're moving toward a future where Emacs.git will represent "core Emacs", and > only contain what core needs (plus a few historical bits, I'm sure). There > should be no argument for keeping a project in core just to gain auxiliary > benefits. Of the points I raise above, which fall under "just to gain auxiliary benefits"? I'm honestly confused. I'm specifically talking about the quality of the Emacs relases. Also, I currently have no idea how to continue with CEDET, as the future where development should happen is unclear, and I get the feeling we're just waisting our time with the ongoing merge. -David
Re: [O] Sync up the org in emacs master to org maint branch?
> It is a mistake because you are creating more moving targets and bring > them together very late in the release process. This reduces the amount > of testing that is done for those packages, so bugs will be noticed > later and the quality of the relases suffer. It moves even more work > into the RC-phase, which is already crowded and where people who can fix > those bugs might not be readily available. It removes those packages > from Emacs CI, so that breakages due to changes in core are not > immediately noticed, and often times they have to be fixed not by those > who created the breakage, but by those who notice them. Indeed, moving something to elpa.git has benefits but also downsides. Another downside is that ELPA packages need to be compatible with various Emacs releases, whereas bundled code can happily ignore those compatibility issues. Of course, the reverse is true as well: if CEDET-core is an ELPA package, then CEDET-client packages don't need to be compatible with older CEDET-core. I think the most important issue is to avoid duplicating branches. Moving (parts of) CEDET to elpa.git is/was mostly motivated by the desire to avoid the difficulties of sync'ing between the Emacs code and the upstream code, since the elpa.git branch could hopefully evolve in-sync with the upstream (or could even *be* the uptream), whereas that was more difficult for the code in emacs.git. If the CEDET code in emacs.git becomes "the upstream", then moving it to elpa.git is much less interesting. Stefan
Re: [O] Sync up the org in emacs master to org maint branch?
David Engster writes: >> (Except perhaps CEDET. There seems to be a lot of merging problems >> there.) > > This is precisely why we're currently moving all development to Emacs > and abandon the external repo. At least we were planning to... Oh, great. One less thing that would be helped by moving to ELPA. :-) -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no
Re: [O] Sync up the org in emacs master to org maint branch?
Lars Ingebrigtsen writes: > The question is: What is the most effective way for Emacs developers to > spend their time? I can't really see that anybody has made the case > that shifting stuff from Emacs core to ELPA will mean less work for... > well, anybody. > > (Except perhaps CEDET. There seems to be a lot of merging problems > there.) This is precisely why we're currently moving all development to Emacs and abandon the external repo. At least we were planning to... -David
Re: [O] Sync up the org in emacs master to org maint branch?
John Wiegley writes: > So far, all of these arguments against a tighter development integration with > ELPA have been predicated on the way that ELPA is used today. ELPA is under > our control; we can adjust our process to suit the needs of Emacs development. Yes, but external packages lose much of their value if they aren't developed in a compatible manner. > LI> Emacs doesn't seem to have a massive surfeit of developers, so I wonder > LI> where this plan comes from. > > It comes from the desire to decouple the development of large, mostly external > projects, from core Emacs. They don't belong in Emacs.git. But you're talking about coupling ELPA tighter with core Emacs, too. "They don't belong" isn't really much of an argument here. The question is: What is the most effective way for Emacs developers to spend their time? I can't really see that anybody has made the case that shifting stuff from Emacs core to ELPA will mean less work for... well, anybody. (Except perhaps CEDET. There seems to be a lot of merging problems there.) -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no
Re: [O] Sync up the org in emacs master to org maint branch?
> "LI" == Lars Ingebrigtsen writes: LI> I'm massively unenthusiastic about this future. Things in ELPA has to be LI> backwards-and-forwards compatible with a wide Emacs version range, which LI> makes maintaining things much more work. When you develop things in "Emacs LI> core", you have one specific target and can make large internal changes LI> without these considerations. So far, all of these arguments against a tighter development integration with ELPA have been predicated on the way that ELPA is used today. ELPA is under our control; we can adjust our process to suit the needs of Emacs development. LI> Emacs doesn't seem to have a massive surfeit of developers, so I wonder LI> where this plan comes from. It comes from the desire to decouple the development of large, mostly external projects, from core Emacs. They don't belong in Emacs.git. -- John Wiegley GPG fingerprint = 4710 CF98 AF9B 327B B80F http://newartisans.com 60E1 46C4 BD1A 7AC1 4BA2
Re: [O] Sync up the org in emacs master to org maint branch?
John Wiegley writes: > We're moving toward a future where Emacs.git will represent "core > Emacs", and only contain what core needs (plus a few historical bits, > I'm sure). There should be no argument for keeping a project in core > just to gain auxiliary benefits. I'm massively unenthusiastic about this future. Things in ELPA has to be backwards-and-forwards compatible with a wide Emacs version range, which makes maintaining things much more work. When you develop things in "Emacs core", you have one specific target and can make large internal changes without these considerations. Emacs doesn't seem to have a massive surfeit of developers, so I wonder where this plan comes from. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no
Re: [O] Sync up the org in emacs master to org maint branch?
> "DE" == David Engster writes: DE> It is a mistake because you are creating more moving targets and bring DE> them together very late in the release process. This reduces the amount of DE> testing that is done for those packages, so bugs will be noticed later and DE> the quality of the relases suffer. It moves even more work into the DE> RC-phase, which is already crowded and where people who can fix those bugs DE> might not be readily available. It removes those packages from Emacs CI, DE> so that breakages due to changes in core are not immediately noticed, and DE> often times they have to be fixed not by those who created the breakage, DE> but by those who notice them. These are issues to be fixed in the way ELPA integrates with our development process. I recognize today's ELPA may have these drawbacks, but I believe they can be fixed. We're moving toward a future where Emacs.git will represent "core Emacs", and only contain what core needs (plus a few historical bits, I'm sure). There should be no argument for keeping a project in core just to gain auxiliary benefits. -- John Wiegley GPG fingerprint = 4710 CF98 AF9B 327B B80F http://newartisans.com 60E1 46C4 BD1A 7AC1 4BA2
Re: [O] Sync up the org in emacs master to org maint branch?
John Wiegley writes: >> "DE" == David Engster writes: > > DE> This is a misunderstanding. I said I wanted to move support for certain > DE> languages and project types into ELPA, not CEDET core. I'm still of the > DE> opinion that moving it completely to ELPA is a mistake. > > It would only be a mistake if other parts of core need to use it. If only > users make use of it, then having it ELPA will be invisible to them. It is a mistake because you are creating more moving targets and bring them together very late in the release process. This reduces the amount of testing that is done for those packages, so bugs will be noticed later and the quality of the relases suffer. It moves even more work into the RC-phase, which is already crowded and where people who can fix those bugs might not be readily available. It removes those packages from Emacs CI, so that breakages due to changes in core are not immediately noticed, and often times they have to be fixed not by those who created the breakage, but by those who notice them. -David
Re: [O] Sync up the org in emacs master to org maint branch?
> "DE" == David Engster writes: DE> This is a misunderstanding. I said I wanted to move support for certain DE> languages and project types into ELPA, not CEDET core. I'm still of the DE> opinion that moving it completely to ELPA is a mistake. It would only be a mistake if other parts of core need to use it. If only users make use of it, then having it ELPA will be invisible to them. -- John Wiegley GPG fingerprint = 4710 CF98 AF9B 327B B80F http://newartisans.com 60E1 46C4 BD1A 7AC1 4BA2
Re: [O] Sync up the org in emacs master to org maint branch?
John Wiegley writes: >> "KM" == Kaushal Modi writes: > > KM> If we are able the release the new packaging method in emacs 26.x, then we > KM> can remove org from emacs master completely, but if not, then at least as > KM> backup we have a newer org version to go out with that release. > > For Emacs 26, I intend the new ELPA process to be in place, whereby "default" > packages can be developed separately, and declare a way to get slip-streamed > into the release tarball so users are unaware of the separate nature of their > development. > > The CEDET developers have agreed to support this, and it sounds like you are > willing to as well. This is a misunderstanding. I said I wanted to move support for certain languages and project types into ELPA, not CEDET core. I'm still of the opinion that moving it completely to ELPA is a mistake. > If Lars is game, I'd like for Gnus to be third major > package we do this for initially. That will reduce considerably the number of > external files we track in Emacs.git. CEDET and Gnus are not external anymore. Both have abandonded their upstream repos and moved to Emacs, because the faster development of Emacs has made that necessary. -David
Re: [O] Sync up the org in emacs master to org maint branch?
> "KM" == Kaushal Modi writes: KM> If we are able the release the new packaging method in emacs 26.x, then we KM> can remove org from emacs master completely, but if not, then at least as KM> backup we have a newer org version to go out with that release. For Emacs 26, I intend the new ELPA process to be in place, whereby "default" packages can be developed separately, and declare a way to get slip-streamed into the release tarball so users are unaware of the separate nature of their development. The CEDET developers have agreed to support this, and it sounds like you are willing to as well. If Lars is game, I'd like for Gnus to be third major package we do this for initially. That will reduce considerably the number of external files we track in Emacs.git. The precise technical details have yet to be worked out, but it shouldn't be too difficult. Phillip Lord has already began advance work on alternatives, and I've received offers of help from others to work on this new process. I think now is a good time to begin. The first step is to solidify what is meant by "tarball EPLA", and the means of slip-streaming a package's contents. This will require at least two bits: - Some form of declaration to indicate how external files should appear in the tarball. In order for the first version of this scheme to be as low impact as possible, this should probably be done with a sexp in a data file, to be checked in alongside the EPLA.git import of the project. - changes to "make dist" to integrate these files, and setup autoloading so their inclusion is transparent to end users. Please comment with your recommendations for the first, and supporting changes for the second, if anyone has ideas. Phillip, how is your work on these coming along? -- John Wiegley GPG fingerprint = 4710 CF98 AF9B 327B B80F http://newartisans.com 60E1 46C4 BD1A 7AC1 4BA2
Re: [O] Sync up the org in emacs master to org maint branch?
> "EZ" == Eli Zaretskii writes: EZ> AFAIR, we have never released a major version so quickly, and I don't see EZ> why this one would be different. A year at least, I'd say, probably more. This was my feeling as well. -- John Wiegley GPG fingerprint = 4710 CF98 AF9B 327B B80F http://newartisans.com 60E1 46C4 BD1A 7AC1 4BA2
Re: [O] Sync up the org in emacs master to org maint branch?
> From: Kaushal Modi > Date: Thu, 26 Jan 2017 16:01:24 + > Cc: emacs-org list , > Stefan Monnier , > Rasmus , Emacs developers > > I don't believe that the target date has yet been set for releasing 26.1. We > are currently on the release > candidate testing stage of 25.2. So I will not be surprised if 26.1 get > released even 3 months or 6 months from > now. Eli? John? AFAIR, we have never released a major version so quickly, and I don't see why this one would be different. A year at least, I'd say, probably more.
Re: [O] Sync up the org in emacs master to org maint branch?
On Thu, Jan 26, 2017 at 10:39 AM Kyle Meyer wrote: > Kaushal Modi writes: > > > On Thu, Jan 26, 2017 at 1:19 AM Kyle Meyer wrote: > > > >> We'd want at least one more release from maint, I think, so that'd be > >> 9.0.5. > > > > Would it be OK to sync the current stable 9.0.4, > > I don't think that's a good idea. Since 9.0.4, I've backported one > remaining commit from Emacs, I've adjusted :version in defcustoms to the > appropriate version for a sync targeting Emacs's master, and I've > cleaned up the spacing in a few places so that all the files pass > Emacs's pre-commit check. > Makes sense. Thanks for the explanation. > > We never know, we might end up with even higher stable releases by the > > time emacs 26.1 is released. > > I suspect we will, given that 9.0.1 -> 9.0.4 have all been released > since this November. > I don't believe that the target date has yet been set for releasing 26.1. We are currently on the release candidate testing stage of 25.2. So I will not be surprised if 26.1 get released even 3 months or 6 months from now. Eli? John? -- Kaushal Modi
Re: [O] Sync up the org in emacs master to org maint branch?
Kaushal Modi writes: > On Thu, Jan 26, 2017 at 1:19 AM Kyle Meyer wrote: > >> We'd want at least one more release from maint, I think, so that'd be >> 9.0.5. > > Would it be OK to sync the current stable 9.0.4, I don't think that's a good idea. Since 9.0.4, I've backported one remaining commit from Emacs, I've adjusted :version in defcustoms to the appropriate version for a sync targeting Emacs's master, and I've cleaned up the spacing in a few places so that all the files pass Emacs's pre-commit check. > and keep on updating with each stable release as time comes? Yes, that should be done. > We never know, we might end up with even higher stable releases by the > time emacs 26.1 is released. I suspect we will, given that 9.0.1 -> 9.0.4 have all been released since this November. -- Kyle
Re: [O] Sync up the org in emacs master to org maint branch?
On Thu, Jan 26, 2017 at 1:19 AM Kyle Meyer wrote: > Rasmus writes: > We'd want at least one more release from maint, I think, so that'd be > 9.0.5. > Would it be OK to sync the current stable 9.0.4, and keep on updating with each stable release as time comes? We never know, we might end up with even higher stable releases by the time emacs 26.1 is released. I have been using emacs master and org master for ages now without any issues. So emacs master + org 9.0.4 should not cause any serious problems. The major issues I forsee are the few backward incompatible changes people might have to make when org changes from 8.2.x to 9.x (though all those changes are documented in ORG-NEWS). On Thu, Jan 26, 2017 at 8:46 AM Rasmus wrote: > So would now be a good time to sync the Emacs master? I guess the > appropriate way would be to make a new branch that can eventually be > merged. > Going by the same argument as above, do you think that merging org maint into emacs master directly is that risky? org master + emacs master has been super-stable for me. On Thu, Jan 26, 2017 at 9:22 AM Stefan Monnier wrote: > > So would now be a good time to sync the Emacs master? > > On `master`, "now" is pretty much *always* a good time to sync. > More specifically, it's better to always keep `master` in sync with the > upstream (applies not just to Org). > > "sync early, sync often", > +1! -- Kaushal Modi
Re: [O] Sync up the org in emacs master to org maint branch?
Rasmus writes: > Hi, Hi, > *AFAIR* it was too late and would thus not have received enough test from > the general Emacs community. org-mode comes with some hundred ert test cases. It would be great if they'll land also in the Emacs master branch; this will give much more testing. > Thanks, > Rasmus Best regards, Michael.
Re: [O] Sync up the org in emacs master to org maint branch?
Rasmus writes: > Kaushal Modi writes: [...] >> As a precaution that that does not repeat when emacs 26.x is released, >> should the org version in emacs master be synced with the now latest stable >> org version 9.0.4? > > Yes. We'd want at least one more release from maint, I think, so that'd be 9.0.5. >> If we are able the release the new packaging method in emacs 26.x, then we >> can remove org from emacs master completely, but if not, then at least as >> backup we have a newer org version to go out with that release. > > What is the current status? I am a bit confused about the policy at this > point. I'm happy to try to update master to 9.0.4, but I was somehow > under the impression that we were waiting for a solution to include ELPA > packages in the Emacs tarball. Rasmus, I've pushed a branch (emacs-sync) to the Org repo that applies several patches on top of maint. These make a few changes to org.texi and orgcard.tex that I think are appropriate for the sync. Feel free to ignore the branch if it's not helpful. Thanks. -- Kyle
Re: [O] Sync up the org in emacs master to org maint branch?
> From: Rasmus > Date: Wed, 25 Jan 2017 17:54:48 +0100 > Cc: emacs-de...@gnu.org > > What is the current status? I am a bit confused about the policy at this > point. I'm happy to try to update master to 9.0.4, but I was somehow > under the impression that we were waiting for a solution to include ELPA > packages in the Emacs tarball. That could take a while, AFAIU, so I wouldn't recommend delaying the update on that behalf. Thanks.
Re: [O] Sync up the org in emacs master to org maint branch?
Hi, Kaushal Modi writes: > I am aware that in emacs 26, there are plans to change the way in how > certain packages can be moved out of the emacs master and still can be > installed seamlessly using the tarballs of those. Indeed. > Currently the org-mode version in emacs master is 8.2.10 and that it too > old (> 2 years, ref: http://orgmode.org/cgit.cgi/org-mode.git/refs/). The > current stable version of org-mode is 9.0.4 (released yesterday). > > At the time of releasing emacs 25.1, the org-mode in emacs master could > have been synced up with the then 1.5 years newer and stable version of org > (probably 8.3.5 or 8.3.6). But that got missed due to some reason. *AFAIR* it was too late and would thus not have received enough test from the general Emacs community. > As a precaution that that does not repeat when emacs 26.x is released, > should the org version in emacs master be synced with the now latest stable > org version 9.0.4? Yes. > If we are able the release the new packaging method in emacs 26.x, then we > can remove org from emacs master completely, but if not, then at least as > backup we have a newer org version to go out with that release. What is the current status? I am a bit confused about the policy at this point. I'm happy to try to update master to 9.0.4, but I was somehow under the impression that we were waiting for a solution to include ELPA packages in the Emacs tarball. Thanks, Rasmus -- However beautiful the theory, one should occasionally look at the evidence
[O] Sync up the org in emacs master to org maint branch?
Hi all, I am aware that in emacs 26, there are plans to change the way in how certain packages can be moved out of the emacs master and still can be installed seamlessly using the tarballs of those. Currently the org-mode version in emacs master is 8.2.10 and that it too old (> 2 years, ref: http://orgmode.org/cgit.cgi/org-mode.git/refs/). The current stable version of org-mode is 9.0.4 (released yesterday). At the time of releasing emacs 25.1, the org-mode in emacs master could have been synced up with the then 1.5 years newer and stable version of org (probably 8.3.5 or 8.3.6). But that got missed due to some reason. As a precaution that that does not repeat when emacs 26.x is released, should the org version in emacs master be synced with the now latest stable org version 9.0.4? If we are able the release the new packaging method in emacs 26.x, then we can remove org from emacs master completely, but if not, then at least as backup we have a newer org version to go out with that release. -- Kaushal Modi