[gentoo-dev] Reverse use of Python/Ruby versions
Not sure if this is practical, it may be less work if the use of Python and Ruby versions ( maybe others ) is reversed. Rather than adding all the versions that the ebuild supports. What if it only included versions it did not support? Rational When new versions are added. Ebuilds must be updated to support the new version. This can require editing a decent amount of ebuilds. Many may work fine with the new version. Making this extra needless work. From a user point of view, You cannot move to the newer version without keeping older around till all are updated to the newer version. Now one could say its the same work to mark versions not supported. But I think there is less of that. Also if something supports and older version and not newer, it may stand out more failing. Requiring someone to reduce to the older version, and maybe filing bugs to get it updated to the newer version. Python 2.7 stuff aside. I am not sure how many Python and Ruby packages break with a newer release. In pythons case I think once they support Python 3.x, there is little chance if it breaking with further 3.x releases. Not sure about Ruby. I could be very wrong and the present way of doing things being the only way. However if this is feasible it may lead to less work. It would allow all packages to move to a newer version. Also allowing punting of older sooner. This leaves the versions solely up to the eclasses. Then ebuilds simply mark the unsupported versions. Just like we do now with adding versions it does support. Which if something was say only Python 2.7. It would have a >= to excluded any newer version of Python. That said, we could do the same with the current way. Saying this supports Python/Ruby >=SLOT. Either way I do not think the current way is the best way. You have to add every version/slot to ebuilds that work fine with any version. When new ones are added, the ebuild has to be touched again. When it may work fine with a new version without requiring the ebuild to be modified. This came up with some stuff requiring ruby23 as I moved to 24. Looking around some stuff has Python 3.5 some 3.6, but all 3.4. So I will stick to 3.4 till everything is at 3.6. Otherwise no point and still have other versions. The approach mentioned above, if the packages do not have issue. I could go ahead and switch to ruby24 and pyton 3.6 across the board. Which I cannot do now till a bunch of ebulids have their targets increased. -- William L. Thomson Jr. pgpE1goV3vWlr.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On 09/04/2017 18:15, William L. Thomson Jr. wrote: Not sure if this is practical, it may be less work if the use of Python and Ruby versions ( maybe others ) is reversed. Rather than adding all the versions that the ebuild supports. What if it only included versions it did not support? Rational When new versions are added. Ebuilds must be updated to support the new version. This can require editing a decent amount of ebuilds. Many may work fine with the new version. Making this extra needless work. From a user point of view, You cannot move to the newer version without keeping older around till all are updated to the newer version. Now one could say its the same work to mark versions not supported. But I think there is less of that. Also if something supports and older version and not newer, it may stand out more failing. Requiring someone to reduce to the older version, and maybe filing bugs to get it updated to the newer version. Python 2.7 stuff aside. I am not sure how many Python and Ruby packages break with a newer release. In pythons case I think once they support Python 3.x, there is little chance if it breaking with further 3.x releases. Not sure about Ruby. I could be very wrong and the present way of doing things being the only way. However if this is feasible it may lead to less work. It would allow all packages to move to a newer version. Also allowing punting of older sooner. This leaves the versions solely up to the eclasses. Then ebuilds simply mark the unsupported versions. Just like we do now with adding versions it does support. Which if something was say only Python 2.7. It would have a >= to excluded any newer version of Python. That said, we could do the same with the current way. Saying this supports Python/Ruby >=SLOT. Either way I do not think the current way is the best way. You have to add every version/slot to ebuilds that work fine with any version. When new ones are added, the ebuild has to be touched again. When it may work fine with a new version without requiring the ebuild to be modified. This came up with some stuff requiring ruby23 as I moved to 24. Looking around some stuff has Python 3.5 some 3.6, but all 3.4. So I will stick to 3.4 till everything is at 3.6. Otherwise no point and still have other versions. The approach mentioned above, if the packages do not have issue. I could go ahead and switch to ruby24 and pyton 3.6 across the board. Which I cannot do now till a bunch of ebulids have their targets increased. +1 to the python part, cannot speak about ruby. or totally automate the addition of python versions to ebuilds at the exact time of bumping dev-lang/python.
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On 04/09/2017 06:15 PM, William L. Thomson Jr. wrote: > Not sure if this is practical, it may be less work if the use of > Python and Ruby versions ( maybe others ) is reversed. Rather than > adding all the versions that the ebuild supports. What if it only > included versions it did not support? It would only work if upstream provide a strong assurance for forward compatibility. Explicit testing and marking working seems the only practical way to ensure stability. -- Kristian Fiskerstrand OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3 signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On 04/09/2017 12:15 PM, William L. Thomson Jr. wrote: Not sure if this is practical, it may be less work if the use of Python and Ruby versions ( maybe others ) is reversed. Rather than adding all the versions that the ebuild supports. What if it only included versions it did not support? Even if this would work better, it would require retraining all developers, completely rewriting several eclasses, tons of documentation, and a few thousand ebuilds. No one's going to jump on that bandwagon without a proof-of-concept that works much better than what we have now.
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Sun, 9 Apr 2017 23:36:18 +0200 Francesco Riosa wrote: > On 09/04/2017 18:15, William L. Thomson Jr. wrote: > > Not sure if this is practical, it may be less work if the use of > > Python and Ruby versions ( maybe others ) is reversed. Rather than > > adding all the versions that the ebuild supports. What if it only > > included versions it did not support? > > > > Rational > > When new versions are added. Ebuilds must be updated to support the > > new version. This can require editing a decent amount of ebuilds. > > Many may work fine with the new version. Making this extra needless > > work. From a user point of view, You cannot move to the newer > > version without keeping older around till all are updated to the > > newer version. > > > > Now one could say its the same work to mark versions not supported. > > But I think there is less of that. Also if something supports and > > older version and not newer, it may stand out more failing. > > Requiring someone to reduce to the older version, and maybe filing > > bugs to get it updated to the newer version. > > > > Python 2.7 stuff aside. I am not sure how many Python and Ruby > > packages break with a newer release. In pythons case I think once > > they support Python 3.x, there is little chance if it breaking with > > further 3.x releases. Not sure about Ruby. > > > > I could be very wrong and the present way of doing things being the > > only way. However if this is feasible it may lead to less work. It > > would allow all packages to move to a newer version. Also allowing > > punting of older sooner. This leaves the versions solely up to the > > eclasses. Then ebuilds simply mark the unsupported versions. Just > > like we do now with adding versions it does support. > > > > Which if something was say only Python 2.7. It would have a >= to > > excluded any newer version of Python. That said, we could do the > > same with the current way. Saying this supports Python/Ruby >=SLOT. > > > > Either way I do not think the current way is the best way. You have > > to add every version/slot to ebuilds that work fine with any > > version. When new ones are added, the ebuild has to be touched > > again. When it may work fine with a new version without requiring > > the ebuild to be modified. > > > > This came up with some stuff requiring ruby23 as I moved to 24. > > Looking around some stuff has Python 3.5 some 3.6, but all 3.4. So > > I will stick to 3.4 till everything is at 3.6. Otherwise no point > > and still have other versions. > > > > The approach mentioned above, if the packages do not have issue. I > > could go ahead and switch to ruby24 and pyton 3.6 across the board. > > Which I cannot do now till a bunch of ebulids have their targets > > increased. > > > +1 to the python part, cannot speak about ruby. > or totally automate the addition of python versions to ebuilds at the > exact time of bumping dev-lang/python. > > This could be partially automated using buildbot. The easiest would be for pkgs containing working test suites. Those that pass could be auto-enabled with that python with ~arch keywords. I think if a stable pkg is to be added the new python, it should be via a revision bump and ~arch'ing the keywords. But we could enforce the bot to rev-bump all ebuilds just as easily. Pkgs without tests, those would be harder and we could do some basic tests on those, like syntax, test imports, but would require additional means of testing in order to qualify for an auto-python addition. It should also be possible for the bot to scrape setup.py for comppatible python versions. Many of the pkgs I have been working with recently have them listed. Also there is travis.yml files in many upstream pkg repos which can also be scraped for tested pythons. It could certainly reduce the manpower needed to keep things up to date. -- Brian Dolbec
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On 09/04/2017 23:44, Kristian Fiskerstrand wrote: On 04/09/2017 06:15 PM, William L. Thomson Jr. wrote: Not sure if this is practical, it may be less work if the use of Python and Ruby versions ( maybe others ) is reversed. Rather than adding all the versions that the ebuild supports. What if it only included versions it did not support? It would only work if upstream provide a strong assurance for forward compatibility. Explicit testing and marking working seems the only practical way to ensure stability. Surely enough forward compatibility may be a problem and python upstream does deprecate and remove features #1 and things that fiddle with python bytecode will easily break. However we keep $KEYWORDS between version of the same package and that it's subject to the same exact kind of problems. Honestly just trying out python 3.6 is a pain at the moment and the situation is the same at every python bump. #1 https://docs.python.org/3/whatsnew/index.html
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Sun, 9 Apr 2017 17:52:44 -0400 Michael Orlitzky wrote: > On 04/09/2017 12:15 PM, William L. Thomson Jr. wrote: > > Not sure if this is practical, it may be less work if the use of > > Python and Ruby versions ( maybe others ) is reversed. Rather than > > adding all the versions that the ebuild supports. What if it only > > included versions it did not support? > > Even if this would work better, it would require retraining all > developers, completely rewriting several eclasses, tons of > documentation, and a few thousand ebuilds. There could be things done to ease any transition. Regarding python, the first of which could be to ignore any targets beyond say 2.7 and just starting enabling support. That would mostly leave just effected packages, ones that break with some 3.x version but not with another. As for modifying a few thousand ebuilds; 1, That is already the case now. If a new python or ruby version comes out. All those ebuilds have to modified to support the new target. That argument isn't really valid. 2. As for the actual changes that can be done pragmatically if just swapping out the TARGETS variable. Or even if more advanced requiring changing lines. If its the same change to all. I have a script, ebuild-batcher[1], that can safely make any change to a massive number of ebuilds. I have used it a few times on hundreds of ebuilds. It can easily handle thousands. > No one's going to jump on that bandwagon without a proof-of-concept > that works much better than what we have now. Anyone who has worked with a Gentoo system long enough that has Python and/or Ruby has experienced this. You end up having multiple TARGETS and are constantly messing with those variables adding new ones, removing old, etc. It creates a considerable amount of work. I have been fighting over ruby23 that I cannot seem to avoid. Despite having had ruby24 for I think at least a week or more. My build server stopped due to needing ruby23. I have been fighting with stuff around that. More than likely anything bound to ruby23 target works with ruby24. Its mostly how the system is designed. I could be wrong in that case and something require ruby23, that does not with ruby24. However most those packages do not have a ruby24 target, so its hard to say if its negated or just not enabled yet. Being a new version, all have to be modified for such. 1. https://github.com/Obsidian-StudiosInc/ebuild-batcher -- William L. Thomson Jr. pgp_eJ93OMeS0.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On 09/04/2017 23:52, Michael Orlitzky wrote: On 04/09/2017 12:15 PM, William L. Thomson Jr. wrote: Not sure if this is practical, it may be less work if the use of Python and Ruby versions ( maybe others ) is reversed. Rather than adding all the versions that the ebuild supports. What if it only included versions it did not support? Even if this would work better, it would require retraining all developers, completely rewriting several eclasses, tons of documentation, and a few thousand ebuilds. Let's assume the retraining will be 2 or 3 orders of magnitude easier than switching from cvs to git, than that's doable. Eclasses, ebuilds and documentation will be the real burden, but at this point it's better to discuss if the feature is wanted or not, then later wait for the volunteer(s) to actually do the work. No one's going to jump on that bandwagon without a proof-of-concept that works much better than what we have now. yep, that could be done, but since it's not trivial maybe better decide if it will be wasted or possibly accepted. by the way eclasses from gentoo repo with PYTHON string inside are the following: $ grep -c PYTHON *.eclass | grep -v :0$ distutils-r1.eclass:28 enlightenment.eclass:6 gnome-python-common-r1.eclass:2 kernel-2.eclass:2 mate.eclass:1 mozcoreconf-v4.eclass:3 python-any-r1.eclass:75 python-r1.eclass:107 python-single-r1.eclass:112 python-utils-r1.eclass:203 ros-catkin.eclass:40 twisted-r1.eclass:2 waf-utils.eclass:7
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On 10/04/2017 00:20, Brian Dolbec wrote: On Sun, 9 Apr 2017 23:36:18 +0200 Francesco Riosa wrote: On 09/04/2017 18:15, William L. Thomson Jr. wrote: Not sure if this is practical, it may be less work if the use of Python and Ruby versions ( maybe others ) is reversed. Rather than adding all the versions that the ebuild supports. What if it only included versions it did not support? Rational When new versions are added. Ebuilds must be updated to support the new version. This can require editing a decent amount of ebuilds. Many may work fine with the new version. Making this extra needless work. From a user point of view, You cannot move to the newer version without keeping older around till all are updated to the newer version. Now one could say its the same work to mark versions not supported. But I think there is less of that. Also if something supports and older version and not newer, it may stand out more failing. Requiring someone to reduce to the older version, and maybe filing bugs to get it updated to the newer version. Python 2.7 stuff aside. I am not sure how many Python and Ruby packages break with a newer release. In pythons case I think once they support Python 3.x, there is little chance if it breaking with further 3.x releases. Not sure about Ruby. I could be very wrong and the present way of doing things being the only way. However if this is feasible it may lead to less work. It would allow all packages to move to a newer version. Also allowing punting of older sooner. This leaves the versions solely up to the eclasses. Then ebuilds simply mark the unsupported versions. Just like we do now with adding versions it does support. Which if something was say only Python 2.7. It would have a >= to excluded any newer version of Python. That said, we could do the same with the current way. Saying this supports Python/Ruby >=SLOT. Either way I do not think the current way is the best way. You have to add every version/slot to ebuilds that work fine with any version. When new ones are added, the ebuild has to be touched again. When it may work fine with a new version without requiring the ebuild to be modified. This came up with some stuff requiring ruby23 as I moved to 24. Looking around some stuff has Python 3.5 some 3.6, but all 3.4. So I will stick to 3.4 till everything is at 3.6. Otherwise no point and still have other versions. The approach mentioned above, if the packages do not have issue. I could go ahead and switch to ruby24 and pyton 3.6 across the board. Which I cannot do now till a bunch of ebulids have their targets increased. +1 to the python part, cannot speak about ruby. or totally automate the addition of python versions to ebuilds at the exact time of bumping dev-lang/python. This could be partially automated using buildbot. The easiest would be for pkgs containing working test suites. Those that pass could be auto-enabled with that python with ~arch keywords. I think if a stable pkg is to be added the new python, it should be via a revision bump and ~arch'ing the keywords. But we could enforce the bot to rev-bump all ebuilds just as easily. Pkgs without tests, those would be harder and we could do some basic tests on those, like syntax, test imports, but would require additional means of testing in order to qualify for an auto-python addition. It should also be possible for the bot to scrape setup.py for comppatible python versions. Many of the pkgs I have been working with recently have them listed. Also there is travis.yml files in many upstream pkg repos which can also be scraped for tested pythons. It could certainly reduce the manpower needed to keep things up to date. All good ideas, if William proposal is rejected, I'd like to request a mandatory step like this when a new minor 3.x python version is added to the tree. No idea if that's feasible for ruby too.
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Sun, 9 Apr 2017 12:15:56 -0400 "William L. Thomson Jr." wrote: > Python 2.7 stuff aside. I am not sure how many Python and Ruby packages > break with a newer release. In pythons case I think once they support > Python 3.x, there is little chance if it breaking with further 3.x > releases. Not sure about Ruby. I'm not going to weigh heavily into this as I don't mind the current setup but as a professional Ruby developer, I can say that breakages between versions are usually overblown by those outside the community. Yeah, there usually are some but they tend to only affect the bigger libraries and frameworks that do some more exotic things. Even then, the changes required are generally very small, sometimes even just one line. People thought the sky would fall when 2.4 deprecated Fixnum and Bignum. It really didn't. It's still just a warning right now but I haven't seen that warning since it was dealt with in Rails. -- James Le Cuirot (chewi) Gentoo Linux Developer pgpr0oLkm0YAA.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Sun, 9 Apr 2017 23:44:50 +0200 Kristian Fiskerstrand wrote: > On 04/09/2017 06:15 PM, William L. Thomson Jr. wrote: > > Not sure if this is practical, it may be less work if the use of > > Python and Ruby versions ( maybe others ) is reversed. Rather than > > adding all the versions that the ebuild supports. What if it only > > included versions it did not support? > > It would only work if upstream provide a strong assurance for forward > compatibility. Explicit testing and marking working seems the only > practical way to ensure stability. Even if things break, you just do the opposite of now. You would disable/mask ( or something to that effect ) any versions the package did not support. Basically what is done now but in reverse. Say what it does not build with; allowing it to build with anything it can, existing today, or coming in the future. In theory at least one would have to modify less ebuilds that break with a new version. Than modifying all adding a new target. There is also the added bonus when a version is dropped. No ebuilds need be modified. I assume if say python 3.4 is dropped. All ebuilds with that target need to be updated. This would considerably reduce work all around, with a much better experience for the end user. No targets to fool with. -- William L. Thomson Jr. pgpdzajB8wC21.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Sun, 9 Apr 2017 15:20:02 -0700 Brian Dolbec wrote: > > This could be partially automated using buildbot. The easiest would > be for pkgs containing working test suites. Those that pass could be > auto-enabled with that python with ~arch keywords. I think if a > stable pkg is to be added the new python, it should be via a revision > bump and ~arch'ing the keywords. But we could enforce the bot to > rev-bump all ebuilds just as easily. If the package failed, all that would need to be done kinda like now is a given variable modified in the ebuild. Just marking what ever it did not work with. As mentioned that could be done via my ebuild-batcher[1], though same functionality is easily replicated. I also have an ebuild-bumper[2], that could be made more advanced. I am aware of ebump, but ebuild-bumper is a bit different. It works with sets of ebuilds for bumps and cleaning. It can do individual as well but so can ebump. I plan to further automate ebuild-bumper with something that tracks upstream releases and attempts to automate bumps. Though even running it manually is not cumbersome. Just prefer zero day as much as possible. 1. https://github.com/Obsidian-StudiosInc/ebuild-batcher 2. https://github.com/Obsidian-StudiosInc/ebuild-bumper -- William L. Thomson Jr. pgp_JhiFSjrcx.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On 04/09/2017 07:15 PM, William L. Thomson Jr. wrote: If the package failed, all that would need to be done kinda like now is a given variable modified in the ebuild. Just marking what ever it did not work with. As mentioned that could be done via my ebuild-batcher[1], though same functionality is easily replicated. How do you plan to test a thousand packages against the new version of python, and then revision/stabilize all of the broken ones immediately? Or is the plan to just break everyone's systems, and ask them to report bugs for the things that stopped working? I think what you will actually get as a result is that nobody will ever add a new version of python to the tree, because you've just made it a huge ordeal to do so.
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On 10/04/2017 01:59, Michael Orlitzky wrote: On 04/09/2017 07:15 PM, William L. Thomson Jr. wrote: If the package failed, all that would need to be done kinda like now is a given variable modified in the ebuild. Just marking what ever it did not work with. As mentioned that could be done via my ebuild-batcher[1], though same functionality is easily replicated. How do you plan to test a thousand packages against the new version of python, and then revision/stabilize all of the broken ones immediately? Or is the plan to just break everyone's systems, and ask them to report bugs for the things that stopped working? python 3.5.0 was released on 2015-09-13 and is still marked unstable and until recently was unusable because there were too many missing packages marked ready for it, stabilization isn't that faster right now. Most of the breakage would be caught when recompiling (bytecode) of a package stable or not and even when not caught it would trigger only eselecting an unstable dev-lang/python if testing all python packages is required to stabilize dev-lang/python (which is kinda the current situation). Python is slotted, that would also help a lot at keeping the breakage unobtrusive * * Provided portage is never broken by a dev-lang/python update, but that's easy to test. I think what you will actually get as a result is that nobody will ever add a new version of python to the tree, because you've just made it a huge ordeal to do so. I do respectfully disagree with this sentence.
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Sun, 9 Apr 2017 19:59:22 -0400 Michael Orlitzky wrote: > > How do you plan to test a thousand packages against the new version > of python, and then revision/stabilize all of the broken ones > immediately? Or is the plan to just break everyone's systems, and ask > them to report bugs for the things that stopped working? If packages have tests, running those is one way. If they do not, and its say a library. Long as other packages that use the library build/run against it then it should be ok. It would get trickier for things lacking tests. Breaking already occurs now but in the form of breaking updates and causing users to fiddle with the targets. I am NOT talking about stabilization at all. Simple reducing the burden of adding targets to ebuild, and users having to fiddle with targets as they come and go. > I think what you will actually get as a result is that nobody will > ever add a new version of python to the tree, because you've just > made it a huge ordeal to do so. This is actually the opposite. To add a new version as is right now. You have to edit every Python or Ruby ebuild. Otherwise they cannot use that new version. There is tremendous work as is now. Not to mention a really bad end user experience. This would considerable reduce the workload. Not to mention making life better for end users. -- William L. Thomson Jr. pgpAJEtNcS4qL.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Sun, 9 Apr 2017 12:15:56 -0400 "William L. Thomson Jr." wrote: > > The approach mentioned above, if the packages do not have issue. I > could go ahead and switch to ruby24 and pyton 3.6 across the board. > Which I cannot do now till a bunch of ebulids have their targets > increased. > This could introduce tree breakage. Why? Because of the whole "X built with python FOO depends on a Y built with python FOO" mechanic. As it stands, when unmasking a new python, people have to go through and mark packages as working, which has the requirement that their dependencies are themselves working in order to satisfy the USE requirements. When you reverse this, you introduce a situation where adding a new version across the board creates a new skeleton-tree of support. And when you find something that *doesnt* work, you may have to recursively mark its *dependents* as "non-working" to avoid a dependency graph breakage. This is the sort of thing that makes life hell, for both developers and users. I could be barking up the wrong tree, buy the python team could give a better idea of what that would look like in practice than me. But I fear this would look like "the hell of dekeywording" made harder by the lack of tools to facilitate such a thing. pgpowcZAg5rj2.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Mon, 10 Apr 2017 13:38:58 +1200 Kent Fredric wrote: > > When you reverse this, you introduce a situation where adding a new > version across the board creates a new skeleton-tree of support. FYI, this is how it is when a new Java JDK comes out. When 1.5 came out, if 1.4 code had issues compiling, the package was addressed. The same thing can happen any time a new version comes out. The impact most times is minimal. Unless there are wide sweeping changes. Which is pretty rare in any language. > And when you find something that *doesnt* work, you may have to > recursively mark its *dependents* as "non-working" to avoid a > dependency graph breakage. This has never been the case with Java. If package A requires version X, but B Y, then B builds with Y as its pulled in as a dep. while A proceeds to build with X. Where this is different for Python, Ruby, and also Perl. They all install files into a directory based on version. You may have multiple copies in each, vs one. Perl does not have targets, nor does Java. > This is the sort of thing that makes life hell, for both developers > and users. The present system is a PITA for users. Fiddling with adding/removing targets for Python/Ruby. In addition to selecting which for the system. All these same problems exist for Java, with the exception of installation locations as mentioned. > I could be barking up the wrong tree, buy the python team could give a > better idea of what that would look like in practice than me. I have direct experience in this. I am experiencing some of this now with JDK 9. It is different regarding Python and Ruby. It would be up to those teams. But I do think the entire TARGETS aspect needs to be revisited. No one likes adding/removing TARGETS. That is a waste of anyone's time. Much less developers adding/removing targets from ebuilds. -- William L. Thomson Jr. pgpDXwfVE6cAE.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Sun, 9 Apr 2017 22:04:13 -0400 "William L. Thomson Jr." wrote: > This has never been the case with Java. Its not a problem with C binaries either, because you have a discrete compile time, and language level interop between compiled binary forms. Meanwhile, you cannot build two parts of a given python dependency chain with different pythons, nor different perls. > If package A requires version X, but B Y, then B builds with Y as its > pulled in as a dep. while A proceeds to build with X. > > Where this is different for Python, Ruby, and also Perl. They all > install files into a directory based on version. You may have multiple > copies in each, vs one. Perl does not have targets, nor does Java. Right, but this is impossible with Ruby, Python, and Perl. Perl *could* have targets, and some people think could do with it, but it and java are very much in different boats. Perl is in the same boat as Python and Ruby where in "new version of thing" means "everything must be compiled with the new target" Perl simply has a *moving* target instead of multiple concurrent targets. Essentially, with Perl we've done the effect of "Add X, Remove Y" for all things. We additionally have a much better precedent than python at syntax-interop between versions, so its more justifiable to only have the single target ( because you practically never need an older perl "just for compat reasons", though at the rate this garbage[1] is going, that could change one day ) If anything, Perl is only avoiding the Python problem you hate with significant amounts of heroism, and its only a matter of time before upstream force our hands in ways that make that heroism non-viable, and we have to dig deep and work out how the hell to maintain concurrent targets. > The present system is a PITA for users. Fiddling with adding/removing > targets for Python/Ruby. In addition to selecting which for the system. > All these same problems exist for Java, with the exception of > installation locations as mentioned. I honestly think you're looking at the wrong problem domain to fix this problem, in ways that will introduce yet more regressions and broken trees. We should have what I've been saying we should have for a while now: * Soft Options. We only have 2 types of option at present from the users perspective, "on" options, and "off" options. Portage doesn't even distinguish between who is setting them, user profile and the gentoo profiles simply flatten into a single logical profile, and then portage then demands that changes be made to this set, failing to discriminate at all between "ok, this was the profile specified default and it needs to be non-default for this problem" and "user doesn't actually want this, how can we avoid that" And portage then compounds this by dictating that any option changes that ebuilds need must be enshrined by the user as things the /user/ wants, when in truth, they're not things the user wants, they're things the ebuilds want, and the user begrudingly accepts. Hence why an option of "on, but only if portage really needs it" is missing, but needed. I would gladly set soft-targets of every python version under the sun, and then allow portage to turn them on *as necessary*, and this would be much preferable to having to either a) turn them on globally and pull in stuff and waste time compiling stuff that's not even getting used. b) Maintain a painful and carefully catered list of things to prevent aforementioned problems. In short, users need a way to discriminate "things I care about" from "things I don't care about" Currently, its just a big cluster of those things in one place, and the complexity is inescapably thrust into the users hands on a daily basis. 1: https://bugs.gentoo.org/showdependencytree.cgi?id=613764&hide_resolved=0 pgpF4zTjVMYjD.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Mon, 2017-04-10 at 00:04 +0100, James Le Cuirot wrote: > People thought the sky would fall when 2.4 deprecated Fixnum > and > Bignum. It really didn't. It's still just a warning right now but I > haven't seen that warning since it was dealt with in Rails. > This change broke the rspec test suite quite significantly and to the point where it isn't clear if rspec 3.5 is actually compatible with ruby24 or not. So, no ruby24 target for now, and consequently all packages depending on it also can't get a ruby24 target. Every ruby release will have some breaking changes, and the impact on the tree will depend on which packages it will affect. Hans signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Reverse use of Python/Ruby versions
Dnia 9 kwietnia 2017 18:15:56 CEST, "William L. Thomson Jr." napisał(a): >Not sure if this is practical, it may be less work if the use of >Python and Ruby versions ( maybe others ) is reversed. Rather than >adding all the versions that the ebuild supports. What if it only >included versions it did not support? It is always nice when a person who: a. did not bother to do any research on the topic (such as reading previous posts or even asking the relevant teams), b. has barely any clue (if any at all) about Python ecosystem or package maintenance in Gentoo, c. is either completely ignorant of how Python packages worked in the past (which quite proves the points made above) or presumes that they were changed for no reason by incompetent developers, decides that the workflow of Python team needs to be changed and goes to discuss it on the mailing list with other people who barely do any Python work. > >Rational >When new versions are added. Ebuilds must be updated to support the new >version. This can require editing a decent amount of ebuilds. Many may >work fine with the new version. Making this extra needless work. From a >user point of view, You cannot move to the newer version without >keeping older around till all are updated to the newer version. > >Now one could say its the same work to mark versions not supported. But >I think there is less of that. Also if something supports and older >version and not newer, it may stand out more failing. Requiring someone >to reduce to the older version, and maybe filing bugs to get it updated >to the newer version. > >Python 2.7 stuff aside. I am not sure how many Python and Ruby packages >break with a newer release. In pythons case I think once they support >Python 3.x, there is little chance if it breaking with further 3.x >releases. Not sure about Ruby. > >I could be very wrong and the present way of doing things being the >only way. However if this is feasible it may lead to less work. It >would allow all packages to move to a newer version. Also allowing >punting of older sooner. This leaves the versions solely up to the >eclasses. Then ebuilds simply mark the unsupported versions. Just like >we do now with adding versions it does support. > >Which if something was say only Python 2.7. It would have a >= to >excluded any newer version of Python. That said, we could do the same >with the current way. Saying this supports Python/Ruby >=SLOT. > >Either way I do not think the current way is the best way. You have to >add every version/slot to ebuilds that work fine with any version. When >new ones are added, the ebuild has to be touched again. When it may >work fine with a new version without requiring the ebuild to be >modified. > >This came up with some stuff requiring ruby23 as I moved to 24. Looking >around some stuff has Python 3.5 some 3.6, but all 3.4. So I will stick >to 3.4 till everything is at 3.6. Otherwise no point and still have >other versions. > >The approach mentioned above, if the packages do not have issue. I >could go ahead and switch to ruby24 and pyton 3.6 across the board. >Which I cannot do now till a bunch of ebulids have their targets >increased. -- Best regards, Michał Górny (by phone) -- Best regards, Michał Górny (by phone) -- Best regards, Michał Górny (by phone) -- Best regards, Michał Górny (by phone) -- Best regards, Michał Górny (by phone) -- Best regards, Michał Górny (by phone)
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Mon, Apr 10, 2017 at 8:37 AM, Michał Górny wrote: > It is always nice when a person who: Please stop the sarcasm. While I understand the reaction, the idea in itself does not seem totally crazy to me, and it seems useful to have a discussion on its merits. At the same time, I would not consider it far-fetched to say that you proposed significant changes to handling of manifest hashes, without deep knowledge of the security aspects of the hashing algorithms up for discussion. This is sometimes how we learn. If you feel the thread wastes time, you can just ignore it (as you seem to have done with the manifest hashes thread after a few critical responses, somewhat to my disappointment). Cheers, Dirkjan
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On 04/09/2017 08:58 PM, William L. Thomson Jr. wrote: I am NOT talking about stabilization at all. Simple reducing the burden of adding targets to ebuild, and users having to fiddle with targets as they come and go. You are: when you find out that a stable package doesn't work with the next version of python, you have to figure out who the maintainer of that package is, and file a bug. Then, whenever he decides to fix it, you have to wait 30 days and file a stabilization request. Wait another few months for that to go through, and repeat however many times to fix every broken package. You can either spend months/years doing that for all affected packages and every new version of python, or just commit the new version of python and let things break. Neither option is an improvement over the way things work now. We have it your way for PHP packages, and I wish it was like Python/Ruby instead.
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Mon, 10 Apr 2017 10:57:43 -0400 > > You are: when you find out that a stable package doesn't work with > the next version of python, you have to figure out who the maintainer > of that package is, and file a bug. That is how things are done for Java, and I think Perl as well. There tend to be tracker bugs for the next version. https://bugs.gentoo.org/show_bug.cgi?id=384609 > Then, whenever he decides to fix > it, you have to wait 30 days and file a stabilization request. Wait > another few months for that to go through, and repeat however many > times to fix every broken package. This has nothing to do with stable. A new version would not go direct to stable. That version would not be marked stable so not effecting stable packages till it is marked stable. > You can either spend months/years doing that for all affected > packages and every new version of python, or just commit the new > version of python and let things break. Neither option is an > improvement over the way things work now. The idea is to stop touching every ebuild per every new python or ruby release. Or when an old is removed. Also to stop having users mess with TARGETS. > We have it your way for PHP packages, and I wish it was like > Python/Ruby instead. That makes PHP, Perl, and Java. Just Python and Ruby are handled differently. -- William L. Thomson Jr. pgppGP54QJSNw.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Mon, 10 Apr 2017 16:35:48 +1200 Kent Fredric wrote: > > Meanwhile, you cannot build two parts of a given python dependency > chain with different pythons, nor different perls. True but this is not changing how things work, just reversing. > Right, but this is impossible with Ruby, Python, and Perl. It is done right now. The problem your describing exist now and is addressed. This would address it the same way but reversed. > Perl *could* have targets, and some people think could do with it, > but it and java are very much in different boats. Easier to deal with as a user. Less work as a developer. > Perl is in the same boat as Python and Ruby where in "new version of > thing" means "everything must be compiled with the new target" Installation wise, but with a new JDK, you can still have compilation failures with existing packages. That it gets installed in the same place is moot regarding differences with Java and other languages. > I honestly think you're looking at the wrong problem domain to fix > this problem, in ways that will introduce yet more regressions and > broken trees. Problem is simple, Targets are a PITA to deal with, ever changing. They lead to issues when you select incompatible ones or options. Best case wild card and let all install. But otherwise its a chore to deal with. > We only have 2 types of option at present from the users perspective, > "on" options, and "off" options. That sounds terrible. Lots of distros with things already turned on/off. Gentoo should never be one. USE flags can be a PITA, but they are a necessary evil. Its the ever changing TARGETS that are annoying, and cumbersome to users. -- William L. Thomson Jr. pgpoXNMHlFEGf.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Mon, 10 Apr 2017 08:37:34 +0200 Michał Górny wrote: > > It is always nice when a person who: Starts off with insults and rudeness... Why I avoid you and I have requested MULITPLE times you just avoid me. Almost did not reply, but unlike your comments I will stick to FACTS. > a. did not bother to do any research on the topic (such as reading > previous posts or even asking the relevant teams), Research was done in the form of packaging some python applications. Also having worked with OTHER languages and teams on Gentoo. There are other ways of doing things. For those who are open minded to considering improvements. > b. has barely any clue (if any at all) about Python ecosystem or > package maintenance in Gentoo, Again I have recently packaged some python libraries and applications. I personally maintain some 300+ Java ebuilds and others. https://github.com/Obsidian-StudiosInc/os-xtoo I think I have a clue when it comes to package maintenance. I was doing it as a Developer back in 2006 thru 2008... https://github.com/wltjr?tab=overview&from=2006-12-01&to=2006-12-31 > c. is either completely ignorant of how Python packages worked in the > past (which quite proves the points made above) or presumes that they > were changed for no reason by incompetent developers, I have seen it evolve ever since 3.x came out in 2008. The situation was never good and should have gone a different route from the start. Thankfully Java went a different route and other teams never shared the same approach. It is long over due to consider a better way. > decides that the workflow of Python team needs to be changed and goes > to discuss it on the mailing list with other people who barely do any > Python work. Because of how Python is handled on Gentoo. As a developer I would NEVER use python. Just working with a few python libraries and apps, packaging them. Its a PITA compared to Java. If for no other reason than I have to go touch the ebuilds anytime a Python version is added or removed. Same for Ruby. That is dumb... There are some 1600 Python ebuilds. That is ALLOT of work to fiddle with adding/removing targets as new things come and go... Working with hundreds of ebuilds myself. I can easily understand the magnitude of such changes. Even my fully automated scripts, take considerable time to make minor changes across lots of ebuilds. If humans have to do this, it will take MUCH longer. Who wants to waste their time on such? Its funny. In the days of CI and CD, we must manually mess with targets There has to be a better way. If not what I am suggesting some other. I do not see any other solutions suggested. Just negativity, insults, and lack of any real facts just opinion and rudeness. Typical status quo... -- William L. Thomson Jr. pgpj7GVrw9h7A.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On pon, 2017-04-10 at 12:03 -0400, William L. Thomson Jr. wrote: > On Mon, 10 Apr 2017 08:37:34 +0200 > Michał Górny wrote: > > > > It is always nice when a person who: > > Starts off with insults and rudeness... Why I avoid you and I have > requested MULITPLE times you just avoid me. Almost did not reply, but > unlike your comments I will stick to FACTS. I would love to avoid you. However, you make this impossible via trying to make the life of Python team (which I am part of) a misery, and Python support in Gentoo (which I use) a mess long-term. What is even worse, you do that without even talking to the Python team, or even bothering to CC them -- what you do instead is start a public discussion about Python without even bothering to invite Python people to it. > > > a. did not bother to do any research on the topic (such as reading > > previous posts or even asking the relevant teams), > > Research was done in the form of packaging some python applications. > Also having worked with OTHER languages and teams on Gentoo. There are > other ways of doing things. For those who are open minded to > considering improvements. FYI, if you want to change something, the first research you ought to do is to ask 'why is it done this way?' Not jump to some random points that might be completely irrelevant. > > > b. has barely any clue (if any at all) about Python ecosystem or > > package maintenance in Gentoo, > > Again I have recently packaged some python libraries and applications. > I personally maintain some 300+ Java ebuilds and others. > https://github.com/Obsidian-StudiosInc/os-xtoo > Well, I've opened the first ebuild in your overlay [1] and it wouldn't pass basic code review. For a start, it doesn't enforce USE dependencies which are absolutely necessary for anything to work by omething else than mere accident. It also explains why you are able to claim that your solution works. [1]:https://github.com/Obsidian-StudiosInc/os-xtoo/blob/master/dev-pytho n/python-efl/python-efl-.ebuild > I think I have a clue when it comes to package maintenance. I was doing > it as a Developer back in 2006 thru 2008... > https://github.com/wltjr?tab=overview&from=2006-12-01&to=2006-12-31 I'm sorry but 10 years ago is not very relevant to Gentoo today. > > > c. is either completely ignorant of how Python packages worked in the > > past (which quite proves the points made above) or presumes that they > > were changed for no reason by incompetent developers, > > I have seen it evolve ever since 3.x came out in 2008. The situation > was never good and should have gone a different route from the start. > Thankfully Java went a different route and other teams never shared the > same approach. It is long over due to consider a better way. > > > decides that the workflow of Python team needs to be changed and goes > > to discuss it on the mailing list with other people who barely do any > > Python work. > > Because of how Python is handled on Gentoo. As a developer I would > NEVER use python. Just working with a few python libraries and apps, > packaging them. Its a PITA compared to Java. > > If for no other reason than I have to go touch the ebuilds anytime a > Python version is added or removed. Same for Ruby. That is dumb... > > There are some 1600 Python ebuilds. That is ALLOT of work to fiddle > with adding/removing targets as new things come and go... Working with > hundreds of ebuilds myself. I can easily understand the magnitude of > such changes. > > Even my fully automated scripts, take considerable time to make minor > changes across lots of ebuilds. If humans have to do this, it will take > MUCH longer. Who wants to waste their time on such? > > Its funny. In the days of CI and CD, we must manually mess with > targets There has to be a better way. If not what I am suggesting > some other. I do not see any other solutions suggested. Just negativity, > insults, and lack of any real facts just opinion and rudeness. > > Typical status quo... The funny part is that you can write walls of text on yourself and your ideas but find it impossible to put the most important question: *why is it done like this?* -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Mon, 10 Apr 2017 19:14:32 +0200 Michał Górny wrote: > I would love to avoid you. However, you make this impossible via > trying to make the life of Python team (which I am part of) a misery, I do not force you to reply. Clearly you are not able to control yourself from replying. I do with facts, you with opinions and comments. > and Python support in Gentoo (which I use) a mess long-term. What is > even worse, you do that without even talking to the Python team, or > even bothering to CC them -- what you do instead is start a public > discussion about Python without even bothering to invite Python > people to it. This discussion is about more than python. You are ONE member of the team, with at least 17 others. You are also NOT the Python Team lead. Who do you think you are, to approach me or ANYONE such way? You are one person. The word team does not mean I, MGORNY Now to step back. The post here is to engage in discussion with said teams. But rather than address Python directly, and Ruby directly. I chose to come here to address BOTH. The problem is not unique to one or the other. What if Ruby team had ideas that could benefit Python? Would they know if the interaction is happening just with the Python team? Think, how do you reach more than one Gentoo team? What if an idea effects more than one? I chose the right list, and the correct method. Even if it does not suit some individuals opinions. Or their preferred way of dictating how others conduct themselves. > FYI, if you want to change something, the first research you ought to > do is to ask 'why is it done this way?' Not jump to some random > points that might be completely irrelevant. I understand, and I believe there are better ways. If you are not capable of coming up with any better ways. That is your own personal limitation. Again to add/remove a new python/ruby version requires touching every python/ruby ebuild. You think that is efficient or the best way? Are you, mgorny, adding/removing these python/ruby targets to lots of ebuilds? I do not see any of that here. Guess you leave that work to others https://github.com/gentoo/gentoo/commits?author=mgorny Not to mention your 2,033 commits, across the life of Gentoo Git repo. With some 1600+ python packages. Just modifying the COMPAT would increase your commit count considerable. I believe you are not doing this work, leaving it to others. You can prove me wrong with commits. Should be close to at least 100 commits. If you are doing serious python work. > Well, I've opened the first ebuild in your overlay [1] and it wouldn't > pass basic code review. Your review. Which your review of Firebird introduced REQUIRED USE flags that did not work and broke the package. Despite the issue being addressed a 1 line fix. You wanted the entire ebuild revised and introduced a much larger issue that did not exist in the first place. I use repoman on my overlay. If something does not meet QA. Then go modify repoman to point such out. If repoman is not pointing it out, then is it really an issue? Maybe just in mgorny's mind > For a start, it doesn't enforce USE > dependencies which are absolutely necessary for anything to work by > omething else than mere accident. It also explains why you are able > to claim that your solution works. I have not implemented what I am suggestion. I fail to see how you can say something not implemented fails to work. What is your case study? Have you re-wrote python eclasses to no use TARGETS? > [1]:https://github.com/Obsidian-StudiosInc/os-xtoo/blob/master/dev-pytho > n/python-efl/python-efl-.ebuild That is new, and FYI mostly a copy form what is in TREE... Go diff and see for yourself. What ever issues I expect YOU mgorny to go fix in tree. Otherwise be quiet. https://github.com/gentoo/gentoo/tree/master/dev-libs/efl It also breaks due to upstream changes, it requires EFL 1.18, and breaks with 1.19. EFL likely needs to be slotted but that is another matter. > > I think I have a clue when it comes to package maintenance. I was > > doing it as a Developer back in 2006 thru 2008... > > https://github.com/wltjr?tab=overview&from=2006-12-01&to=2006-12-31 > > I'm sorry but 10 years ago is not very relevant to Gentoo today. Funny, given stuff 10years ago is still in tree... https://github.com/gentoo/gentoo/tree/master/dev-java/servletapi I was working on removing that in 2007 Repomans commit message is more than 10yrs old. Considerable stuff in gentoo has been around for some time. Things have been added but hardly changed, equery, euse, emerge, eselect, etc... Any EAPI=0 ebuilds around? Guess how old those are? Or others... > The funny part is that you can write walls of text on yourself and > your ideas but find it impossible to put the most important question: > *why is it done like this?* Likely cause of people like you, who cannot come up with a better way. What are your ideas to improve things? Any? Or the status quo is utopi
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On pon, 2017-04-10 at 15:21 +0200, Dirkjan Ochtman wrote: > On Mon, Apr 10, 2017 at 8:37 AM, Michał Górny wrote: > > It is always nice when a person who: > > Please stop the sarcasm. While I understand the reaction, the idea in > itself does not seem totally crazy to me, and it seems useful to have > a discussion on its merits. > > At the same time, I would not consider it far-fetched to say that you > proposed significant changes to handling of manifest hashes, without > deep knowledge of the security aspects of the hashing algorithms up > for discussion. I'm not sure if you're trying to insult me or just make a random point. Even letting that pass by, I find quite a difference between not having a 'deep knowledge' of something, and not having a 'basic knowledge'. > This is sometimes how we learn. If you feel the thread > wastes time, you can just ignore it (as you seem to have done with the > manifest hashes thread after a few critical responses, somewhat to my > disappointment). Ignoring threads on thread that is mostly abandoned to terribly low level of posts frequently results in people putting their terrible ideas without even bothering to wait for a competent reply. As for that Manifest thread, I didn't notice any post that would request any reply. As far as I can see, it was mostly hijacked into 'why we still don't have proper verification, and why asking questions about it does not make it happen?!' -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On April 9, 2017 7:04:13 PM PDT, "William L. Thomson Jr." wrote: >The present system is a PITA for users. Fiddling with adding/removing >targets for Python/Ruby. As an ordinary user, that does sound like a real annoyance. As an ordinary user, I also never do it. I don’t have any targets set by hand. I probably never will. And yes, I do some Python development myself (not much packaging but “using” Python in the sense of writing Python code). I find the Python experience largely painless: I currently have 2.7.12 and 3.4.5 installed. Eventually 3.5 will get installed and 3.4 will go away. Just like every other package. I won’t need to do any config file editing, just a revdep-rebuild run perhaps. So regardless of the situation for maintainers, as a user, I don’t see this pain. -- Christopher Head signature.asc Description: PGP signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
(CC-ing comrel@) On pon, 2017-04-10 at 13:49 -0400, William L. Thomson Jr. wrote: > and Python support in Gentoo (which I use) a mess long-term. What is > > even worse, you do that without even talking to the Python team, or > > even bothering to CC them -- what you do instead is start a public > > discussion about Python without even bothering to invite Python > > people to it. > > This discussion is about more than python. You are ONE member of the > team, with at least 17 others. You are also NOT the Python Team lead. I don't see how attempting to discredit me is a fact regarding your idea. > > Who do you think you are, to approach me or ANYONE such way? You are > one person. The word team does not mean I, MGORNY Personal attack. Does not seem very factual. > > Now to step back. The post here is to engage in discussion with said > teams. But rather than address Python directly, and Ruby directly. I > chose to come here to address BOTH. The problem is not unique to one or > the other. > > What if Ruby team had ideas that could benefit Python? Would they know > if the interaction is happening just with the Python team? Think, how > do you reach more than one Gentoo team? What if an idea effects more > than one? > > I chose the right list, and the correct method. Even if it does not > suit some individuals opinions. Or their preferred way of dictating how > others conduct themselves. Except that the constant low level of posts on this list has resulted in most of the developers avoiding it. If you cared about opinion of the teams, you should have CC-ed them. > > > FYI, if you want to change something, the first research you ought to > > do is to ask 'why is it done this way?' Not jump to some random > > points that might be completely irrelevant. > > I understand, and I believe there are better ways. If you are not > capable of coming up with any better ways. That is your own personal > limitation. > > Again to add/remove a new python/ruby version requires touching every > python/ruby ebuild. You think that is efficient or the best way? Are > you, mgorny, adding/removing these python/ruby targets to lots of > ebuilds? This is the best *working* way. I don't see you being able to figure out a way that wouldn't randomly result in huge semi-random breakages of dependency tree (as others have already pointed out), and that wouldn't in the end require even more effort to fix them and keep people able to upgrade anything without hitting huge conflicts. > > I do not see any of that here. Guess you leave that work to others > https://github.com/gentoo/gentoo/commits?author=mgorny > > Not to mention your 2,033 commits, across the life of Gentoo Git repo. > With some 1600+ python packages. Just modifying the COMPAT would > increase your commit count considerable. I believe you are not doing > this work, leaving it to others. > > You can prove me wrong with commits. Should be close to at least 100 > commits. If you are doing serious python work. Once again, you are focusing on attempting to discredit me by throwing some random useless stats. This is how factual you are. > > > Well, I've opened the first ebuild in your overlay [1] and it wouldn't > > pass basic code review. > > Your review. Which your review of Firebird introduced REQUIRED USE > flags that did not work and broke the package. Despite the issue being > addressed a 1 line fix. You wanted the entire ebuild revised and > introduced a much larger issue that did not exist in the first place. Again, you're attempting to discredit me, through some semi-relevant oversimplification of history. And I'm afraid all that is proven by this example is that your ebuild skills are seriously lacking and you refuse to learn, and just rage quit. > > I use repoman on my overlay. If something does not meet QA. Then go > modify repoman to point such out. If repoman is not pointing it out, > then is it really an issue? Not every single issue can be caught correctly by an automated system. That's why we still employ people rather than leaving everything to be done by machines with simple algorithms. > > Maybe just in mgorny's mind This is a clear personal attack, not a fact. > > > For a start, it doesn't enforce USE > > dependencies which are absolutely necessary for anything to work by > > omething else than mere accident. It also explains why you are able > > to claim that your solution works. > > I have not implemented what I am suggestion. I fail to see how you can > say something not implemented fails to work. What is your case study? > Have you re-wrote python eclasses to no use TARGETS? My point is that if you do not know how to write correct Python ebuilds, you do not have a correct test case to even start planning out your idea. > > > [1]:https://github.com/Obsidian-StudiosInc/os-xtoo/blob/master/dev-pytho > > n/python-efl/python-efl-.ebuild > > That is new, and FYI mostly a copy form what is in TREE... Go diff and >
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Mon, 10 Apr 2017 10:58:10 -0700 Christopher Head wrote: > On April 9, 2017 7:04:13 PM PDT, "William L. Thomson Jr." > wrote: > >The present system is a PITA for users. Fiddling with adding/removing > >targets for Python/Ruby. > > As an ordinary user, that does sound like a real annoyance. As an > ordinary user, I also never do it. I don’t have any targets set by > hand. I probably never will. This is why it is not an issue for you. Your basically saying I do not care what version of Python is on my system. I do not care how many versions of python. I mentioned in a post, doing a wildcard on the targets being the ONLY painless option for users. > And yes, I do some Python development > myself (not much packaging but “using” Python in the sense of writing > Python code). I find the Python experience largely painless: I > currently have 2.7.12 and 3.4.5 installed. Are you running stable? There are other versions in tree. 3.4, 3.5, 3.6. If you were running unstable, you would have 4 pythons, including 2.7. That you only have 2 seems like you are running stable. If your writing new python code against say 3.4 and not 3.6. Not sure about that... Seems like it would keep things bound to older versions and never let things move forward. Usually when writing new code, you use the latest version of stuff. Not always but usually best. If anything make code support older while targeting newer. > Eventually 3.5 will get > installed and 3.4 will go away. Just like every other package. I > won’t need to do any config file editing, just a revdep-rebuild run > perhaps. So regardless of the situation for maintainers, as a user, I > don’t see this pain. Because you are not setting or dealing with the targets. You went with the mindless approach. Like doing a wildcard on USE flags. Your enabling support for all versions across the board for anything that supports it. That is quite a different experience if you go trying to use a specific one. -- William L. Thomson Jr. pgptnPrHpN0yq.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Mon, 10 Apr 2017 20:10:44 +0200 Michał Górny wrote: > > I don't see how attempting to discredit me is a fact regarding your > idea. You may assume what ever. I simply pointed out you are 1 of on a team of many. I have no requirement or duty to bring my ideas to you. If anything maybe the team lead. None the less, this issue crosses teams. Thus -dev ml and not directly with teams individually. Another thing you have missed. > > > > > Who do you think you are, to approach me or ANYONE such way? You are > > one person. The word team does not mean I, MGORNY > > Personal attack. Does not seem very factual. Stating a fact is not an attack. But the previous statement stands. Funny you send a copy to comrel. You start with insults much greater than any I lobbed your way. Yet instead of being man and taking what your shoveling out. You run off to the police Really funny, but I did not personally insult you as you did with your statements of ignorance, etc. If comrel was to act, it should be against you for your emails. Starting with your first. You should not approach anyone that way on a public list. > Except that the constant low level of posts on this list has resulted > in most of the developers avoiding it. If you cared about opinion of > the teams, you should have CC-ed them. You do not need to tell me or anyone to contact teams etc. Again if one team had a good idea. How would the other team know? Having redundant conversations on two lists with two groups is pointless. Kinda like spending time adding/removing targets from ebuilds. I am sorry you do not agree with my approach. But that is your opinion. > This is the best *working* way. I don't see you being able to figure > out a way that wouldn't randomly result in huge semi-random breakages > of dependency tree (as others have already pointed out), and that > wouldn't in the end require even more effort to fix them and keep > people able to upgrade anything without hitting huge conflicts. The idea here is to discuss better ways. This same thing happens to Java, Perl, and PHP. I think if those three can manage a better way. Python and Ruby can as well. Packaging things like Java is CONSIDERABLY more difficult than most other languages. If Java can do it, so can others. There used to be several Java VMs etc. There still are at least 3, Oracle, OpenJDK, and IBM. Go add JDK 9. See what that process is like and what all it entails. > Once again, you are focusing on attempting to discredit me by throwing > some random useless stats. This is how factual you are. Take it how ever you will. I am talking about WHO will do the work. Your commenting on something you are not doing yourself. That is not discrediting. Its showing you are not spending your time on this. But you expect others to. This is similar to the eapply thing of patch p1. Which I do not disagree with. But that also means allot of working patches need be modified just for that change. I do not like major changnes like that. When the people initiating the change are not doing the work. Pointing out that you are not the one managing python targets. Its showing you are not spending your time on this. If you were, I think you would feel otherwise. I think you would look to make things better since you would be doing minor edits on hundreds of ebuilds. That you are not doing that. Your trying to avoid something that may reduce work for others. Work that you are not doing. Yet your saying it is bad. Maybe let those who are managing the PYTHON_COMPAT in ebuilds to comment. I doubt they like that, or thing it is a efficient use of their time. > Again, you're attempting to discredit me, through some semi-relevant > oversimplification of history. And I'm afraid all that is proven by > this example is that your ebuild skills are seriously lacking and you > refuse to learn, and just rage quit. No that was a fact. You thought you were doing QA and making things better. You were not using the package nor testing out changes you were suggesting. I assuming you knew better allowed it, and others had to fix. I had a 1 line fix to correct an issue with logrotate permissions https://bugs.gentoo.org/show_bug.cgi?id=547442 Your series of comments here https://github.com/gentoo/gentoo/pull/101 Let to the entire revision of the ebuild, per your QA https://github.com/gentoo/gentoo/pull/154 Which you put in tree https://github.com/gentoo/gentoo/commit/9b00135f4696e539a3cbee711ac687f4f9ded105 However you broke things and missed others Bug you created https://bugs.gentoo.org/show_bug.cgi?id=577956 A user fixed with more fixes to the ebuild you missed. https://github.com/gentoo/gentoo/pull/3357 > > > Maybe just in mgorny's mind > > This is a clear personal attack, not a fact. Get over yourself. If you are concerned with being attacked. Maybe do not attack people to begin with. Your first post set the tone. Which another commented on before I did. > My point is that
Re: [gentoo-dev] Reverse use of Python/Ruby versions
Ühel kenal päeval, E, 10.04.2017 kell 14:44, kirjutas William L. Thomson Jr.: > Again go modify a few hundred python packages to remove say 3.4. I > think about 10-20 ebuilds in. You will be scripting and looking for > another way No, for that you simple remove python3_4 from _PYTHON_ALL_IMPLS in python-utils-r1.eclass and call it a day. Some other day you might make a mass commit to remove 3_4 from PYTHON_COMPAT of all in-tree ebuilds, but that's just janitorial and no other effect. The current implementation makes perfect sense to me, and follows one of the zens of python: Explicit is better than implicit. Declare explicitly what version is supported, don't implicitly do so and merely hope there are no issues. If some lower level module doesn't work with new python and your higher level module wants to use the new python, repoman will catch it for you due to it having been explicit via PYTHON_USE_DEP. There is no difference with reverse approach if one would mass commit the new COMPAT into all ebuilds upon the introduction of a new python slot, but this is not done, because things would break. Mart
Re: [gentoo-dev] Reverse use of Python/Ruby versions
> If Java can do it, so can others. And here I come with my 5¢. And my point here is simple: No, Java (Team) can't. Every time I come to Java team with some report they suggest (as joke, partially) to become a "full" developer (but not a contributor) and take care of this by myself. And they never fixed the reported issue in less than few month. And even if I doing a PR, it can take an eternity to be merged. It looks like their infrastructure is so brainf**ing, that they prefer to slack instead of doing maintenance. I asking excuse of every Java team member, if my words hurt any one of them, but that is just my vision of the situation.
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Mon, 10 Apr 2017 21:57:10 +0300 Mart Raudsepp wrote: > Ühel kenal päeval, E, 10.04.2017 kell 14:44, kirjutas William L. > Thomson Jr.: > > Again go modify a few hundred python packages to remove say 3.4. I > > think about 10-20 ebuilds in. You will be scripting and looking for > > another way > > No, for that you simple remove python3_4 from _PYTHON_ALL_IMPLS in > python-utils-r1.eclass and call it a day. Some other day you might > make a mass commit to remove 3_4 from PYTHON_COMPAT of all in-tree > ebuilds, but that's just janitorial and no other effect. Ebuilds still must be touched right? Even if just for house cleaning. That is some 1600+ ebuilds right? What about when a new version is added? Re-touch all those same ebuilds right? Am I missing something? -- William L. Thomson Jr. pgp9MRBcGURgH.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Tue, 11 Apr 2017 02:31:54 +0700 "Vadim A. Misbakh-Soloviov" wrote: > > If Java can do it, so can others. > > And here I come with my 5¢. And my point here is simple: > > No, Java (Team) can't. Ah, you appear to be thinking of the Gentoo Java team as it currently exists, rather than the mythical perfect Gentoo Java team that existed ten years ago and which will rise again soon, which is what William meant. -- Ciaran McCreesh
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On 10/04/2017 19:58, Christopher Head wrote: > On April 9, 2017 7:04:13 PM PDT, "William L. Thomson Jr." > wrote: >> The present system is a PITA for users. Fiddling with adding/removing >> targets for Python/Ruby. > > As an ordinary user, that does sound like a real annoyance. As an ordinary > user, I also never do it. I don’t have any targets set by hand. I probably > never will. And yes, I do some Python development myself (not much packaging > but “using” Python in the sense of writing Python code). I find the Python > experience largely painless: I currently have 2.7.12 and 3.4.5 installed. > Eventually 3.5 will get installed and 3.4 will go away. Just like every other > package. I won’t need to do any config file editing, just a revdep-rebuild > run perhaps. So regardless of the situation for maintainers, as a user, I > don’t see this pain. > As another regular user, you most definitely will see this pain if you need to deviate from your profile defaults for python. I'm like you - use lots of python, package some, write some. I also don't go past the current ~arch python-3 because I have a good sense of what waits for me if I do. That you and I don't suffer too much breakage at all since years now is a testament that *someone* is touching all those ebuilds when they need to be touched, that they are managing to do it without much visible fallout is a minor engineering miracle or sheer hard work. I think William has a point; sometimes making a criteria a negative one result in a lot less work. A good survey usually gives numbers that let you tell if it will. -- Alan McKinnon alan.mckin...@gmail.com
Re: [gentoo-dev] Reverse use of Python/Ruby versions
Ühel kenal päeval, E, 10.04.2017 kell 15:38, kirjutas William L. Thomson Jr.: > On Mon, 10 Apr 2017 21:57:10 +0300 > Mart Raudsepp wrote: > > > Ühel kenal päeval, E, 10.04.2017 kell 14:44, kirjutas William L. > > Thomson Jr.: > > > Again go modify a few hundred python packages to remove say 3.4. > > > I > > > think about 10-20 ebuilds in. You will be scripting and looking > > > for > > > another way > > > > No, for that you simple remove python3_4 from _PYTHON_ALL_IMPLS in > > python-utils-r1.eclass and call it a day. Some other day you might > > make a mass commit to remove 3_4 from PYTHON_COMPAT of all in-tree > > ebuilds, but that's just janitorial and no other effect. > > Ebuilds still must be touched right? Even if just for house cleaning. > That is some 1600+ ebuilds right? What about when a new version is > added? Re-touch all those same ebuilds right? After testing they actually work with the new version, instead of throwing known breakages onto ~arch users. > Am I missing something? You are missing the fact that this is pure janitorial in case of removal of a python3 SLOT, which can be done with scripts in one big commit. The effective change is all done in one touch of some eclass and then any 3_4 in PYTHON_COMPAT just doesn't have any effect. So removal of old stuff is not a concern whatsoever. The janitorial part doesn't have to be done, but because there are scripts that can do it mostly automatically one evening, it can get done after it's sure it won't have to be re-added for some reason in the eclass.
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Mon, 10 Apr 2017 20:38:22 +0100 Ciaran McCreesh wrote: > On Tue, 11 Apr 2017 02:31:54 +0700 > "Vadim A. Misbakh-Soloviov" wrote: > > > If Java can do it, so can others. > > > > And here I come with my 5¢. And my point here is simple: > > > > No, Java (Team) can't. > > Ah, you appear to be thinking of the Gentoo Java team as it currently > exists, rather than the mythical perfect Gentoo Java team that existed > ten years ago and which will rise again soon, which is what William > meant. Wow, someone actually has a clue about the state of Java :) The Java team then handled the migration from 1.4 to 1.5. If you know anything about Java code that was NOT trivial. A system was developed not only to allow that transition but make it so the same would never need to be repeated in the future. That is even a Java quiz question. Q2. What was the big deal about Java 1.5? Why did it take so long to become unmasked? Q3. Will the new Java system support Java 1.6 when it is released? Java 1.7? Or what about java 1.8 in tree, or 9 coming... https://wiki.gentoo.org/wiki/Project:Java/Developer_Quiz Which NO other area of the tree has a 3rd quiz. Not Perl, PHP, Python, nor Ruby. Since Java is so trivial to package. Why its actively worked on in Gentoo Love to see people maintaining ebuilds for these other languages try on Java for size Put your big boy pants on. Try to package Hadoop. I am ~3 months into Jenkins from source Most go into tree as -bin People are funny :) -- William L. Thomson Jr. pgp1pFWa5jhtJ.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Mon, 10 Apr 2017 22:51:35 +0300 Mart Raudsepp wrote: > > After testing they actually work with the new version, instead of > throwing known breakages onto ~arch users. Ebuilds cannot use the new version till it is added to their PYTHON_COMPAT correct? There does not seem to be any way around touching all ebuilds when adding a new version. > > Am I missing something? > > You are missing the fact that this is pure janitorial in case of > removal of a python3 SLOT, which can be done with scripts in one big > commit. Ok I concede on removing older versions. Lets put old version aside. What about adding new? You still have to add the new version to PYTHON_COMPAT in each ebuild right? What about users? Do they need do anything if they have specific targets set? Or all should just use Wildcard and if in ~arch have 3-4+ pythons. Even if house cleaning, removing old is not an issue. You still have adding new, and the end user experience. -- William L. Thomson Jr. pgp_B43QIoO_4.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
> painless option for users. Well... If a bit of mind work is pain... So, then I'd say that Gentoo is not about avoiding such pain. Did you hear about Gentoo Philosophy? It says that point of Gentoo to appear was to give users possibility to make exact "tool" they wants to use, but not decide anything for them. So, if a person wants to avoid thinking about some aspect of the system maintenance, then Gentoo is not recommended for this person and the person should consider to use some distro made by "Big Fat Corporations" (like Ubuntu, Fedora/RHEL, SuSE and whatever). They're very like to dictate what and how should user do, and what should not. And there is all that things already decided by BigBro's and users should not take care of any of that. I can't understand people who refuse to get as complete knowledge as possible about tools/instruments they using (including OS). Would you learn how does a hammer work before applying it to the nails? Or do you say "The only painless option for users is to make it spheric" instead? And same for Gentoo: if you want to use something — please, consider to get a bit knowledge about how do it working. And it will be huge karma bonus for you, if you'll research the reasons why did it done in that way, and not another before complaining (why did wheels invented to be round, but not square? Square wheels are more stable on a flat surface, while round ones makes wagon to constantly move, while I loading my baggage on it). I mean: most of the time, if you having trouble with something, it is most likely *you* (not you personally, but some abastract Freud-ish "you") doing something wrong, then the tool you're using is badly designed. And if you dislike the tool's design, you're free to take analogous tool from the rivals (and take that one you'd like). Thanks for attention, wbr, mva
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Mon, 10 Apr 2017 16:01:55 -0400 "William L. Thomson Jr." wrote: > > Ok I concede on removing older versions. Lets put old version aside. > > What about adding new? You still have to add the new version to > PYTHON_COMPAT in each ebuild right? > > What about users? Do they need do anything if they have specific > targets set? Or all should just use Wildcard and if in ~arch have > 3-4+ pythons. > > Even if house cleaning, removing old is not an issue. You still have > adding new, and the end user experience. What about dependencies? Their slots must be increased as well right? In fact that effects removal. You cannot remove an old version if any depend on that slot specifically. Rather in Java's case. If an older is removed, the ebuild does not need to be modified ever Nor if a new one is added unless it breaks. -- William L. Thomson Jr. pgpROkdO5qH88.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
Ühel kenal päeval, E, 10.04.2017 kell 16:01, kirjutas William L. Thomson Jr.: > On Mon, 10 Apr 2017 22:51:35 +0300 > Mart Raudsepp wrote: > > > > After testing they actually work with the new version, instead of > > throwing known breakages onto ~arch users. > > Ebuilds cannot use the new version till it is added to their > PYTHON_COMPAT correct? > > There does not seem to be any way around touching all ebuilds when > adding a new version. > > > > Am I missing something? > > > > You are missing the fact that this is pure janitorial in case of > > removal of a python3 SLOT, which can be done with scripts in one > > big > > commit. > > Ok I concede on removing older versions. Lets put old version aside. > > What about adding new? You still have to add the new version to > PYTHON_COMPAT in each ebuild right? > > What about users? Do they need do anything if they have specific > targets > set? Or all should just use Wildcard and if in ~arch have 3-4+ > pythons. > > Even if house cleaning, removing old is not an issue. You still have > adding new, and the end user experience. The user experience is suboptimal either way. Some ideas to improve that seems to be e.g something like Kent brought up. But all this requires manpower and so on to actually do; potentially also limiting potential manpower to whom has or can get some deep graph theory knowledge. Explicitly adding things is better, so you don't get some huge unknown breakages of some lower level module not working and then having to work your way towards all consumers and adding the fact they don't work there too, because a dependency doesn't work. The reverse is not feasible due to the breakages, and when you are entering automated testing to catch breakages, you might as well do automated testing and semi-automated addition to PYTHON_COMPAT for stuff that succeeds in this automated testing. We are stuck on defaulting to 3_5 mostly because not everything having 3_5 in COMPAT isn't stable yet or whatnot, combined with python teams conscious decision to tie the stabling of a new dev-lang/python SLOT (that didn't have stable versions before) to flipping default PYTHON_TARGETS as well, and then the tracker bug for that being delayed or something. Mart
Re: [gentoo-dev] Reverse use of Python/Ruby versions
To back up a bit. I package Java why do I care about Python and Ruby? 1. Its annoying as a user to fiddle with targets, short of doing a wild card and having multiple versions. 2. Unlike most other languages. Java has support for other languages. Running PHP, Python, and Ruby on the JVM. This java package depends on Ruby https://github.com/Obsidian-StudiosInc/os-xtoo/tree/master/dev-java/jruby Which in working with latest version of Ruby 2.4.x there was an API change, which broke a Spring dependency https://github.com/spring-projects/spring-framework/pull/1344 https://github.com/Obsidian-StudiosInc/os-xtoo/blob/master/dev-java/spring-context/files/jrubyexception.patch This java package depends on Python https://github.com/gentoo/gentoo/tree/master/dev-java/jython While other languages package just themselves. Not sure I have ever seen a perl, php, python or ruby package that supports Java. As in running Java code on perl, php, python, or ruby. At best its optional Java support. Java is a different beast, and people run those languages in Java... And others, Groovy, Scala, Clojure, etc -- William L. Thomson Jr. pgpJrs_Ww5MOf.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
Am I right in assumption that you arguing about *_TARGETS rework to be enabled by default for packages that was not tested on this TARGETs with ... hardness of packaging java software?.. Or does it just argmentum ad verecundiam (with argumentum ad hominem partially)? And yes, I personally packaged Java software from scratch (including all the depends). And yes, some times I even thought "F**k this sh*t!" (but finished packaging afterwards). And yes, I packaged Go software. And yes, I packaged NodeJS software. And no, they are NOT much easy to package then Java one (even including they still have no TARGETS... As java? :D). But how does it apply to TARGETS logic breakage? The purpose of TARGETS is that package holds only that TARGETs that it was tested to work against. It is wrong to have any targets enabled by default for all the packages and removing in case if it is broken. Instead, if new target appeared a month (year, decade) ago, but package, that you're interested in, still doesn't support it... Well.. It meant, maintainer is a slacker and package is a candidate to last-rites and removal...
Re: [gentoo-dev] Reverse use of Python/Ruby versions
Ühel kenal päeval, E, 10.04.2017 kell 16:17, kirjutas William L. Thomson Jr.: > On Mon, 10 Apr 2017 16:01:55 -0400 > "William L. Thomson Jr." wrote: > > > > Ok I concede on removing older versions. Lets put old version > > aside. > > > > What about adding new? You still have to add the new version to > > PYTHON_COMPAT in each ebuild right? > > > > What about users? Do they need do anything if they have specific > > targets set? Or all should just use Wildcard and if in ~arch have > > 3-4+ pythons. > > > > Even if house cleaning, removing old is not an issue. You still > > have > > adding new, and the end user experience. > > What about dependencies? Their slots must be increased as well right? > > In fact that effects removal. You cannot remove an old version if any > depend on that slot specifically. The USE dep requirement on the old python target will go away with the removal of it in eclass and I don't know what slots have to do with it. This circles back to first gathering the basic knowledge of how these things work right now and why, even if I don't necessarily agree with the way this was pointed out. > Rather in Java's case. If an older is removed, the ebuild does not > need > to be modified ever Nor if a new one is added unless it breaks. Nor does in python, it's just a janitorial process to reduce the ebuilds by 2 bytes or something. One which can be scripted and rather easily pushed thanks to not CVS anymore.
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Mon, 10 Apr 2017 23:21:24 +0300 Mart Raudsepp wrote: > > The user experience is suboptimal either way. Some ideas to improve > that seems to be e.g something like Kent brought up. But all this > requires manpower and so on to actually do; potentially also limiting > potential manpower to whom has or can get some deep graph theory > knowledge. I hear you, but 3 other languages already do it another way. Java is easily as complex if not more complex. This problem was seen coming a long time ago with 1.4 -> 1.5. Any issues with Python/Ruby versions were and are still experienced with Java versions. Java is more complex all around. If it can be done for Java, it can be done for the others. It is more a question of will than man power. Any argument that can be made for Python or Ruby. Applies to Java, Perl and PHP, with minor differences. Less with PHP and Ruby, as those install location is based on version. Java is the only that does not install based on version. Anything in Gentoo that goes against the status quo gets heavy resistance and thus Gentoo does not change. But continues on with the status quo -- William L. Thomson Jr. pgpD7i7WH2uBE.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Tue, 11 Apr 2017 03:29:25 +0700 "Vadim A. Misbakh-Soloviov" wrote: > Am I right in assumption that you arguing about *_TARGETS rework to > be enabled by default for packages that was not tested on this > TARGETs with ... hardness of packaging java software?.. I think TARGETS should not exist. They do not for Java, Perl, or PHP. > And yes, some times I even thought "F**k this sh*t!" (but finished > packaging afterwards). You must have been packaging small stuff. Anything big requires tremendous time. Even for simple ebuilds. Short of using an automated tool to generate the ebuilds like the java-ebuilder https://github.com/gentoo/java-ebuilder > And yes, I packaged Go software. I have as well, it sucks, no versions. Rackspace rack command is in go https://github.com/Obsidian-StudiosInc/os-xtoo/tree/master/dev-go https://github.com/Obsidian-StudiosInc/os-xtoo/tree/master/app-admin/rack > And yes, I packaged NodeJS software. > > And no, they are NOT much easy to package then Java one (even > including they still have no TARGETS... As java? :D). I do not just package but maintain. This discussion is in part about ebuild maintenance. As you have to manage PYTHON/RUBY stuff in the ebuild. Not just the TARGETS users see. > But how does it apply to TARGETS logic breakage? If you haven't run into it, then your not aware. Its an update issue. You set a target to say Ruby 24. But something wants Ruby23. It could be it only builds with ruby23. Or more than likely no one has gotten around to adding it to the package. Since for every new version. EVERY ebuild must be touched. That is thousands wrt to Python, and hundreds wrt to Ruby. Even if scripted. Added a new version would take some time to update all ebuilds. Its silly work. > The purpose of TARGETS is that package holds only that TARGETs that > it was tested to work against. I am aware. Java could do things that way. So could Perl and PHP. Why is they do not? For anyone saying go look at Python Ruby. 3 other systems exist, those teams are ignoring. Coming up with a complex, cumbersome solution that the others are not plagued by. > It is wrong to have any targets > enabled by default for all the packages and removing in case if it is > broken. Instead, if new target appeared a month (year, decade) ago, > but package, that you're interested in, still doesn't support it... > Well.. It meant, maintainer is a slacker and package is a candidate > to last-rites and removal... All you should have to do is set the python/ruby versions via eselect. Eclasses and ebuilds should handle the rest as they do for the other 3 main languages. -- William L. Thomson Jr. pgpf0XP82GGyT.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On pon, 2017-04-10 at 16:33 -0400, William L. Thomson Jr. wrote: > On Mon, 10 Apr 2017 23:21:24 +0300 > Mart Raudsepp wrote: > > > > The user experience is suboptimal either way. Some ideas to improve > > that seems to be e.g something like Kent brought up. But all this > > requires manpower and so on to actually do; potentially also limiting > > potential manpower to whom has or can get some deep graph theory > > knowledge. > > I hear you, but 3 other languages already do it another way. Java is > easily as complex if not more complex. This problem was seen coming a > long time ago with 1.4 -> 1.5. Any issues with Python/Ruby versions > were and are still experienced with Java versions. > > Java is more complex all around. If it can be done for Java, it can be > done for the others. It is more a question of will than man power. Any > argument that can be made for Python or Ruby. Applies to Java, Perl and > PHP, with minor differences. Less with PHP and Ruby, as those install > location is based on version. Java is the only that does not install > based on version. The difference is in quality expectations. We did Python this way to make sure things will work, and all obvious breakage will immediately be caught. Your alternative does not provide for that. > > Anything in Gentoo that goes against the status quo gets heavy > resistance and thus Gentoo does not change. But continues on with the > status quo > You are talking *nonsense*. The python-r1 was *against* status quo. We changed it. Now you want the old status quo back. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Reverse use of Python/Ruby versions
> or PHP. Wouldn't you be so kind to re-check this part, please? :)
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On pon, 2017-04-10 at 16:40 -0400, William L. Thomson Jr. wrote: > On Tue, 11 Apr 2017 03:29:25 +0700 > "Vadim A. Misbakh-Soloviov" wrote: > > > Am I right in assumption that you arguing about *_TARGETS rework to > > be enabled by default for packages that was not tested on this > > TARGETs with ... hardness of packaging java software?.. > > I think TARGETS should not exist. They do not for Java, Perl, or PHP. I'm sorry but do you even use Gentoo, these days? Like the real Gentoo, not just some little part you've installed years ago and then modified only Java stuff in it? Perl does not use TARGETS. It uses subslots, after it used horrible custom rebuild tool. The latter brought many bug reports of users being hit by random breakage on upgrades, the former just brings *tons* of problems with Portage not being able to deal with Perl upgrades. PHP *uses* PHP_TARGETS. Python used not to use TARGETS. The results were random incompatibilities between packages that were hard to track and random breakage. Now we're past that. But I can understand it's not the Gentoo of your times where user was expected to watch his every step to have his system boot again. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Reverse use of Python/Ruby versions
Also, > Its an update issue. You set a target to say Ruby 24. But something > wants Ruby23. It could be it only builds with ruby23. Or more than > likely no one has gotten around to adding it to the package. Since for > every new version. EVERY ebuild must be touched. As I said above, this only happens if package manintainer is a slacker and a package wasn't touched for years. Chances that it will work with new ruby is... about 0%. Why should you auto- add modern ruby targets for it then? Instead, you should blame package (that causes regression) maintainer and/or take maintenance in your hands (if you need that package), or just drop it from your system (if not). // Although, it is another question to discuss: Most of time in such situations (with ruby crap mess, lol) Portage is unable to tell which exact package is guilty in all that crap (even with --verbose- conflicts --backtrack=100500 and so on) and you need to mask old rubys and re- run world-upgrade to catch the one who fails because of mask. I agree that it is not expected portage bahaviour, but I have not done deep research to write detailed report and suggest a solutions for this problem, abd just reporting it was useless, since, predictably, nobody cares (everybody ok with this). That ^ is the point to discuss. But I disagree that '>=foo-1 <=foo2' stuff should be used instead of targets.
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Mon, 10 Apr 2017 22:51:11 +0200 Michał Górny wrote: > > I'm sorry but do you even use Gentoo, these days? Like the real > Gentoo, not just some little part you've installed years ago and then > modified only Java stuff in it? Um yes... Maybe someday you will learn to stop assuming Having so many systems running Gentoo is one of the few reasons I still run Gentoo. Its considerable work to move to another. Also unlike many developers. My Laptop and Desktop are also gentoo. Not just all my servers I have run Gentoo on everything since 2002. I have my own business and lots of servers. Let me repeat 100% Gentoo since 2002. What ever the "real" Gentoo is. I was given access to Funtoo and could contribute. But I have never even messed with putting it on any of my systems Really funny to assume otherwise You could also do a search on Bugzilla. If I wasn't using stuff, why comment on some bugs... I do it very sparingly due to people like you. Why I do not do any PRs. If I had no gentoo systems. Why would I have taken over the Portage Ansible module. Despite my hatred for Python... https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/packaging/os/portage.py > Perl does not use TARGETS. It uses subslots, after it used horrible > custom rebuild tool. The latter brought many bug reports of users > being hit by random breakage on upgrades, the former just brings > *tons* of problems with Portage not being able to deal with Perl > upgrades. I am aware. One of the main applications I use and packaged for some time is ASSP. Anti Spam Server Proxy written in perl. It is not small nor trivial. ( Horribly outdated in tree as I was the mainatiner... ) https://github.com/Obsidian-StudiosInc/os-xtoo/tree/master/mail-filter/assp I have never had issues maintain perl ebuilds. I do not have to mess with versions in them most times. https://github.com/Obsidian-StudiosInc/os-xtoo/tree/master/dev-perl > PHP *uses* PHP_TARGETS. I see that, I have mine set to nothing, so wildcarded I guess. That said I have only every see one version of PHP installed not more than one. I only run that on nagios servers. Rather OpenNMS but its not trivial to package. Though been years since I last looked at it. > Python used not to use TARGETS. The results were random > incompatibilities between packages that were hard to track and random > breakage. Now we're past that. But I can understand it's not the > Gentoo of your times where user was expected to watch his every step > to have his system boot again. This has nothing to do with booting. This BS broke my build server. Many times over many years have I had to mess with Python targets. Now with Ruby its double. It was mostly the headache as a system admin having emerges not run due to unmet requirements etc. The Rubty 23/24 issue was new. Since I did work with Ruby 24 sometime back for spring-context. This as a month ago https://github.com/Obsidian-StudiosInc/os-xtoo/tree/master/dev-java/spring-context I have been running with just Ruby 24 targets that hole time. Then my build server and manual updates were prevented till I took action. I ended up having to mask the crap out of ruby to keep < 24 off my systems. Also dropped a couple desktop packages I did not need that was bring in ruby on those systems. It was nice to have nothing change and builds fail. Which seems things modified as I am still only Ruby 24 and some of the things that were trying to bring in 23 were updated. Look at the recent commits, you see add ruby24 https://github.com/gentoo/gentoo/commits/master/dev-ruby -- William L. Thomson Jr.
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Tue, 11 Apr 2017 04:17:31 +0700 "Vadim A. Misbakh-Soloviov" wrote: > Also, > > > Its an update issue. You set a target to say Ruby 24. But something > > wants Ruby23. It could be it only builds with ruby23. Or more than > > likely no one has gotten around to adding it to the package. Since > > for every new version. EVERY ebuild must be touched. > > As I said above, this only happens if package manintainer is a > slacker and a package wasn't touched for years. > > Chances that it will work with new ruby is... about 0%. Why should > you auto- add modern ruby targets for it then? See my other post where I talk about recent breakage that did not exist a month ago. When I was updating jruby for spring-context. Which a month later ends up preventing nightly builds on my build server. In fact I spent several hours yesterday trying to deal with the recent Ruby situation. After days if it not building or being fixed in tree. Git commit shows ruby24 being added. Something must have been a dep that was ruby 23 thus it wanting to come back onto my systems, Despite having been gone for a month. I ended up masking < Ruby 24. That was a PITA. It kept dropping. Masked 23 it went down to 22, masked 22, it dropped to 21. Not to mention all the other headaches before I went to hard mask the package. -- William L. Thomson Jr.
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Tue, 11 Apr 2017 03:48:37 +0700 "Vadim A. Misbakh-Soloviov" wrote: > > or PHP. > > Wouldn't you be so kind to re-check this part, please? :) > I was incorrect, PHP has targets. The systems I have it on just have PHP_TARGET="" Which is the wild card solution I was saying was the only headache less option for users. Still does not change the work from the maintainer perspective either. As mentioned though. I do not recall ever seeing more than one PHP version installed. I do not even recall doing eselect php. If I try now it fails # eselect php list !!! Error: Please choose one of the following modules: cli apache2 fpm cgi phpdbg No clue what that is about. PHP is working fine for nagios. -- William L. Thomson Jr. pgpO1U_ijPhHV.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Mon, 10 Apr 2017 11:52:02 -0400 "William L. Thomson Jr." wrote: > > > > Meanwhile, you cannot build two parts of a given python dependency > > chain with different pythons, nor different perls. > > True but this is not changing how things work, just reversing. You mean going back to the old days where you'd upgrade Perl, and your system would simply be broken, and you'd need to run some 3rd party tool outside of portage to fix it, otherwise all your compiles would randomly break because portage had no way of knowing that everything Perl based was now broken? Or where we had python-updater for the same reasons? TARGETS is a *solution* to these kinds of problems. Don't dismiss the solution simply because you don't understand the problem in the first place. Its not *ideal*, but its far better than things were. We actually *do* have many users now having effortless upgrades as a result of both targets and subslot usage, which you'd have us repeal. 1. Install new perl, subslots change, portage reinstalls everything ( portage currently does a shit job though of getting this right in all cases, but enough users have it JustWork and perl-cleaner is only necessary because they had ages of old stuff that wasn't installed with subslot binding ) 2. Globally change PYTHON_TARGETS, all things needed to be rebuilt with python get rebuilt. "Lets not rebuild" is _not an option_. Not rebuilding is simply opting to have a broken system. You need to present a strategy for causing the rebuild, and presenting a rebuild that doesn't result in end users having a fractal of brokenness. And a "Supports everything by default, but then we mask things off as broken" creates a "broken by default" system. > > > Right, but this is impossible with Ruby, Python, and Perl. > > It is done right now. The problem your describing exist now and is > addressed. This would address it the same way but reversed. Its *not* addressed now. For instance, if you recompile Perl with USE=threads or USE=debug, you have to recompile literally every package you have installed. We have no solution for this in place, and we have tempted the idea of a TARGETS analogue for a few years now. The only reason it *kinda* works now is because Perl's upstream takes massive efforts to ensure that when they break things, the people whos shit got broken is fore-warned of the impending doom, with aggressive smoke testing on the pre-releases. Subsequently, the fact we don't already have Perl with TARGETS is simply because the breakage is typically fixed upstream *before* it hits us. If shit is broken sufficient that it won't work on a newer Perl, that's a tree breakge for us. And its soley by the grace of God that such a thing has not been significant enough to cause a blocker yet. Python and Ruby don't have any sort of culture to make this possible. > > Perl *could* have targets, and some people think could do with it, > > but it and java are very much in different boats. > > Easier to deal with as a user. Less work as a developer. Which kind of user? The user who lives on the bleeding edge and doesn't mind randomly having emerge break? What about the audience of users who bitch every time we stabilize a new version of Perl, because they're not ready for the changes? Its a good thing nobody users Gentoo in production, because we move far too fast for anyone to rely on our operating system for anything serious. There's plenty of codebases out there that still require older versions of Perl, and if the *massive* catastrophe of 5.26 doesn't abate, no doubt we'll need a way to concurrently install multiple Perl versions to support a legacy audience. And as soon as we have concurrent Perl versions available, a TARGETS analogue becomes a *necessity*, portage has no alternative. How else does portage understand the following facts: X package exists X package has 3 dependencies X package needs Perl 5.26 X therefor needs all 3 dependencies to be installed on Perl 5.26 That is, you can either slot *every* package in tree for *every* perl there is and create a *massive* maintainer burden ( read that as: I quit ), or you introduce targets, so that portage can track the interconnectedness of packages and what they're installed on. There's literally no other mechanism to do this at present. Python used to have an ENV hack that was *like* targets, except portage couldn't see it, and was basically just broken by design. Thus, going back to the old way is "still have targets, just you're fucked if you expect portage to help you with them and you'll need 3rd party tools and a daily battle with portage not understanding why its dependencies are broken" > Problem is simple, Targets are a PITA to deal with, ever changing. They > lead to issues when you select incompatible ones or options. Best case > wild card and let all install. But otherwise its a chore to deal with. > > > We only have 2 types of opt
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On pon, 2017-04-10 at 17:18 -0400, William L. Thomson Jr. wrote: > > Python used not to use TARGETS. The results were random > > incompatibilities between packages that were hard to track and random > > breakage. Now we're past that. But I can understand it's not the > > Gentoo of your times where user was expected to watch his every step > > to have his system boot again. > > This has nothing to do with booting. This BS broke my build server. > Many times over many years have I had to mess with Python targets. Now > with Ruby its double. It was mostly the headache as a system admin > having emerges not run due to unmet requirements etc. So, to summarize: you want to destroy a reasonably reliable dependency system in favor of thing that randomly explodes because you failed at hacking at it? Well, you've already dismissed the users for which it works out of the box... obviously they're not a proper Gentoo users if they don't break their system and then complain that Gentoo is doing everything wrong because they can break their systems. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Mon, 10 Apr 2017 22:43:18 +0200 Michał Górny wrote: > > The difference is in quality expectations. We did Python this way to > make sure things will work, and all obvious breakage will immediately > be caught. Your alternative does not provide for that. Add a new Java version and recompiling packages with it, will also immediately show breakage if any. If your saying Python code is of higher quality than Java. I would digress heavily on that. You have leniency in python not being strong typed. Lack of generics and stuff could only mean that could be worse. Relying on internals to handle data types for you. I found so many bugs in java-config when porting to C/Jem. Most were hidden due to how Python operates I never bothered filing bugs. I have mention of most all in my IRC logs. It was pretty considerable. > > Anything in Gentoo that goes against the status quo gets heavy > > resistance and thus Gentoo does not change. But continues on with > > the status quo > > > > You are talking *nonsense*. The python-r1 was *against* status quo. We > changed it. Now you want the old status quo back. Regardless of new eclass, the TARGETS remain. Things did not change from a user perspective. Recently packaging some ebuilds, the COMPAT/VERSION does not seem to have changed. Despite what ever changes to the eclass. -- William L. Thomson Jr. pgpqAcdBCmFhu.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
Ühel kenal päeval, E, 10.04.2017 kell 17:33, kirjutas William L. Thomson Jr.: > Add a new Java version and recompiling packages with it, will also > immediately show breakage if any. > > If your saying Python code is of higher quality than Java. I would > digress heavily on that. You have leniency in python not being strong > typed. Lack of generics and stuff could only mean that could be > worse. > Relying on internals to handle data types for you. Which is why python modules can't just pretend to work with a newer python by merely happening to "compile" and install. It is not strongly typed and it does not involve a AOT phase (pyc is just a semi-binary representation of the source code really) and issues are not found unless properly tested at runtime or test suite. > Regardless of new eclass, the TARGETS remain. Things did not change > from a user perspective. Recently packaging some ebuilds, the > COMPAT/VERSION does not seem to have changed. Despite what ever > changes to the eclass. Users don't get unexpected failures, as things that are claimed to work with a given python version, probably actually do so.
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Mon, 10 Apr 2017 12:03:51 -0400 "William L. Thomson Jr." wrote: > That is ALLOT of work to fiddle Unrelated to thread and is not intended as a "I'm better because I grammar well" thing, but this drives me nuts and I've bitten my tongue on it for months. But you use that word that isn't a word in the context you mean, frequently. You want *two* words, "a lot", which mean "a large volume of" "allot" is a *verb*, https://en.wiktionary.org/wiki/allot#Verb Which means "to proportion out" By analogy, this makes as much sense as if you'd written: "That is a distributing of work to fiddle" Or "That is an apportion of work to fiddle" Which is nonsense. I myself used to make this mistake, and now I just avoid the word in entirety as a defensive strategy. "I use that word a lot" -> "I use that word frequently" "It is a lot of work" -> "It is substantial work" "I have a lot of time" -> "I have significant time" "I will allot you 5 units of rice" -> "I will apportion you 5 units of rice" ( I try not to play grammar nazi, but when you make only one mistake that I notice, I ignore it, but when you make the same single mistake over and over and over again, on a daily basis, I feel somebody should point it out. Please accept my apologies for having some flavour of mental disorder for being triggered by this ) pgplTvl2hxvWX.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On pon, 2017-04-10 at 17:33 -0400, William L. Thomson Jr. wrote: > On Mon, 10 Apr 2017 22:43:18 +0200 > Michał Górny wrote: > > > > The difference is in quality expectations. We did Python this way to > > make sure things will work, and all obvious breakage will immediately > > be caught. Your alternative does not provide for that. > > Add a new Java version and recompiling packages with it, will also > immediately show breakage if any. Except that the packages don't get recompiled unless you take manual action to recompile them. If you fail at this action, you may end up having broken software because the rebuild has not been complete. > > > > Anything in Gentoo that goes against the status quo gets heavy > > > resistance and thus Gentoo does not change. But continues on with > > > the status quo > > > > > > > You are talking *nonsense*. The python-r1 was *against* status quo. We > > changed it. Now you want the old status quo back. > > Regardless of new eclass, the TARGETS remain. Things did not change > from a user perspective. Recently packaging some ebuilds, the > COMPAT/VERSION does not seem to have changed. Despite what ever > changes to the eclass. > TARGETS *have been added*. This is *the new way*. This *did change*. I have no clue why you pretend it's some ancient status quo when the remnants of old code were removed two months ago. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Mon, 10 Apr 2017 23:33:06 +0200 Michał Górny wrote: > > So, to summarize: you want to destroy a reasonably reliable dependency > system in favor of thing that randomly explodes because you failed at > hacking at it? Interesting comments. If the system is so well engineered. Why am I having to "hack" it as you call it? When did changing targets to only have 1 version of Ruby, or 2 pythons becoming hacking. I do like how it was phrased. It shows right there the issue. If ANYONE has to hack around it, it sucks > Well, you've already dismissed the users for which it works out of > the box... obviously they're not a proper Gentoo users if they don't > break their system and then complain that Gentoo is doing everything > wrong because they can break their systems. Only users who it works for, is those who are not wanting specific versions and not others. As in those who do not set the targets and let them be wide open, or wildcard. So they do not care what is installed. They are likely also not doing much with USE flags or other things. They obviously do not care what is on their systems. Most any system admin does care about what is on their system. Every other version is another potential for security issues etc. What good system adminstrator just installs needless stuff because they are lazy. Neither of these are good points. hacking nor users who do not care to even set targets -- William L. Thomson Jr. pgpqbdUJeLWgI.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Mon, 10 Apr 2017 16:01:55 -0400 "William L. Thomson Jr." wrote: > You still have > adding new, and the end user experience. You're running ~arch, I recall. This means adding new is slow for arch users. But it also means there's a clear line in the sand when something can be stabilized. ~arch is not great here, but that's why arch exists: ~arch is the buffer zone where the horrors are supposed to be exorcised. But as annoying as "oh, doesn't support new target yet" is, its much less annoying than "oh, it says it supports the new target, but actually doesn't, and now I have portage screaming at me to toggle use flags while I report this, and then some poor gentoo developer is going to have to recursively find all the broken dependents and remove their use flags" Its the same hell of keywording. Its much easier to *add* new keywords/useflags as repoman can trivially tell you if you made any mistakes, because repoman can only see how your package is, and how your dependencies are. *removing* useflags/keywords is much messier, because repoman can't tell you what you broke. Not without doing a full tree check, which takes 30 minutes+ on my hardware. Hence, that's the sort of problem I'm more inclined to throw grep at and then run it through an automated test PR to make sure I didn't break anything if I was really concerned. pgpC6jElve2z7.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Tue, 11 Apr 2017 03:29:25 +0700 "Vadim A. Misbakh-Soloviov" wrote: > The purpose of TARGETS is that package holds only that TARGETs that it was > tested to work against Targets are more than that. Targets also regulate compilation stage for concurrency. For instance: If you have 2 pythons. Imagine you have no TARGETS X depends on Y Now, X and Y both compile against both pythons. But you installed your packages like this: python 2.7 Y python 3.5 X Thus, when "Y" was installed, only python 2.7 could be an installation candidate, so it only installed against python 2.7 But now, "X" will compile against both python 2.7 and python 3.5, but horrors!... Python 3.5 doesn't have Y! Portage has no way of knowing this. introduce targets. install python 2.7 install Y with TARGETS='python2.7' install python 3.5 install X with TARGETS='python2.7' # no problem, because it doesn't try to compile against 3.5 But then you decide you wanted python 3.5 support after all TARGETS="python3.5" Install X X[python3.5] requires Y[python3.5] Portage recognizes Y doesn't have python3.5 , and forces the useflag change and a recompile before installing X with python3.5 THIS IS WHY WE HAVE THIS. The only reason this is "hell" is because users end up having to flip those toggles or set them globally, instead of portage intelligently auto-setting them on an as-needed basis. In essence, users have to micro-manage every portage decision, even though portage is making the decisions and the user is just bitching and rubber stamping it ;) pgpfuaHBaWKOj.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Tue, 11 Apr 2017 10:09:51 +1200 Kent Fredric wrote: > On Mon, 10 Apr 2017 16:01:55 -0400 > > You're running ~arch, I recall. Yes but most servers run stable. Though they have some ~arch packages. I may move to fully ~arch. I think stabilization on Gentoo is a misnomer. Usually newer stuff tends to be more stable than older with bugs etc. Some upstreams have release schedules that would ensure the package be outdated in Gentoo stable. My development env is all ~arch. That way I can be forewarned. Though I have allot of the same issues in stable systems. Why I do not really consider them to be such. > This means adding new is slow for arch users. Same for Java, and slower to get a JDK actually stabilized. Even worse with the from source java requirement. Since from source will most always lag binary from Oracle. > But it also means there's a clear line in the sand when something can > be stabilized. We went the route of masking and unmasking. > ~arch is not great here, but that's why arch exists: ~arch is the > buffer zone where the horrors are supposed to be exorcised. In the days I came to Gentoo. You were not allowed to break stuff in ~arch. Even though it did occur. It was very rare. If it was broken, it needed to be masked for testing till it can be safely unmasked. > But as annoying as "oh, doesn't support new target yet" is, its much > less annoying than "oh, it says it supports the new target, but > actually doesn't, and now I have portage screaming at me to toggle > use flags while I report this, and then some poor gentoo developer is > going to have to recursively find all the broken dependents and > remove their use flags" I had the issue where a month ago I swapped everything to Ruby 2.4. Then something changed, in the deps. Something wanted Ruby 2.3. as of April 3rd. My build servers last build was on the 2nd. Then I spent several hours dealing with a problem that did not exist a few week back or on April 2nd. Since more Ruby packages are getting ruby24. > Its the same hell of keywording. > > Its much easier to *add* new keywords/useflags as repoman can > trivially tell you if you made any mistakes, because repoman can only > see how your package is, and how your dependencies are. There were other things that were better on removal. paludis had some useful features. But that has changed so much last I tried to install it was a mess. Seemed to want to deviate from how portage was setup. More like one or the other not both. Both would require work. I was not very committed just playing and experimenting. Trying to get at the information I recall from using it in the past. It used to give you an idea on deps so if you were to drop something the impact. I maybe wrong, but I recall something along those lines. I imagine it still has that ability. > *removing* useflags/keywords is much messier, because repoman can't > tell you what you broke. No but I believe other tools can. > Not without doing a full tree check, which takes 30 minutes+ on my > hardware. Have you worked with paludis? If you can get that setup it should give you more useful output in less time. Ciaran would know there, and maybe some others. > Hence, that's the sort of problem I'm more inclined to throw grep at > and then run it through an automated test PR to make sure I didn't > break anything if I was really concerned. Check out paludis -- William L. Thomson Jr.
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Mon, 10 Apr 2017 23:56:04 +0200 > > Except that the packages don't get recompiled unless you take manual > action to recompile them. If you fail at this action, you may end up > having broken software because the rebuild has not been complete. Which is the duty of the team, or whom ever is adding the new Java version to tree. Not like this stuff ends up in tree magically. They should be running something to rebuild and reinstall packages. I did that recently but I ran into other issues. You cannot go backwards with Java on Gentoo. If you use 1.9 to compile and then go back to 1.8 you have serious RUNTIME problems. https://wiki.gentoo.org/wiki/Java_Developer_Guide#Bootstrap_class_path For anyone accusing me of making assumptions about other languages they do the same for Java on Gentoo. Very few know that system well. Much less the issues that still exist. The solutions are much more complex than for other languages. To safely build 1.8 java code under say 1.9/9. You need 1.8 rt.jar. Gentoo has no means for this. The solutions are not pretty. > TARGETS *have been added*. This is *the new way*. This *did change*. I > have no clue why you pretend it's some ancient status quo when > the remnants of old code were removed two months ago. Things changed, but users still have TARGET variables to maintain or ignore. Developers still have to add new versions to packages. Touching every ebuild for every new version. No one has said that is not the case yet That is a lot of work. -- William L. Thomson Jr. pgpsfB2vrfG9y.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Tue, 11 Apr 2017 00:44:30 +0300 Mart Raudsepp wrote: > Ühel kenal päeval, E, 10.04.2017 kell 17:33, kirjutas William L. > Thomson Jr.: > > Add a new Java version and recompiling packages with it, will also > > immediately show breakage if any. > > > > If your saying Python code is of higher quality than Java. I would > > digress heavily on that. You have leniency in python not being > > strong typed. Lack of generics and stuff could only mean that could > > be worse. > > Relying on internals to handle data types for you. > > Which is why python modules can't just pretend to work with a newer > python by merely happening to "compile" and install. It is not > strongly typed and it does not involve a AOT phase (pyc is just a > semi-binary representation of the source code really) and issues are > not found unless properly tested at runtime or test suite. Java is strong typed. Lots things in Java have tests. That does not ensure no bugs. Nor does that mean things are the same all the time. Case in point. I have issues that upstream does not. Both on JDK 1.8, and java is java so it should be the same right? Fix for Java 1.8 and Guice 4.1 https://github.com/jclouds/jclouds/pull/1036 Its likely a matter of dependencies. Their guice may not have been compiled as Java 1.8. Thus I may be triggering something they are not. It is not easily figured out if the fix is needed or not. Though in my case without it fails. In their case without it does not fail > > Regardless of new eclass, the TARGETS remain. Things did not change > > from a user perspective. Recently packaging some ebuilds, the > > COMPAT/VERSION does not seem to have changed. Despite what ever > > changes to the eclass. > > Users don't get unexpected failures, as things that are claimed to > work with a given python version, probably actually do so. This really is no different. We are not talking about a new python version going straight to stable. Any issues would be in ~arch, and short of speculation. It may not effect as many packages as people think. If python is really breaking that much between even 3.x releases. Then just shows that much more how it sucks. Though I think breakage could be looked out for. Code modified. Fixes sent upstream. etc. When I see lots of versions. Seems more like people are maintaining vs fixing/patching the code and sending stuff upstream. I would have more but not everything do I take upstream. Depends on if I feel they will be receptive or a waste of my time. Thankfully most in Java are forward looking. Most active projects already support 1.8 and have things being tested under Java 9. -- William L. Thomson Jr. pgpgQ2t4gDxbk.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Mon, 10 Apr 2017 18:35:13 -0400 "William L. Thomson Jr." wrote: > Have you worked with paludis? If you can get that setup it should give > you more useful output in less time. Ciaran would know there, and maybe > some others. > > > Hence, that's the sort of problem I'm more inclined to throw grep at > > and then run it through an automated test PR to make sure I didn't > > break anything if I was really concerned. > > Check out paludis I've run a box with it since 2008. More pain, less help. Have to write my own tools just for keeping things up-to-date. It was good once, but it and the portage tree no longer seem good friends. ( I still use it mind, but this is because I hate myself ) pgpqHhB9QNOLo.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Tue, 11 Apr 2017 10:56:51 +1200 Kent Fredric wrote: > On Mon, 10 Apr 2017 18:35:13 -0400 > > I've run a box with it since 2008. More pain, less help. Have to write > my own tools just for keeping things up-to-date. This was not an update thing. This was some feature you could run it and it produced like a chart, depgraph sort of thing or something. Its been a long time, since I used it. > It was good once, but it and the portage tree no longer seem good > friends. That was my experience. Seemed it was one or the other not both. At least not easily. -- William L. Thomson Jr. pgpwtoB3xZtqo.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On April 10, 2017 11:12:10 AM PDT, "William L. Thomson Jr." wrote: >Are you running stable? There are other versions in tree. 3.4, 3.5, >3.6. If you were running unstable, you would have 4 pythons, including >2.7. That you only have 2 seems like you are running stable. Yep. Absolutely. I bring in ~ versions of packages when I have no choice. >If your writing new python code against say 3.4 and not 3.6. Not sure >about that... Seems like it would keep things bound to older versions >and never let things move forward. Not true. I will certainly move forward when a newer version becomes stable. >Usually when writing new code, you use the latest version of stuff. Not >always but usually best. If anything make code support older while >targeting newer. No, not how I develop. I always start by determining my target audience and then develop using a feature set that allows my target audience to use the code as easily as is practical. I wouldn’t use a syscall introduced in kernel 4.9 if I could avoid it, even if it made my code simpler, because most of my colleagues run Ubuntu LTS, they are part of my target audience, and it wouldn’t be available there. To me, responsible development practices mean NOT forcing my target audience to do a manual kernel build. Eventually the syscall will be “generally available” to my target audience, at which point I may go back and change the code. I wouldn’t build conditional branches that do it either way if I could possibly avoid it, either, because then I would have to do all the work of doing it the old way plus more, and it would also be more code which means more maintenance. >> Eventually 3.5 will get >> installed and 3.4 will go away. Just like every other package. I >> won’t need to do any config file editing, just a revdep-rebuild run >> perhaps. So regardless of the situation for maintainers, as a user, I >> don’t see this pain. > >Because you are not setting or dealing with the targets. You went with >the mindless approach. Like doing a wildcard on USE flags. Yes, exactly. I don’t want to manually choose what version of Python I have installed, even though I sometimes do Python development. Just like I do a lot of C/C++ development, but I don’t want to manually choose which version of glibc I have installed. Or libfoo, for some foo. That’s what a depgraph is for, and if I do need some specific thing, then I will ask for it, but not before. >Your enabling support for all versions across the board for anything >that supports it. That is quite a different experience if you go trying >to use a specific one. I’m not trying to invalidate the pain that some people experience, just pointing out that I think it may be inaccurate to call that the “ordinary user” use case. -- Christopher Head signature.asc Description: PGP signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Thu, 20 Apr 2017 10:49:04 -0700 Christopher Head wrote: > > >If your writing new python code against say 3.4 and not 3.6. Not sure > >about that... Seems like it would keep things bound to older versions > >and never let things move forward. > > Not true. I will certainly move forward when a newer version becomes > stable. Stable according to whom? Gentoo or Python? Take Ansible as an example. Any contributions must work in both 2.7 and 3.x. While I think they require 2.7. Any modifications must also be good for 3.x. Its forward looking. https://docs.ansible.com/ansible/dev_guide/developing_python3.html > >Usually when writing new code, you use the latest version of stuff. > >Not always but usually best. If anything make code support older > >while targeting newer. > > No, not how I develop. It is how other projects, some rather large like Ansible are addressing the issue. >I always start by determining my target > audience and then develop using a feature set that allows my target > audience to use the code as easily as is practical. I wouldn’t use a > syscall introduced in kernel 4.9 if I could avoid it, even if it made > my code simpler, because most of my colleagues run Ubuntu LTS, they > are part of my target audience, and it wouldn’t be available there. > To me, responsible development practices mean NOT forcing my target > audience to do a manual kernel build. Eventually the syscall will be > “generally available” to my target audience, at which point I may go > back and change the code. Interesting you mention Ubuntu LTS, as that is specifically mentioned on the Ansible python FAQ. > I wouldn’t build conditional branches that do it either way if I > could possibly avoid it, either, because then I would have to do all > the work of doing it the old way plus more, and it would also be more > code which means more maintenance. That is a result of the language of choice. Why I do not bother doing anything myself with Python. I am forced to with Ansible till an alternative in another language comes about. > >Because you are not setting or dealing with the targets. You went > >with the mindless approach. Like doing a wildcard on USE flags. > > Yes, exactly. I don’t want to manually choose what version of Python > I have installed, even though I sometimes do Python development. That addresses 2 of the issues right there. You do not care what version of Python you have. Most any systems administrator does care about what version of things are installed. Not to mention what is installed. Like hardended binary systems, tend to not have gcc, etc. I am doing some python dev for Ansible. But also having to package some python stuff, and do not like it from either user, developer, nor packager perspectives. >Just > like I do a lot of C/C++ development, but I don’t want to manually > choose which version of glibc I have installed. Or libfoo, for some > foo. That’s what a depgraph is for, and if I do need some specific > thing, then I will ask for it, but not before. If you were targeting embedded and working with others like ulibc, etc then you may care. But that is interesting for anyone to be coding not caring about versions etc. I do Java dev mostly and some C. I do very much care about versions of stuff when coding. In C I have to write ifdef, etc for such. In java slotting and making sure using the right version. Only Go language seems to not care about versions, just pulling live from major versioned branches. Most everything else is versioned for a reason. > >Your enabling support for all versions across the board for anything > >that supports it. That is quite a different experience if you go > >trying to use a specific one. > > I’m not trying to invalidate the pain that some people experience, > just pointing out that I think it may be inaccurate to call that the > “ordinary user” use case. That you are on -dev mailing list. Not sure that would put you in the normal use category. Though there are normal users who run ~arch. They would experience such issues. It is different on stable, as I do not believe there is as many. -- William L. Thomson Jr. pgp6yIljSIMxC.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
On Thu, 20 Apr 2017 10:49:04 -0700 Christopher Head wrote: > >Usually when writing new code, you use the latest version of stuff. Not > >always but usually best. If anything make code support older while > >targeting newer. > > No, not how I develop. I always start by determining my target audience and > then develop using a feature set that allows my target audience to use the > code as easily as is practical. I wouldn’t use a syscall introduced in kernel > 4.9 if I could avoid it, even if it made my code simpler, because most of my > colleagues run Ubuntu LTS, they are part of my target audience, and it > wouldn’t be available there. To me, responsible development practices mean > NOT forcing my target audience to do a manual kernel build. Eventually the > syscall will be “generally available” to my target audience, at which point I > may go back and change the code. Slightly seems like minor misinterpretation (possibly) I think the development maxim is not that you "use newer stuff and push it downstream" but "make sure you yourself are using newer things so that you're aware of changes that are occurring in the pipeline and you have accommodated for them before it becomes a critical to do so". For comparison, I'll be using the latest Perl version possible, and I'll be testing everything possible on the latest version, and fixing every problem that the new versions introduce, ... while simultaneously also not using *any* features consciously that have been introduced since 5.8 This is just the principle of "make your own code the most accommodating", accommodate maximally for both old and new versions of things outside your control. Or to re-phrase this yet another way: - I run the latest everything so that when you do, you won't have problems - But I don't expect you to run the latest everything, though you might some day. pgp8ZFFJlbX18.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Reverse use of Python/Ruby versions
09.04.2017 19:15, William L. Thomson Jr. пишет: > Not sure if this is practical, it may be less work if the use of > Python and Ruby versions ( maybe others ) is reversed. Rather than > adding all the versions that the ebuild supports. What if it only > included versions it did not support? It was like that before we switch to python-r1 and it was major PITA. Please, do not do this! -- Best regards, Sergey Popov Gentoo developer Gentoo Desktop Effects project lead Gentoo Quality Assurance project lead signature.asc Description: OpenPGP digital signature