Re: CVS commit: src/external/gpl3
Date:Thu, 26 Mar 2020 23:22:57 + From:Andrew Doran Message-ID: <20200326232257.gf27...@homeworld.netbsd.org> | > Modern CPUs like Ryzen Threadripper 3990X can execute that extra amount | > of instructions (around 600) in a single clock cycle. | | NetBSD-current would probably take longer to build on that AMD chip, than | Research Unix took to build on a PDP-11/70 in the late 1970s. It still | counts. Not only tat, but Kamil's measurements only count the kernel cost, not the cost for the shell adding the var to the env on every exec, or the additional cost on env var lookups of having to scan one extra var that isn't the one being sought (50% of the time on average - the other lookups find the var before the extra one) - and of course, all the lookups for vars like LD_LIBRARY_PATH that have to scan everything, as those are almost never present. But discussion isn't worth the time it is taking. kre
Re: CVS commit: src/lib/libc/string
On Thu, Mar 26, 2020 at 10:54:21AM +0700, Robert Elz wrote: > Date:Wed, 25 Mar 2020 20:51:25 + > From:David Holland > Message-ID: <20200325205125.ga11...@netbsd.org> > > | I don't agree -- because applications shouldn't attempt to modify the > | result, it should be const. > > The only reason apps shouldn't modify the string is in case of porting > the app to an (well, some) ancient implementation. Because of the NLS > requirements, the message these days (any modern implementation) must be > read from some external file - which means the storage for it must be > writable. Actually, the only reason why we really need writable space is the unknown error case. NLS could in principle be using mmaped data as well, modulo not being able to unmap it again. Joerg
Re: CVS commit: src/external/gpl3
On Thu, Mar 26, 2020 at 06:07:54PM +0100, Kamil Rytarowski wrote: > On 26.03.2020 17:49, Robert Elz wrote: > > Date:Thu, 26 Mar 2020 16:28:18 +0100 > > From:Kamil Rytarowski > > Message-ID: <84460ebb-b4bf-f3ee-e51b-e27d0b6e2...@gmx.com> > > > > > > | That is negligible cost of getting TMPDIR propagated to most programs. > > > > Sure, it isn't much, for one program, but when you're running tens of > > thousands, it all adds up. > > > > It's still negligible for hundreds of millions calls. > > Modern CPUs like Ryzen Threadripper 3990X can execute that extra amount > of instructions (around 600) in a single clock cycle. NetBSD-current would probably take longer to build on that AMD chip, than Research Unix took to build on a PDP-11/70 in the late 1970s. It still counts. https://www.bell-labs.com/usr/dmr/www/retro.pdf Andrew
Re: CVS commit: src/external/gpl3
Date:Thu, 26 Mar 2020 18:07:54 +0100 From:Kamil Rytarowski Message-ID: <723e979c-cb16-1a39-a51e-9d756f372...@gmx.com> | But nonetheless, I'm trying to fix it un GCC without TMPDIR. Not without, just without requiring. Something like tempnam(3) works - if TMPDIR is set, use it, if not, use /tmp. kre
Re: CVS commit: src
Le 26/03/2020 à 15:32, Jonathan A. Kollasch a écrit : > On Tue, Aug 14, 2018 at 02:49:14PM +, Maxime Villard wrote: >> Module Name: src >> Committed By:maxv >> Date:Tue Aug 14 14:49:14 UTC 2018 >> Log Message: >> Retire EtherIP, we have L2TP instead. > > Why was this not discussed? It was discussed. https://mail-index.netbsd.org/tech-net/2018/08/07/msg006998.html A bit of a blurted-out email using the bad word "sh*tty", which was ill-placed, but other than that I was not technically incorrect. > L2TP does not implement EtherIP > functionality, and is therefore NOT a replacement. Not sure what you mean? We have L2TPv3, and it does come as a replacement of EtherIP. Maxime
Re: CVS commit: src/external/gpl3
On 26.03.2020 17:49, Robert Elz wrote: > Date:Thu, 26 Mar 2020 16:28:18 +0100 > From:Kamil Rytarowski > Message-ID: <84460ebb-b4bf-f3ee-e51b-e27d0b6e2...@gmx.com> > > > | That is negligible cost of getting TMPDIR propagated to most programs. > > Sure, it isn't much, for one program, but when you're running tens of > thousands, it all adds up. > It's still negligible for hundreds of millions calls. Modern CPUs like Ryzen Threadripper 3990X can execute that extra amount of instructions (around 600) in a single clock cycle. If we want to optimize exec(), there would be better places to do so. But nonetheless, I'm trying to fix it un GCC without TMPDIR.
Re: CVS commit: src/external/gpl3
Date:Thu, 26 Mar 2020 16:28:18 +0100 From:Kamil Rytarowski Message-ID: <84460ebb-b4bf-f3ee-e51b-e27d0b6e2...@gmx.com> | That is negligible cost of getting TMPDIR propagated to most programs. Sure, it isn't much, for one program, but when you're running tens of thousands, it all adds up. And, for most users, there should be no point - only those who need to alter the default need to set it. There has to be some fallback for programs run like env -i prog for TMPDIR (and everything else, like PATH, ...) if those defaults are suitable for most users, then less leeds to be added to the environment. kre
Re: CVS commit: src/external/gpl3
On 26.03.2020 17:03, Kamil Rytarowski wrote: > On 26.03.2020 16:17, Greg Troxel wrote: >> I don't think we can go from "upstream is unwilling to do X" >> to "impose pain on NetBSD" by making "divergence bad" be the >> highest-weighted concern. We have to say "what is the minimally >> painful way to cope". > > I agree here that we want to get /tmp by default, but in real life we > lost these local modifications anyway. None of the pkgsrc packages ships > with this and upstream GCC knows better.. so I keep trying to find a > creative way to get it done right. > > Patching every libiberty copy is almost not doable (we don't patch any > user of it in pkgsrc). > > If I will fail to convince GCC to add a way to tune this tmpdir, we will > live with these libiberty patches in src. > > If GCC will be unconvinced, I don't plan to remove this local modification. > > On 26.03.2020 16:56, Taylor R Campbell wrote: >> We don't need to set TMPDIR=/tmp anywhere because that's already what >> correct programs use by default (for example, via _PATH_TMP). > > Good idea. I will try to reuse _PATH_TMP and submit upstream. > OK, I have got another idea how to address it. Thank you for the feedback.
Re: CVS commit: src/external/gpl3
On 26.03.2020 16:17, Greg Troxel wrote: > I don't think we can go from "upstream is unwilling to do X" > to "impose pain on NetBSD" by making "divergence bad" be the > highest-weighted concern. We have to say "what is the minimally > painful way to cope". I agree here that we want to get /tmp by default, but in real life we lost these local modifications anyway. None of the pkgsrc packages ships with this and upstream GCC knows better.. so I keep trying to find a creative way to get it done right. Patching every libiberty copy is almost not doable (we don't patch any user of it in pkgsrc). If I will fail to convince GCC to add a way to tune this tmpdir, we will live with these libiberty patches in src. If GCC will be unconvinced, I don't plan to remove this local modification. On 26.03.2020 16:56, Taylor R Campbell wrote: > We don't need to set TMPDIR=/tmp anywhere because that's already what > correct programs use by default (for example, via _PATH_TMP). Good idea. I will try to reuse _PATH_TMP and submit upstream. signature.asc Description: OpenPGP digital signature
Re: CVS commit: src/external/gpl3
> Date: Thu, 26 Mar 2020 14:57:40 +0100 > From: Kamil Rytarowski > > Maybe we could specify TMPDIR somewhere in /etc and point to /tmp? > > The build of tools could be fixed independently. It is wrong for gcc to abuse /var/tmp for files that aren't meaningfully persistent, just like it would be wrong to abuse /usr/bin for configuration files that the operator is expected to edit. If upstream inexplicably insisted on abusing /usr/bin for a configuration file because it was once on a larger partition than /etc, we would patch gcc to fix that bug too. If gcc doesn't respect TMPDIR, that's a separate bug that should be fixed as well. We don't need to set TMPDIR=/tmp anywhere because that's already what correct programs use by default (for example, via _PATH_TMP).
Re: CVS commit: src/external/gpl3
On 26.03.2020 16:02, Robert Elz wrote: > EVery extra env var has a cost in every exec of absolutely everything. As this is true.. but as I have benchmarked it. For true(1) on amd64, we execute 0.05% more instructions for extra TMPDIR environment variable. $ ./singlestepper true Total count: 1286742 $ TMPDIR=/tmp ./singlestepper true Total count: 1287399 That is negligible cost of getting TMPDIR propagated to most programs. signature.asc Description: OpenPGP digital signature
Re: CVS commit: src/external/gpl3
Generally speaking divergence with upstream is a problem and we lost these changes anyway in pkgsrc and every standalone GNU toolchain copy. Agreed that divergence is a problem, but so is having bugs. It's a question of the right balance in each case. Finding a creative way to make this at least tunable is challenging with the GCC people. I think this is the real issue. As I understand it, upstream is making a choice that is inconsistent with longstanding norms and against the consensus of an OS community. So if the best that can happen is that we have to patch, that's how it is. It would be really nice if there were a --default-tmpdir= configure argument, but it sounds like they are unwilling to do that. Basically, I don't think we can go from "upstream is unwilling to do X" to "impose pain on NetBSD" by making "divergence bad" be the highest-weighted concern. We have to say "what is the minimally painful way to cope".
Re: CVS commit: src/external/gpl3
On 26.03.2020 15:52, Greg Troxel wrote: > What is the real problem here? I think it's great that NetBSD-specific > fixes are being upstreamed, and that we are reducing gratuitous changes > from upstream. But upstream's position that the default tmp should be > /var/tmp is at odds with our and traditional norms, and really seems to > just not make sense. (Perhaps it does make sense on typical Linux, and > perhaps upstream should have OS-specific tmp defaults.) Adding > complexity to NetBSD config files and users for the sake of reducing a > diff seems like a bad tradeoff. Generally speaking divergence with upstream is a problem and we lost these changes anyway in pkgsrc and every standalone GNU toolchain copy. Finding a creative way to make this at least tunable is challenging with the GCC people. signature.asc Description: OpenPGP digital signature
Re: CVS commit: src/external/gpl3
Greg Troxel writes: > It seem really obvious to me, and obvious that it is netbsd consensus, > that a tool that needs tmp (without needing persistence) should default > to /tmp. So I think it's unreasonable to follow upstream here, and > unreasonable to ask everybody to either inherit some system profile or > adjust their own to make tmp files be put in /tmp. > > Certainly if something in the tools build doesn't honor TMPDIR, that's a > bug to be fixed -- but that seems uncontroversial. > > > People who really want gcc tmp files to be put in /var/tmp can set > TMPDIR. As I understand it people that actually want that are a > minority, and we haven't had a single person express that they want it > during this discussion. > > What is the real problem here? I think it's great that NetBSD-specific > fixes are being upstreamed, and that we are reducing gratuitous changes > from upstream. But upstream's position that the default tmp should be > /var/tmp is at odds with our and traditional norms, and really seems to > just not make sense. (Perhaps it does make sense on typical Linux, and > perhaps upstream should have OS-specific tmp defaults.) Adding > complexity to NetBSD config files and users for the sake of reducing a > diff seems like a bad tradeoff. If you were only talking about fixing tools build, and have withdrawn the notion of removing our patch to fix the default, I apologize for writing this way. My note is based on a perception that you are still pushing to remove our patch to fix the tmp behavior.
Re: CVS commit: src/external/gpl3
Date:Thu, 26 Mar 2020 15:11:46 +0100 From:Kamil Rytarowski Message-ID: <26eaf84f-616a-d48b-9a2f-7dd74cd69...@gmx.com> | Right. Addressing tools build independently (honoring TMPDIR?) This we should do. | defining TMPDIR in /etc shell profile for common use inside the | distribution. but this we should not. The default (when TMPDIR is not specified) should be what we would put there as the default - env vars to override defaults should only be used by users who want something different, not as the standard way of doing things. EVery extra env var has a cost in every exec of absolutely everything. kre
Re: CVS commit: src/external/gpl3
Kamil Rytarowski writes: > On 26.03.2020 15:01, Martin Husemann wrote: >> On Thu, Mar 26, 2020 at 02:57:40PM +0100, Kamil Rytarowski wrote: >>> The build of tools could be fixed independently. >> The problem is that we build the whole system with the tools gcc, and that >> gcc misbehaves (or so I understood). >> >> So pointing TMPDIR anywhere does not help. >> > > Right. Addressing tools build independently (honoring TMPDIR?) and > defining TMPDIR in /etc shell profile for common use inside the > distribution. It seem really obvious to me, and obvious that it is netbsd consensus, that a tool that needs tmp (without needing persistence) should default to /tmp. So I think it's unreasonable to follow upstream here, and unreasonable to ask everybody to either inherit some system profile or adjust their own to make tmp files be put in /tmp. Certainly if something in the tools build doesn't honor TMPDIR, that's a bug to be fixed -- but that seems uncontroversial. People who really want gcc tmp files to be put in /var/tmp can set TMPDIR. As I understand it people that actually want that are a minority, and we haven't had a single person express that they want it during this discussion. What is the real problem here? I think it's great that NetBSD-specific fixes are being upstreamed, and that we are reducing gratuitous changes from upstream. But upstream's position that the default tmp should be /var/tmp is at odds with our and traditional norms, and really seems to just not make sense. (Perhaps it does make sense on typical Linux, and perhaps upstream should have OS-specific tmp defaults.) Adding complexity to NetBSD config files and users for the sake of reducing a diff seems like a bad tradeoff.
Re: CVS commit: src
On Tue, Aug 14, 2018 at 02:49:14PM +, Maxime Villard wrote: > Module Name: src > Committed By: maxv > Date: Tue Aug 14 14:49:14 UTC 2018 > Log Message: > Retire EtherIP, we have L2TP instead. Why was this not discussed? L2TP does not implement EtherIP functionality, and is therefore NOT a replacement. Jonathan
Re: CVS commit: src/external/gpl3
On 26.03.2020 15:01, Martin Husemann wrote: > On Thu, Mar 26, 2020 at 02:57:40PM +0100, Kamil Rytarowski wrote: >> The build of tools could be fixed independently. > The problem is that we build the whole system with the tools gcc, and that > gcc misbehaves (or so I understood). > > So pointing TMPDIR anywhere does not help. > Right. Addressing tools build independently (honoring TMPDIR?) and defining TMPDIR in /etc shell profile for common use inside the distribution. > Martin > signature.asc Description: OpenPGP digital signature
Re: CVS commit: src/external/gpl3
On Thu, Mar 26, 2020 at 02:57:40PM +0100, Kamil Rytarowski wrote: > The build of tools could be fixed independently. The problem is that we build the whole system with the tools gcc, and that gcc misbehaves (or so I understood). So pointing TMPDIR anywhere does not help. Martin
Re: CVS commit: src/external/gpl3
On 26.03.2020 09:48, Martin Husemann wrote: > On Thu, Mar 26, 2020 at 12:51:24AM +, Taylor R Campbell wrote: >> We should keep the change. There is no semantic justification for >> putting build-time temporary files in the directory for temporary >> files that are meant to persist across reboot. These temporary files >> _cannot_ be used if interrupted -- let alone by a reboot. > > The original bug is somewhere else though: it seems (for reasons not well > understood) that the NetBSD tools build of gcc (contrary to the installed > one and all native build gcc) does *NOT* honour $TMPDIR in env. (Somthing > broken in the ifdef maze in libberty?) > > We should fix that. My local build scripts (and the build cluster) properly > sets TMPDIR, so if that worked, the default would not matter. > Maybe we could specify TMPDIR somewhere in /etc and point to /tmp? The build of tools could be fixed independently. > Martin > signature.asc Description: OpenPGP digital signature
Re: CVS commit: src/external/gpl3
On Thu, Mar 26, 2020 at 12:51:24AM +, Taylor R Campbell wrote: > We should keep the change. There is no semantic justification for > putting build-time temporary files in the directory for temporary > files that are meant to persist across reboot. These temporary files > _cannot_ be used if interrupted -- let alone by a reboot. The original bug is somewhere else though: it seems (for reasons not well understood) that the NetBSD tools build of gcc (contrary to the installed one and all native build gcc) does *NOT* honour $TMPDIR in env. (Somthing broken in the ifdef maze in libberty?) We should fix that. My local build scripts (and the build cluster) properly sets TMPDIR, so if that worked, the default would not matter. Martin
