daily CVS update output
Updating src tree: P src/distrib/sets/lists/tests/mi P src/doc/3RDPARTY P src/games/adventure/init.c P src/games/adventure/io.c P src/games/adventure/main.c P src/games/adventure/setup.c P src/games/adventure/subr.c P src/games/adventure/vocab.c P src/games/adventure/wizard.c P src/games/atc/extern.h P src/games/atc/graphics.c P src/games/atc/input.c P src/games/atc/list.c P src/games/atc/log.c P src/games/atc/main.c P src/games/atc/update.c P src/games/backgammon/backgammon/main.c P src/games/backgammon/common_source/fancy.c P src/games/backgammon/teachgammon/tutor.c P src/games/banner/banner.c P src/games/battlestar/command1.c P src/games/battlestar/command2.c P src/games/battlestar/command3.c P src/games/battlestar/command4.c P src/games/battlestar/command5.c P src/games/battlestar/command6.c P src/games/battlestar/command7.c P src/games/battlestar/cypher.c P src/games/battlestar/fly.c P src/games/battlestar/getcom.c P src/games/battlestar/parse.c P src/games/battlestar/room.c P src/games/boggle/boggle/extern.h P src/games/boggle/boggle/mach.c P src/games/boggle/boggle/prtable.c P src/games/boggle/boggle/word.c P src/games/boggle/mkdict/mkdict.c P src/games/boggle/mkindex/mkindex.c P src/games/canfield/canfield/canfield.c P src/games/countmail/countmail P src/games/cribbage/score.c P src/games/dab/random.h P src/games/dm/dm.c P src/games/fish/fish.c P src/games/fortune/strfile/strfile.c P src/games/gomoku/bdisp.c P src/games/gomoku/main.c P src/games/gomoku/stoc.c P src/games/hack/config.h P src/games/hack/extern.h P src/games/hack/hack.c P src/games/hack/hack.terminfo.c P src/games/hack/hack.zap.c P src/games/hals_end/hals_end.c P src/games/hunt/hunt/hunt_private.h P src/games/hunt/huntd/answer.c P src/games/hunt/huntd/draw.c P src/games/hunt/huntd/driver.c P src/games/hunt/huntd/execute.c P src/games/hunt/huntd/expl.c P src/games/hunt/huntd/extern.c P src/games/hunt/huntd/hunt.h P src/games/hunt/huntd/makemaze.c P src/games/hunt/huntd/shots.c P src/games/hunt/huntd/terminal.c P src/games/hunt/include/hunt_common.h P src/games/hunt/include/pathnames.h P src/games/larn/action.c P src/games/larn/global.c P src/games/larn/io.c P src/games/larn/main.c P src/games/larn/monster.c P src/games/larn/moreobj.c P src/games/larn/scores.c P src/games/larn/store.c P src/games/mille/extern.c P src/games/mille/mille.h P src/games/mille/misc.c P src/games/morse/morse.c P src/games/number/number.c P src/games/phantasia/fight.c P src/games/phantasia/misc.c P src/games/phantasia/setup.c P src/games/pom/pom.c P src/games/primes/pattern.c P src/games/random/random.c P src/games/rogue/message.c P src/games/rogue/save.c P src/games/rogue/use.c P src/games/sail/array.h P src/games/testpat/testpat.c P src/games/tetris/scores.c P src/games/tetris/screen.c P src/games/warp/EXTERN.h P src/games/warp/INTERN.h P src/games/warp/bang.c P src/games/warp/bang.h P src/games/warp/init.c P src/games/warp/init.h P src/games/warp/intrp.c P src/games/warp/intrp.h P src/games/warp/move.c P src/games/warp/move.h P src/games/warp/object.c P src/games/warp/object.h P src/games/warp/play.c P src/games/warp/play.h P src/games/warp/score.c P src/games/warp/score.h P src/games/warp/sig.c P src/games/warp/sig.h P src/games/warp/sm.c P src/games/warp/term.c P src/games/warp/term.h P src/games/warp/them.c P src/games/warp/them.h P src/games/warp/us.c P src/games/warp/us.h P src/games/warp/util.c P src/games/warp/util.h P src/games/warp/version.c P src/games/warp/version.h P src/games/warp/warp.c P src/games/warp/warp.h P src/games/warp/weapon.c P src/games/warp/weapon.h P src/games/wump/wump.c P src/sys/conf/lint.mk P src/sys/dev/audio/audio.c P src/sys/kern/kern_event.c P src/sys/kern/kern_exec.c U src/sys/stand/efiboot/bootriscv64/Makefile U src/sys/stand/efiboot/bootriscv64/efibootriscv64.c P src/sys/sys/eventvar.h P src/tests/lib/libc/gen/posix_spawn/h_spawn.c P src/tests/lib/libc/gen/posix_spawn/t_spawnattr.c P src/tests/usr.bin/xlint/lint1/Makefile U src/tests/usr.bin/xlint/lint1/gcc_attribute_aligned.c U src/tests/usr.bin/xlint/lint1/gcc_attribute_aligned.exp U src/tests/usr.bin/xlint/lint1/gcc_bit_field_types.c U src/tests/usr.bin/xlint/lint1/gcc_bit_field_types.exp P src/tests/usr.bin/xlint/lint1/msg_035.c U src/tests/usr.bin/xlint/lint1/msg_035.exp P src/tests/usr.bin/xlint/lint1/t_integration.sh P src/usr.bin/xlint/lint1/cgram.y P src/usr.bin/xlint/lint1/decl.c P src/usr.bin/xlint/xlint/lint.1 P src/usr.bin/xlint/xlint/xlint.c Updating xsrc tree: Killing core files: Updating release-8 src tree (netbsd-8): Updating release-8 xsrc tree (netbsd-8): Updating release-9 src tree (netbsd-9): U doc/CHANGES-9.2 P external/cddl/osnet/dist/cmd/zfs/zfs.8 P external/cddl/osnet/dist/cmd/zpool/zpool.8 Updating release-9 xsrc tree (netbsd-9): Updating file list: -rw-rw-r-- 1 srcmastr netbsd 39538922 May 3 03:10 ls-lRA.gz
Re: Problem reports for version control systems
On 2021-05-02 18:21, Michael van Elst wrote: b...@update.uu.se (Johnny Billquist) writes: And as a "fun" fact. On my 4000/90, it takes about 3h after I start a cvs update until I actually start having any network traffic... A SCSI SSD could help. :) Definitely. Because the disk is 100% busy all the time. Not a very modern disk either. RZ29 if I remember right. SSD could make a big difference. And that kind of capacity isn't that much today. 4G or so... Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
Re: Problem reports for version control systems
> I suspect what is commonly the problem here is related to the fact > that cvs has such a phase at the beginning where it is scanning > through the file system, which can take quite a while. Some NAT > devices along the path sometimes have timeouts on existing connections > that if no traffic is happening for a while, they are dropped, even > though there hasn't been any FINs on the connection. > So a connection that just don't have any traffic for a while are hit > by this, which is exactly the pattern you have with cvs. This is the reason some of my ~/.ssh/config files has host * ServerAliveInterval 240 in them. :) Regards, - Håvard
Re: Problem reports for version control systems
b...@update.uu.se (Johnny Billquist) writes: >And as a "fun" fact. On my 4000/90, it takes about 3h after I start a >cvs update until I actually start having any network traffic... A SCSI SSD could help. :)
Re: Problem reports for version control systems
On 2021-05-02 16:32, Anders Magnusson wrote: Den 2021-05-02 kl. 15:57, skrev Johnny Billquist: On 2021-05-02 13:51, Anders Magnusson wrote: Den 2021-05-02 kl. 13:44, skrev Johnny Billquist: I suspect what is commonly the problem here is related to the fact that cvs has such a phase at the beginning where it is scanning through the file system, which can take quite a while. Some NAT devices along the path sometimes have timeouts on existing connections that if no traffic is happening for a while, they are dropped, even though there hasn't been any FINs on the connection. So a connection that just don't have any traffic for a while are hit by this, which is exactly the pattern you have with cvs. I've seen the same effect on a simple telnet session, where ssh survives fine. And there it's just that when the connection is idle, telnet is not creating any traffic at all, while ssh do generate a bit of traffic even if there is no activity. So one obvious solution is to use something like ssh as a carries for the cvs traffic, if possible, or else see if some kind of keepalives can be enabled on a connection, to defeat NAT and similar devices which aggressively drop connections on which there is no traffic for a while. (Or, of course, if there is a NAT you have control over, you might be able to change how it behaves...) This is quite common, yes. I ususlly add ssh keepalive to ssh_config for all hosts to avoid this problem (which may occur, as written, when doing cvs update). And as a "fun" fact. On my 4000/90, it takes about 3h after I start a cvs update until I actually start having any network traffic... In total it takes something like 8h to do a cvs update on /usr/src. (I guess I'm a bit masochistic is still insisting on trying to do things on my VAXen...) Have you tried it on the 8650? :-) Last time I did manage, I think it was close to a day. That with RA73 drives. But you know there's been problems with having two unibuses lately, and so on. So it's been quite a while since I actually managed to much of anything natively on that machine. :-( Another reason I'd like to get gcc working native again. Also trying to get a chance to fix things on the 8650. Not sure how much longer we will be able to keep that machine around ready to run. We might need to shrink our locales soon... Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
Re: Problem reports for version control systems
Robert Swindells writes: > Lloyd Parkes wrote: >>The network is a 1Gb/s LAN through to a smaller NetBSD router running >>NPF with MSS clamping enabled so that I can get Netflix. My ISP does not >>use CGN for my IPv4 connection. My IPv6 connection is tunnelled through >>to Hurricane Electric in Sydney, Australia. > > Have you tried disabling IPv6 or explicitly connecting using IPv4 ? > > I don't see any problems using IPv6 through NPF to update cvs but I have > native IPv6 and can use a 1500 byte MTU. I'm also using cvs.n.o instead > of anoncvs.n.o but they have adjacent IPv4 addresses. > > I'm guessing that your IPv6 tunnel has a lower MTU than your IPv4 > connection to your ISP. I update over HE tunnels all the time with no issues (cvs, not anoncvs). IPv6 tends to 1280 MTU; my gif for the tunnel is that, and wthat has all apparently worked fine. So always good to look with tcpdump, but I suspect tunenl MTU is not really an issue. There are sometimes problems with reachability via v6, and sometimes the speeds are lower. More recently, I had one case of v4 reachability failure while v6 worked, and have sometimes seen lower ping times on v6. signature.asc Description: PGP signature
Re: Problem reports for version control systems
Den 2021-05-02 kl. 15:57, skrev Johnny Billquist: On 2021-05-02 13:51, Anders Magnusson wrote: Den 2021-05-02 kl. 13:44, skrev Johnny Billquist: I suspect what is commonly the problem here is related to the fact that cvs has such a phase at the beginning where it is scanning through the file system, which can take quite a while. Some NAT devices along the path sometimes have timeouts on existing connections that if no traffic is happening for a while, they are dropped, even though there hasn't been any FINs on the connection. So a connection that just don't have any traffic for a while are hit by this, which is exactly the pattern you have with cvs. I've seen the same effect on a simple telnet session, where ssh survives fine. And there it's just that when the connection is idle, telnet is not creating any traffic at all, while ssh do generate a bit of traffic even if there is no activity. So one obvious solution is to use something like ssh as a carries for the cvs traffic, if possible, or else see if some kind of keepalives can be enabled on a connection, to defeat NAT and similar devices which aggressively drop connections on which there is no traffic for a while. (Or, of course, if there is a NAT you have control over, you might be able to change how it behaves...) This is quite common, yes. I ususlly add ssh keepalive to ssh_config for all hosts to avoid this problem (which may occur, as written, when doing cvs update). And as a "fun" fact. On my 4000/90, it takes about 3h after I start a cvs update until I actually start having any network traffic... In total it takes something like 8h to do a cvs update on /usr/src. (I guess I'm a bit masochistic is still insisting on trying to do things on my VAXen...) Have you tried it on the 8650? :-) -- R
Re: Problem reports for version control systems
On 2021-05-02 13:51, Anders Magnusson wrote: Den 2021-05-02 kl. 13:44, skrev Johnny Billquist: I suspect what is commonly the problem here is related to the fact that cvs has such a phase at the beginning where it is scanning through the file system, which can take quite a while. Some NAT devices along the path sometimes have timeouts on existing connections that if no traffic is happening for a while, they are dropped, even though there hasn't been any FINs on the connection. So a connection that just don't have any traffic for a while are hit by this, which is exactly the pattern you have with cvs. I've seen the same effect on a simple telnet session, where ssh survives fine. And there it's just that when the connection is idle, telnet is not creating any traffic at all, while ssh do generate a bit of traffic even if there is no activity. So one obvious solution is to use something like ssh as a carries for the cvs traffic, if possible, or else see if some kind of keepalives can be enabled on a connection, to defeat NAT and similar devices which aggressively drop connections on which there is no traffic for a while. (Or, of course, if there is a NAT you have control over, you might be able to change how it behaves...) This is quite common, yes. I ususlly add ssh keepalive to ssh_config for all hosts to avoid this problem (which may occur, as written, when doing cvs update). And as a "fun" fact. On my 4000/90, it takes about 3h after I start a cvs update until I actually start having any network traffic... In total it takes something like 8h to do a cvs update on /usr/src. (I guess I'm a bit masochistic is still insisting on trying to do things on my VAXen...) Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
Re: Problem reports for version control systems
Lloyd Parkes wrote: >The network is a 1Gb/s LAN through to a smaller NetBSD router running >NPF with MSS clamping enabled so that I can get Netflix. My ISP does not >use CGN for my IPv4 connection. My IPv6 connection is tunnelled through >to Hurricane Electric in Sydney, Australia. Have you tried disabling IPv6 or explicitly connecting using IPv4 ? I don't see any problems using IPv6 through NPF to update cvs but I have native IPv6 and can use a 1500 byte MTU. I'm also using cvs.n.o instead of anoncvs.n.o but they have adjacent IPv4 addresses. I'm guessing that your IPv6 tunnel has a lower MTU than your IPv4 connection to your ISP.
Re: Problem reports for version control systems
Den 2021-05-02 kl. 13:44, skrev Johnny Billquist: I suspect what is commonly the problem here is related to the fact that cvs has such a phase at the beginning where it is scanning through the file system, which can take quite a while. Some NAT devices along the path sometimes have timeouts on existing connections that if no traffic is happening for a while, they are dropped, even though there hasn't been any FINs on the connection. So a connection that just don't have any traffic for a while are hit by this, which is exactly the pattern you have with cvs. I've seen the same effect on a simple telnet session, where ssh survives fine. And there it's just that when the connection is idle, telnet is not creating any traffic at all, while ssh do generate a bit of traffic even if there is no activity. So one obvious solution is to use something like ssh as a carries for the cvs traffic, if possible, or else see if some kind of keepalives can be enabled on a connection, to defeat NAT and similar devices which aggressively drop connections on which there is no traffic for a while. (Or, of course, if there is a NAT you have control over, you might be able to change how it behaves...) This is quite common, yes. I ususlly add ssh keepalive to ssh_config for all hosts to avoid this problem (which may occur, as written, when doing cvs update). -- R
Re: Problem reports for version control systems
On 2021-05-01 23:54, Brett Lymn wrote: On Sat, May 01, 2021 at 12:58:50PM +1200, Lloyd Parkes wrote: Germany is pretty much the opposite of New Zealand. It's close to everywhere, but its last mile access speeds are a bit infamous. Just for you info... there are a few NetBSD developers in .au, my self included. I haven't had any issues with cvs disconnects. Not to deny you have an issue, just letting you know it works ok for people near you. Not anywhere near such a location, but just adding that cvs works fine for me too, but yes, there is a lot of disk activity on the local machine before anything even starts downloading, and a lot of activity at the end where it updates file metadata as well as clean out empty directories (if you added pruning). I'm running some tests on other local clients and against other CVS mirrors in the hope that come up with a better characterisation of the problem than "it doesn't work". If you have the space, a tcpdump from both sides of your firewall may provide a clue. I suspect what is commonly the problem here is related to the fact that cvs has such a phase at the beginning where it is scanning through the file system, which can take quite a while. Some NAT devices along the path sometimes have timeouts on existing connections that if no traffic is happening for a while, they are dropped, even though there hasn't been any FINs on the connection. So a connection that just don't have any traffic for a while are hit by this, which is exactly the pattern you have with cvs. I've seen the same effect on a simple telnet session, where ssh survives fine. And there it's just that when the connection is idle, telnet is not creating any traffic at all, while ssh do generate a bit of traffic even if there is no activity. So one obvious solution is to use something like ssh as a carries for the cvs traffic, if possible, or else see if some kind of keepalives can be enabled on a connection, to defeat NAT and similar devices which aggressively drop connections on which there is no traffic for a while. (Or, of course, if there is a NAT you have control over, you might be able to change how it behaves...) Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
Re: posix_spawn issue?
On Sun, May 02, 2021 at 10:49:20AM +0200, Martin Husemann wrote: > On Sun, May 02, 2021 at 10:41:53AM +0200, Martin Husemann wrote: > > On Sun, May 02, 2021 at 10:36:46AM +0200, Martin Husemann wrote: > > > > > > It does a posix_spawn call for: > > > > > > make -C test-a test-a > > > > > > (I tested with the "make" binary in the pkgobj dir) > > > > No, no, it doesn't - I mixed up argv indices in gdb, it does the correct > > thing - not sure yet why the child has trouble. > > Ok, and *of course* it is the one flag we don't have a test for in our ATF > tests So it was a kernel bug and is now fixed. Pullup requested. Martin
Re: Problem reports for version control systems
Brett Lymn wrote: > Just for you info... there are a few NetBSD developers in .au, my self > included. I haven't > had any issues with cvs disconnects. Not to deny you have an issue, just > letting you know > it works ok for people near you. For what it's worth, my connections to anoncvs from Finland frequently break in the middle of a transfer, though I'm using rsync rather than cvs. I have an admins@ ticket open about this since a couple of years ago (#160795). -- Andreas Gustafsson, g...@gson.org
Re: posix_spawn issue?
On Sun, May 02, 2021 at 10:41:53AM +0200, Martin Husemann wrote: > On Sun, May 02, 2021 at 10:36:46AM +0200, Martin Husemann wrote: > > > > It does a posix_spawn call for: > > > > make -C test-a test-a > > > > (I tested with the "make" binary in the pkgobj dir) > > No, no, it doesn't - I mixed up argv indices in gdb, it does the correct > thing - not sure yet why the child has trouble. Ok, and *of course* it is the one flag we don't have a test for in our ATF tests Martin
Re: posix_spawn issue?
On Sun, May 02, 2021 at 10:36:46AM +0200, Martin Husemann wrote: > > It does a posix_spawn call for: > > make -C test-a test-a > > (I tested with the "make" binary in the pkgobj dir) No, no, it doesn't - I mixed up argv indices in gdb, it does the correct thing - not sure yet why the child has trouble. Martin
Re: posix_spawn issue?
On Sun, May 02, 2021 at 03:33:32PM +0900, Ryo ONODERA wrote:
> Hi,
>
> I have uploaded simple test Makefiles:
> https://www.netbsd.org/~ryoon/gmake-tests-20210502.tar.gz
>
> It is as follows:
>
> $ cat Makefile
> all:
> ${MAKE} -C test-a
>
> clean:
> ${MAKE} -C test-a clean
>
>
It does a posix_spawn call for:
make -C test-a test-a
(I tested with the "make" binary in the pkgobj dir)
and that child gmake says:
make: Entering directory '/tmp/gmake-tests/test-a'
make: *** No rule to make target 'test-a'. Stop.
make: Leaving directory '/tmp/gmake-tests/test-a'
So I'd say the argv[] creation for the posix_spawn call is bogus.
Martin
Re: Problem reports for version control systems
On Sat, May 01, 2021 at 12:58:50PM +1200, Lloyd Parkes wrote: > > Germany is pretty much the opposite of New Zealand. It's close to > everywhere, but its last mile access speeds are a bit infamous. > Just for you info... there are a few NetBSD developers in .au, my self included. I haven't had any issues with cvs disconnects. Not to deny you have an issue, just letting you know it works ok for people near you. > I'm running some tests on other local clients and against other CVS mirrors > in the hope that come up with a better characterisation of the problem than > "it doesn't work". > If you have the space, a tcpdump from both sides of your firewall may provide a clue. -- Brett Lymn -- Sent from my NetBSD device. "We are were wolves", "You mean werewolves?", "No we were wolves, now we are something else entirely", "Oh"
Re: posix_spawn issue?
Hi,
I have uploaded simple test Makefiles:
https://www.netbsd.org/~ryoon/gmake-tests-20210502.tar.gz
It is as follows:
$ cat Makefile
all:
${MAKE} -C test-a
clean:
${MAKE} -C test-a clean
$ cat test-a/Makefile
all: a.txt
a.txt:
touch a.txt
clean:
rm -f a.txt
Results is here:
Without posix_spawn:
$ gmake
gmake -C test-a
gmake[1]: Entering directory '/home/ryoon/gmake-tests/test-a'
touch a.txt
gmake[1]: Leaving directory '/home/ryoon/gmake-tests/test-a'
With posix_spawn:
$ gmake
gmake -C test-a
gmake: *** [Makefile:2: all] Error 127
The tarball also includes a result of 'ktrace gmake' in posix_spawn case.
I wish this could be useful for debugging.
Thank you.
Thomas Klausner writes:
> Hi!
>
> NetBSD implements posix_spawn() and Linux does too.
>
> gmake since version 4.3 uses posix_spawn(), but that breaks the build
> of firefox (and libreoffice). Disabling posix_spawn() support in gmake
> works around this problem.[1]
>
> Is there a bug/incompatibility in our posix_spawn() or is there a bug
> in gmake?
>
> If you want to try it out, use wip/gmake and remove the
> CONFIGURE_ARGS.NetBSD+= --disable-posix-spawn
> line.
>
> Cheers,
> Thomas
>
> [1] https://mail-index.netbsd.org/tech-pkg/2021/05/01/msg024834.html
> https://mail-index.netbsd.org/tech-pkg/2021/05/01/msg024835.html
--
Ryo ONODERA // r...@tetera.org
PGP fingerprint = 82A2 DC91 76E0 A10A 8ABB FD1B F404 27FA C7D1 15F3
