Re: Fwd: gawk core dumped on too many input values
On Sun, 27 Aug 2023 13:24:55 -0500
Ed Morton wrote:
> This (original email below) turned out to be a general cygwin issue, not
> a gawk issue:
>
> $ LC_ALL=C sed 's/x/y/' $(seq 100)
> Segmentation fault (core dumped)
>
> $ LC_ALL=C grep 'foo' $(seq 100)
> Segmentation fault (core dumped)
>
> Regards,
>
> Ed.
>
> Forwarded Message
> Subject: gawk core dumped on too many input values
> Date: Sun, 27 Aug 2023 08:09:54 -0500
> From: Ed Morton
> To: bug-g...@gnu.org
>
>
>
> Configuration Information [Automatically generated, do not change]:
> Machine: x86_64
> OS: cygwin
> Compiler: gcc
> Compilation CFLAGS: -ggdb -O2 -pipe -Wall -Werror=format-security
> -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector-strong
> --param=ssp-buffer-size=4
> -fdebug-prefix-map=/cygdrive/d/a/scallywag/gawk/gawk-5.2.2-1.x86_64/build=/usr/src/debug/gawk-5.2.2-1
>
> -fdebug-prefix-map=/cygdrive/d/a/scallywag/gawk/gawk-5.2.2-1.x86_64/src/gawk-5.2.2=/usr/src/debug/gawk-5.2.2-1
>
> -DNDEBUG
> uname output: CYGWIN_NT-10.0-22621 TournaMart_2023 3.4.8-1.x86_64
> 2023-08-17 17:02 UTC x86_64 Cygwin
> Machine Type: x86_64-pc-cygwin
>
> Gawk Version: 5.2.2
>
> Attestation 1:
> I have read
> https://www.gnu.org/software/gawk/manual/html_node/Bugs.html.
> Yes
>
> Attestation 2:
> I have not modified the sources before building gawk.
> True
>
> Description:
> I was trying to test something related to ARG_MAX when I ran the
> awk script below and it core dumped instead of reporting an error
> and exiting gracefully. In case it's useful getconf ARG_MAX outputs
> 32000.
>
> Repeat-By:
> $ LC_ALL=C awk 'BEGIN{print ARGC}' $(seq 100)
> Segmentation fault (core dumped)
Thanks for the report.
I will submit a patch for this issue shortly.
--
Takashi Yano
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: process substitution to create a virtual file doesn't work in chroot environment
On Aug 27 18:13, Cary Lewis via Cygwin wrote: > In a cygwin process that is started either from mintty or bash directly the > following: > > $ user=234 > > $ ./cat <(echo $user) > 234 > works as expected. > > But after a chroot: >From https://cygwin.com/cygwin-ug-net/highlights.html: chroot is supported. Kind of. Chroot is not a concept known by Windows. This implies some serious restrictions. First of all, the chroot call isn't a privileged call. Any user may call it. Second, the chroot environment isn't safe against native windows processes. Given that, chroot in Cygwin is only a hack which pretends security where there is none. For that reason the usage of chroot is discouraged. Don't use it unless you really, really know what you're doing. > $ chroot . ./bash > user=234 > $ ./cat <(echo $user) > ./cat: /dev/fd/63: No such file or directory > > In the directory I am chrooting in, I created a tmp folder, as well as > proc, proc/self, and proc/self/fd, and a dev directory. > > Can someone explain why process substitution to create a virtual file > doesn't work in a chroot environment? /dev/fd is a symlink pointing into nirvana after using chroot. /dev/fd symlinks to /proc/self/fd, but in the chroot'ed environment there's no /proc anymore. I would like to underline what is written in the above Cygwin documentation snippet: The chroot implementation is old, bad, and deprecated. I was going to rip it out entirely for I don't know how often already, but there was always somebody asking to keep it. Given that it never did what chroot is intended, don't use it. Corinna -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Documenting Cygwin on NFS, no longer only exchange-medium! Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a d
On Aug 28 07:35, Cedric Blancher via Cygwin wrote: > On Sun, 27 Aug 2023 at 14:35, Corinna Vinschen via Cygwin > wrote: > > As for FIFOs on NFS, as I wrote, they never worked as desired on NFS. > > For kicks I tried back until Cygwin 3.3. Given this isn't a regression, > > it won't be fixed for Cygwin 3.4. > > I am pretty sure this is a regression, as blast, staden and other > software were built and operated on Cygwin. The fact that some software works doesn't mean they use a certain feature. What, for instance, if they just created their FIFOs on a local tmp dir? I actually *tested* it on - Cygwin 3.3.1-1 from 2021-10-28 - Cygwin 3.2.0-1 from 2021-03-29 - Cygwin 3.1.1-1 from 2019-12-18 - Cygwin 3.0.7-1 from 2019-04-30 - Cygwin 2.9.0-3 from 2017-09-12 FIFOs on NFS didn't work in all cases. We can go back in time prior to 2017 if you like, but Cygwin FIFOs were always just created as symlinks, and NFS symlinks were never tested for being a device, so they never have been treated as FIFOs. Also, NFS FIFOs have never been recognized by the file handling code as Cygwin FIFOs, they were just shown as such in stat(2). Corinna -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Documenting Cygwin on NFS, no longer only exchange-medium! Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a d
On Sun, Aug 27, 2023 at 2:35 PM Corinna Vinschen via Cygwin wrote: > > On Aug 26 19:44, Martin Wege via Cygwin wrote: > > On Fri, Aug 25, 2023 at 2:19 PM Corinna Vinschen via Cygwin > > wrote: > > > > > > On Aug 23 01:05, Roland Mainz via Cygwin wrote: > > > > Note that Cygwin does not interpret the file |myfifo.fifo| as FIFO, > > > > instead it comes back as a symlink "myfifo.fifo -> ':\0:c4:1000'". > > > > > > > > AFAIK there are (at least) these two options to fix the problems: > > > > 1. Check whether the filesystem for the fifos path is NFS > > > > (cgywin.dll's |fs.fs_is_nfs()|), and if it is a symlink check if it > > > > starts with ':\0:c4:' (assuming "c4" is the prefix for inodes created > > > > with |mkfifo()|). If this condition is |true|, then cygwin |stat()|, > > > > |open()| etc. should treat this inode as FIFO. > > > > > > The downside is that it is not possible to diffentiate between Cygwin > > > FIFOs and real FIFOs created from the remote side in `ls -l' > > > output. Note that Cygwin returns the NFS stat info verbatim, so > > > a real FIFO is returned as a real FIFO: > > > > > > linux$ mkfifo bar > > > cygwin$ ls -l bar > > > prw-r--r-- 1 corinna vinschen 0 Aug 25 13:58 bar > > > > > > The idea was always to use NFS as exchange medium, but not as > > > installation medium for the entire distro or to keep Cygwin home > > > dirs on NFS. There were times where NFS was pretty unstable. > > > I used NFS for quite some time to build Cygwin packages, but at one > > > point I got trouble (performance problems with multiple concurrent > > > processes accessing an NFS share, build errors out of the blue), > > > so I switched to Samba shares, albeit grudgingly. I'm not yet > > > sure if the problems are fixed. At least a recent OpenSSH build > > > ran through without problems... > > > > Corinna, could you please update the Cygwin documentation then? We use > > NFS, both Windows builtin and CITI NFSv4.1, with Cygwin for years > > without major problems, so that comment about exchange-medium-only is > > clearly wrong! > > I'm talking about the idea how users would use NFS, and the idea was > that nobody in their right mind would use a crippled filesystem (from > the Windows POV) for the entire installation. Before you came up with > your complaints, nobody here even knew that somebody actually did that. > > NFS is a niche filesystem on Windows and was never in the focus of > anybody here, except me, because I was toying around with it and was > happy to have it working. As far as I knew until a few days ago, > I was the only user of Cygwin's NFS support. > > As for FIFOs on NFS, as I wrote, they never worked as desired on NFS. > For kicks I tried back until Cygwin 3.3. Given this isn't a regression, > it won't be fixed for Cygwin 3.4. > > As Brian wrote in his reply to your mail, we are all volunteers here. > Everybody working on Cygwin is working in their spare time and/or on the > goodwill of their employer. > > Cygwin is open source. You can fix or implement stuff yourself and send > the patches to the cygwin-patches ML for review. Otherwise you will > have to let us, the people actually working on the project, decide when > and how something gets implemented. > > Having said that, I have a local patch enabling FIFOs on NFS. I might > push it at one point to the main branch, but the pressure you guys are > currently building up on this mailing list puts me right off. Corinna, please do not let people put you off. I don't even think it was intended as pressure. Could you please share the patch for testing? -- Josh -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: gawk core dumped on too many input values
On Mon, Aug 28, 2023 at 1:08 AM Jeremy Hetzler via Cygwin
wrote:
>
> On Sun, Aug 27, 2023 at 2:25 PM Ed Morton via Cygwin
> wrote:
> >
> > This (original email below) turned out to be a general cygwin issue, not
> > a gawk issue:
> >
> > $ LC_ALL=C sed 's/x/y/' $(seq 100)
> > Segmentation fault (core dumped)
> >
> > $ LC_ALL=C grep 'foo' $(seq 100)
> > Segmentation fault (core dumped)
> >
>
> Seems that all commands linked with cygwin1.dll will fault if you pass them
> a long enough arglist.
>
> For me, /bin/true faults on {1..258231} but not {1..258230}.
>
> > $ /bin/true {1..258230}
> >
>
>
> > $ /bin/true {1..258231}
> > Segmentation fault (core dumped)
>
>
> strace, which is not linked with cygwin1.dll, exits cleanly.
>
> > $ /bin/strace {1..30}
> > -bash: /bin/strace: Argument list too long
>
>
> See this page [1] on maximum argument lengths.
>
> It would be nice to document this limit, whatever it is.
Is this limit?
$ getconf -a | grep -E 'ARG_MAX'
_POSIX_ARG_MAX 4096
ARG_MAX 32000
>
> It would also be nice to return an error to the shell on too-long arglist.
+1
--
Josh
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Difference between cygwin versions
On one machine I have cygwin version 3.1.7(0.340/5/3), I can copy cygwin binaries and the corresponding dlls to a folder and then run the cygwin programs without any problems. But on another machine I have a newer cygwin, 3.4.7-1.x86_64, and when I do the same thing, the programs do not work: e.g. mkdir /tmp/test cd /tmp/test $ ls -1 bash.exe chroot.exe cygiconv-2.dll cygintl-8.dll cygncursesw-10.dll cygreadline7.dll cygwin1.dll I copied all of bash's required dlls to /tmp/test When I do a ./bash, it exits immediately on 3.4.7, but on 3.1.7 it works correctly. Has something changed in cygwin that prevents this from working? -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
RE: [EXTERNAL] Re: scp stalls on uploading in cygwin 3.5 current master.
> You seems to reffer Linux man, however, this patch calls I was referring to a known behavior. Your patch gets to call send(s,"",0), which is technically a write call, and which in this case, falls into an undefined domain for its argument, and hence, may be expected to change without notice. That's all I was referring to. Anton Lavrentiev Contractor NIH/NLM/NCBI -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: [EXTERNAL] Re: scp stalls on uploading in cygwin 3.5 current master.
On Mon, 28 Aug 2023 13:37:11 + "Lavrentiev, Anton (NIH/NLM/NCBI) [C]" wrote: > > You seems to reffer Linux man, however, this patch calls > > I was referring to a known behavior. Your patch gets to call send(s,"",0), > which is technically a write call, and which in this case, falls into an > undefined > domain for its argument, and hence, may be expected to change without notice. > That's all I was referring to. https://learn.microsoft.com/en-us/windows/win32/api/winsock2/nf-winsock2-send Calling send with a len parameter of zero is permissible and will be treated by implementations as successful. In such cases, send will return zero as a valid value. For message-oriented sockets, a zero-length transport datagram is sent. -- Takashi Yano -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
RE: [EXTERNAL] Re: scp stalls on uploading in cygwin 3.5 current master.
> For message-oriented sockets, a zero-length transport datagram is sent. And how is that acceptable? This will interject a message into some application protocol, which may not be expected at the application level. Anton Lavrentiev Contractor NIH/NLM/NCBI -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Difference between cygwin versions
On 8/28/23, Cary Lewis via Cygwin wrote: > On one machine I have cygwin version 3.1.7(0.340/5/3), I can copy cygwin > binaries and the corresponding dlls to a folder and then run the cygwin > programs without any problems. > > But on another machine I have a newer cygwin, 3.4.7-1.x86_64, and when I > do the same thing, the programs do not work: > > e.g. > > mkdir /tmp/test > cd /tmp/test > $ ls -1 > bash.exe > chroot.exe > cygiconv-2.dll > cygintl-8.dll > cygncursesw-10.dll > cygreadline7.dll > cygwin1.dll > > I copied all of bash's required dlls to /tmp/test > > When I do a ./bash, it exits immediately on 3.4.7, but on 3.1.7 it works > correctly. *forward slash* ?? That doesn't work for me, but .\bash does $ uname -a CYGWIN_NT-10.0-19044 i3668 3.4.7-1.x86_64 2023-06-16 14:04 UTC x86_64 Cygwin works for me: C:\Temp\test>dir Directory of C:\Temp\test 08/28/2023 10:04 AM . 08/28/2023 10:04 AM .. 08/28/2023 09:59 AM 854,035 bash.exe 08/28/2023 10:04 AM43,539 chroot.exe 08/28/2023 10:00 AM 1,088,019 cygiconv-2.dll 08/28/2023 10:00 AM 108,563 cygintl-8.dll 08/28/2023 10:01 AM 334,867 cygncursesw-10.dll 08/28/2023 10:00 AM 262,675 cygreadline7.dll 08/28/2023 09:59 AM 2,952,245 cygwin1.dll C:\Temp\test>.\bash.exe bash.exe: warning: could not find /tmp, please create! bash-5.2$ uname windows32 bash-5.2$ exit exit Regards, Lee -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: [EXTERNAL] Re: scp stalls on uploading in cygwin 3.5 current master.
On Mon, 28 Aug 2023 14:07:44 + "Lavrentiev, Anton (NIH/NLM/NCBI) [C]" wrote: > > For message-oriented sockets, a zero-length transport datagram is sent. > > And how is that acceptable? This will interject a message into some > application protocol, > which may not be expected at the application level. Yeah, due to these undesired behaviours, we decided not to adopt this patch. Instead, we reverted the original patch. -- Takashi Yano -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: gawk core dumped on too many input values
On 2023-08-28 05:47, Joshuah Hurst via Cygwin wrote:
On Mon, Aug 28, 2023 at 1:08 AM Jeremy Hetzler via Cygwin
wrote:
On Sun, Aug 27, 2023 at 2:25 PM Ed Morton via Cygwin
wrote:
This (original email below) turned out to be a general cygwin issue, not
a gawk issue:
$ LC_ALL=C sed 's/x/y/' $(seq 100)
Segmentation fault (core dumped)
$ LC_ALL=C grep 'foo' $(seq 100)
Segmentation fault (core dumped)
Seems that all commands linked with cygwin1.dll will fault if you pass them
a long enough arglist.
For me, /bin/true faults on {1..258231} but not {1..258230}.
$ /bin/true {1..258230}
$ /bin/true {1..258231}
Segmentation fault (core dumped)
strace, which is not linked with cygwin1.dll, exits cleanly.
$ /bin/strace {1..30}
-bash: /bin/strace: Argument list too long
See this page [1] on maximum argument lengths.
It would be nice to document this limit, whatever it is.
Is this limit?
$ getconf -a | grep -E 'ARG_MAX'
_POSIX_ARG_MAX 4096
ARG_MAX 32000
On my system, /bin/true and /bin/echo get to between 250K and 256K before seg
faulting, requiring between digits + separator + ptr (10+8)*(250-256)KB ==
3750-3840KB;
current Cygwin and `bash` `:` and `echo` internal commands get to between 128M
and 256M before `bash` fork fails, and Windows gets unhappy with memory full,
paging high, and page thrashing on both paging devices: calmed eventually with a
few interrupts then `exec bash`;
requiring between digits + separator + ptr (10+8)*(128-256)MB == 2.25-4.5GB;
This seems to be extremely conservative:
$ xargs -r --show-limits <<< ' '
Your environment variables take up 9246 bytes
POSIX upper limit on argument length (this system): 20706
POSIX smallest allowable upper limit on argument length (all systems): 4096
Maximum length of command we could actually use: 11460
Size of command buffer we are actually using: 20706
Maximum parallelism (--max-procs must be no greater): 2147483647
$ seq $((256*1024)) | xargs | wc -lwcL
102 262144 1723903 18094
Once a patch is in a test version of Cygwin, I can rebuild and release a test
version of findutils, which includes xargs, with updated results in the test
release announcement.
--
Take care. Thanks, Brian Inglis Calgary, Alberta, Canada
La perfection est atteinte Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter not when there is no more to add
mais lorsqu'il n'y a plus rien à retirer but when there is no more to cut
-- Antoine de Saint-Exupéry
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
json-c.pc seems incorrect in libjson-c-devel-0.16-1
The variables appear to be set to the path of the compiled environment.
$ cygcheck -cd libjson-c-devel
Cygwin Package Information
Package Version
libjson-c-devel 0.16-1
$ cat /usr/lib/pkgconfig/json-c.pc
prefix=/pub/devel/json-c/json-c-0.16-1.x86_64/inst/usr
exec_prefix=/pub/devel/json-c/json-c-0.16-1.x86_64/inst/usr
libdir=/pub/devel/json-c/json-c-0.16-1.x86_64/inst/usr/lib
includedir=/pub/devel/json-c/json-c-0.16-1.x86_64/inst/usr/include
Name: json-c
Description: A JSON implementation in C
Version: 0.16
Requires:
Libs.private:
Libs: -L${libdir} -ljson-c
Cflags: -I${includedir} -I${includedir}/json-c
$ pkg-config --libs json-c
-L/pub/devel/json-c/json-c-0.16-1.x86_64/inst/usr/lib -ljson-c
$ ls /pub/devel/json-c/json-c-0.16-1.x86_64/inst/usr/lib
ls: cannot access
'/pub/devel/json-c/json-c-0.16-1.x86_64/inst/usr/lib': No such file or
directory
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
