On 06.06.22 23:07, Eric Gallager wrote:
I was trying to use wget to download all of the calendar files on my city website: http://concordnh.gov/iCalendar.aspx Admittedly I'm kind of inexperienced at wget, so I probably didn't use the right command-line, but anyways it was: $ wget -r --tries=3 -c --random-wait -E --preserve-permissions --no-parent https://www.concordnh.gov/iCalendar.aspx Here is the tail end of the terminal output leading up to the crash:--2022-06-04 21:23:09-- http://www.concordnh.gov/1737/Concord-Municipal-Airport Reusing existing connection to www.concordnh.gov:80. HTTP request sent, awaiting response... 302 Found Location: http://www.concordnh.gov/344/Concord-Municipal-Airport [following] --2022-06-04 21:23:09-- http://www.concordnh.gov/344/Concord-Municipal-Airport Reusing existing connection to www.concordnh.gov:80. HTTP request sent, awaiting response... 200 OK Length: 96714 (94K) [text/html] Saving to: ‘www.concordnh.gov/1737/Concord-Municipal-Airport.html’ www.concordnh.gov/1737/Concord-Municipal-Airport.html 100%[====================================================================================================================================================>] 94.45K 370KB/s in 0.3s 2022-06-04 21:23:10 (370 KB/s) - ‘www.concordnh.gov/1737/Concord-Municipal-Airport.html’ saved [96714/96714] Segmentation fault: 11 $ Here is my wget --version info: $ wget --version GNU Wget 1.21.3 built on darwin20.6.0. -cares +digest -gpgme +https +ipv6 +iri +large-file -metalink +nls +ntlm +opie +psl +ssl/gnutls Wgetrc: /Users/ericgallager/.wgetrc (user) /opt/local/etc/wgetrc (system) Locale: /opt/local/share/locale Compile: /usr/bin/clang -DHAVE_CONFIG_H -DSYSTEM_WGETRC="/opt/local/etc/wgetrc" -DLOCALEDIR="/opt/local/share/locale" -I. -I../lib -I../lib -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk -DNDEBUG -pipe -Os -std=c99 -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk -arch x86_64 Link: /usr/bin/clang -DNDEBUG -pipe -Os -std=c99 -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk -arch x86_64 -L/opt/local/lib -Wl,-headerpad_max_install_names -Wl,-syslibroot,/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk -arch x86_64 -L/opt/local/lib -lpcre2-8 -L/opt/local/lib -lidn2 -L/opt/local/lib -lnettle /opt/local/lib/libgnutls.dylib -L/opt/local/lib -lz -L/opt/local/lib -lpsl ../lib/libgnu.a /opt/local/lib/libiconv.dylib /opt/local/lib/libintl.dylib -Wl,-framework -Wl,CoreFoundation /opt/local/lib/libunistring.dylib Copyright (C) 2015 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://www.gnu.org/licenses/gpl.html>. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Originally written by Hrvoje Niksic <[email protected]>. Please send bug reports and questions to <[email protected]>. $ I have it installed via MacPorts, and MacPorts tells me that this is the latest version: $ port -v livecheck wget wget seems to be up to date $ It took several hours before the crash happened, so I don't really want to run the command again just to get a backtrace... any hints on how to reduce the command-line to get a minimized test-case are welcome. Thanks, Eric Gallager
Hi Eric,for a backtrace you'll need the debug symbols for the wget binary. You can check that with the `file` command, e.g. `file /usr/bin/wget` (replace /usr/bin with the location of wget on your machine).
If you don't have the debug symbols, it doesn't make sense to run the wget command for a backtrace.
In case you have a binary with debug symbols, run the command over night with valgrind (valgrind wget -r ...).
You can likely drop --random-wait to speed up the download.When the crash happened, did you start with a clean directory or did you continue the download ? I am asking because of the -c option - could help to figure out how to reproduce.
Another question: can you check whether your machine runs out of memory when running wget ? For a long download wget may take a lot of memory...
Regards, Tim
OpenPGP_signature
Description: OpenPGP digital signature
