Re: [Bug-wget] lex compile problem on AIX 7.1

On Dienstag, 3. Oktober 2017 18:31:45 CEST Avinash Sonawane wrote: > On Mon, Oct 2, 2017 at 8:56 PM, Tim Rühsen <tim.rueh...@gmx.de> wrote: > > On 10/02/2017 10:00 AM, l...@langs.se wrote: > >> Hi! > >> > >> I get the following error when compiling wget

Re: [Bug-wget] Signature verification support in wget?

Hi Ludo, thanks for heads up :-) Darshit just opened an issue at https://gitlab.com/gnuwget/wget2/issues/266. If you don't mind, I would add your suggestions there. With Best Regards, Tim On 08/30/2017 02:52 PM, Ludovic Courtès wrote: > Hello! > > Following the GNU Hackers Meeting there

Re: [Bug-wget] Signature verification support in wget?

On 08/30/2017 05:14 PM, Darshit Shah wrote: > * Unknown PGP keys are always an interesting problem due to the > various differences in how people would like to deal with it. By > default, I would suggest that Wget reports a warning about a missing > key and continues. There would

Re: [Bug-wget] Compiling against libwget built with ubsan

On 08/29/2017 05:03 PM, Avinash Sonawane wrote: > Hello! > > $ cat foo.c > #include > #include > > int main(int argc, char **argv) > { > char foo[] = "FOO"; > printf("%s\n", wget_strtolower(foo)); > > return 0; > } > > $ gcc -lwget foo.c -o foo > //usr/local/lib/libwget.so:

Re: [Bug-wget] Problems building wget2

Hi Josef, please see README.md on how to build wget2. If you installed from git repo (https://gitlab.com/gnuwget/wget2.git) then ./bootstrap ./configure make make check With Best Regards, Tim On 09/27/2017 02:41 PM, Josef Moellers wrote: > I was trying to see if a bug I fixed in wget

Re: [Bug-wget] cipher_list string when using OpenSSL

Hi Jeffrey, thanks for heads up ! Does OpenSSL meanwhile have a PFS for their cipher list ? Currently it looks like that each and every client has to amend their cipher list from time to time. Instead, this should be done in the library. So that new versions automatically make the client code

Re: [Bug-wget] Patch: Make url_file_name also convert remote path to local encoded

On Donnerstag, 2. November 2017 21:09:46 CET YX Hao wrote: > Dear Tim, > > > > The 2nd patch is attached. Please take a review :) Hmmm, is this a stand-alone patch and working without your patch #1 (Fix printing...) ? Please give at least one example (better more) to show what your patch

Re: [Bug-wget] New wget (1.19.2): Unexpected download behaviour for gzip-compressed tarballs (HTTP-header dependent)

On 11/03/2017 09:30 AM, Daniel Stenberg wrote: > On Thu, 2 Nov 2017, Tim Rühsen wrote: > >> How would you (or curl) handle >>  Content-Type: application/x-tar >>  Content-Encoding: gzip > >> when downloading 'x.tar.gz' or 'x.tgz' ? Save the file compres

Re: [Bug-wget] New wget (1.19.2): Unexpected download behaviour for gzip-compressed tarballs (HTTP-header dependent)

On 11/03/2017 06:37 AM, James Cloos wrote: >>>>>> "TR" == Tim Rühsen <tim.rueh...@gmx.de> writes: > > TR> I downloaded/tested thousands of web pages and they behave as if 'Content- > TR> Encoding: gzip' is a compression for the transport. U

Re: [Bug-wget] New wget (1.19.2): Unexpected download behaviour for gzip-compressed tarballs (HTTP-header dependent)

On Mittwoch, 1. November 2017 22:21:38 CET Daniel Stenberg wrote: > On Wed, 1 Nov 2017, Tim Rühsen wrote: > > Content-Encoding: gzip means that the data has been compressed for > > transportation purposes only. > > That's actually not what it means. There's transfer-encodi

Re: [Bug-wget] New wget (1.19.2): Unexpected download behaviour for gzip-compressed tarballs (HTTP-header dependent)

Hi Jens, On Mittwoch, 1. November 2017 17:27:58 CET Jens Schleusener wrote: > Hi, > > the new "wget" release 1.19.2 has got a new feature: > > "gzip Content-Encoding decompression" > > But that feature - at least for my self-conmpiled binary - leads to a > problem if one downloads

Re: [Bug-wget] Support for HTTP 308 (Permanent Redirect)

On Sonntag, 29. Oktober 2017 00:35:54 CET Vasya Pupkin wrote: > Hello, > > Some websites started using HTTP 308 redirects > (https://tools.ietf.org/html/rfc7538) and Wget fails to follow such > redirects currently. I'm not sure if this can be considered a bug, though. > Anyway, would be nice if

Re: [Bug-wget] Failing to compile without depricated features from openssl

On 12/04/2017 01:13 AM, Matthew Thode wrote: > Yep, confirmed that this fixed a possible issue, also tested it with > openssl-1.1. > > We are currently using the attached patch. Thanks, I already pushed your patch (with my small changes) to master yesterday. If there is any issue with it, let me

Re: [Bug-wget] wget 1.19.2 fails on Test-metalink-http.py and Test-metalink-http-quoted.py

On Sonntag, 29. Oktober 2017 21:00:35 CET Arkadiusz Miśkiewicz wrote: > On Sunday 29 of October 2017, Tim Rühsen wrote: > > On Sonntag, 29. Oktober 2017 13:45:53 CET Arkadiusz Miśkiewicz wrote: > > > Hi. > > > > > > Test suite for wget fails here on Test-meta

Re: [Bug-wget] wget 1.19.2 fails on Test-metalink-http.py and Test-metalink-http-quoted.py

On 10/30/2017 12:49 PM, Arkadiusz Miśkiewicz wrote: > On Monday 30 of October 2017, Tim Rühsen wrote: >> On 10/29/2017 09:39 PM, Arkadiusz Miśkiewicz wrote: >>> On Sunday 29 of October 2017, Tim Rühsen wrote: >>>> On Sonntag, 29. Oktober 2017 21:00:35

Re: [Bug-wget] Patch: Avoid unnecessary UTF-8 encoded fallback

Thanks, applied. With Best Regards, Tim On 10/25/2017 12:56 PM, YX Hao wrote: > Dear there, > > Things are clear as the patch shows. ☺ > > == > diff --git a/src/retr.c b/src/retr.c > index a27d58af..c1bc600e 100644 > --- a/src/retr.c > +++

Re: [Bug-wget] one strlen on loop

On 10/25/2017 01:10 AM, Rodgger Bruno wrote: > right? Not quite >> And there seems to be two buffer underflow issues in the old code. >> Please consider fixing it as well: >> >>> if (!c_strncasecmp((tok + (tok_len - 4)), ".DIR", 4)) >> >>> else if (!c_strncasecmp ((tok + (tok_len -

Re: [Bug-wget] Self test results on OS X 10.9

On Montag, 30. Oktober 2017 12:28:23 CET Jeffrey Walton wrote: > Hi Everyone, > > I'm building 1.19.2 from sources. Does anyone have an idea about the > "Use of uninitialized value $addr in concatenation ..."? Looks like a copy misuse of an uninitialized value :-) E.g. in Test-https-pfs.px is

Re: [Bug-wget] Need help

Hi Sameeran, people only answer questions if the know the answer. Your original email was quite confusing. At least I didn't understand your problem and thus didn't answer. Try to be precise and if possible add an example or your error output (the relevant parts). With Best Regards, Tim On

Re: [Bug-wget] Wget not ignoring the return value of a void function in 1.19.5

On 05/08/2018 09:16 AM, Josef Moellers wrote: > Hi, > > While trying to upgrade to 1.19.5, we found a bug in wget (src/host.c) > where the (non-existing) return value of a void function is assigned to > a variable. > > A patch is appended. Thanks, setting timer to NULL is not needed here.

Re: [Bug-wget] Removing commented function

On 05.05.2018 15:00, Sameeran Joshi wrote: > 1.i have a doubt regarding the > static void _urls_to_absolute(WGET_VECTOR *urls, WGET_IRI *base) > > function in > wget2/libwget/html_url.c > file .As its commented why are we keeping it there,can't we

Re: [Bug-wget] gcc: error: unrecognized command line option '-R'

On 19.05.2018 20:53, Jeffrey Walton wrote: > On Sat, May 19, 2018 at 12:27 PM, Tim Rühsen <tim.rueh...@gmx.de> wrote: >> Hi Jeff, >> >> could you 'cd fuzz', then 'make -j1 V=1' and send us the ouput ? >> >> It should include the full gcc command li

Re: [Bug-wget] gcc: error: unrecognized command line option '-R'

Hi Jeff, could you 'cd fuzz', then 'make -j1 V=1' and send us the ouput ? It should include the full gcc command line. Please attach your config.log. Regards, Tim On 19.05.2018 17:28, Jeffrey Walton wrote: > Hi Everyone, > > This looks like a new issue with Wget 1.19.5: > > make >

Re: [Bug-wget] gcc: error: unrecognized command line option '-R'

On 19.05.2018 23:44, Jeffrey Walton wrote: > On Sat, May 19, 2018 at 5:21 PM, Tim Rühsen <tim.rueh...@gmx.de> wrote: >> On 19.05.2018 20:53, Jeffrey Walton wrote: >>> On Sat, May 19, 2018 at 12:27 PM, Tim Rühsen <tim.rueh...@gmx.de> wrote: >>> ... >&

Re: [Bug-wget] Bug-wget mailing list

On 15.05.2018 13:32, Graeme wrote: > I opened the mailing list archives. > http://lists.gnu.org/archive/html/bug-wget/ > > I searched for Windows, looking for information on the latest Wget for > windows version. > > I get 1091 documents but they are in order of score.  I change the > sort to by

Re: [Bug-wget] Wget for WIndows download

On 15.05.2018 13:48, Graeme wrote: > I am trying to find the up to date version of Wget for Windows. > > On https://www.gnu.org/software/wget/faq.html#download there are 4 links. > > Sourceforge and Bart Puype links seem to be dead links. > > Christopher Lewis' link times out. > > Jernej

Re: [Bug-wget] query

Hi, On 06/11/2018 07:30 AM, Md. Qudratullah Phd2011, MGL wrote: > Hi, > I started downloading with wget but it stopped after some time (25% > completion) and I restarted with -c option but it does not show that it > resume download. Every time I restart with -c option, it starts from 0%. > I have

Re: [Bug-wget] Retry on host error

Hi Elliot, thanks for your contribution. I'll care for the integration / merge tomorrow. We maintainers decided to accept your work without the FSF Copyright Assignment from you (will be tagged as Copyright-paperwork-exempt). Let us know if you plan to work on more and we'll send you the FSF

Re: [Bug-wget] Retry on host error

Congrats your patch has been pushed, though I detected a small issue right after that (fixed now). And don't you think --tries should be considered ? Regards, Tim On 12.06.2018 20:53, Tim Rühsen wrote: > Hi Elliot, > > thanks for your contribution. > > I'll care for the inte

Re: [Bug-wget] [unit-test] Crash on test_hsts_new_entry()

Thanks for your report. I just merged another branch into master, this issue seemed to be fixed in there. Please try again latest master. Regards, Tim On 05/22/2018 05:06 PM, Gisle Vanem wrote: > I've built unit-test on Windows (clang-cl). But when running > it, it crashes after the message: >  

Re: [Bug-wget] Error when compiling wget 1.19.5 with c-ares 1.14.0

On 06/04/2018 01:14 PM, Krzysztof Malinowski wrote: > Hello, > > I am trying to build wget 1.19.5 with c-ares 1.14.0 and the > compilation fails with error: > > make[3]: Entering directory '/dev/shm/akm022/wget-1.19.5/src' > CC host.o > host.c: In function 'wait_ares': > host.c:735:11:

Re: [Bug-wget] Wget follows "button" links

On 06/05/2018 11:53 AM, CryHard wrote: > Hey there, > > I've used the following: > > wget --user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36" > --user=myuser --ask-password --no-check-certificate --recursive

Re: [Bug-wget] Wget follows "button" links

xpage=watch.* ? > > Thanks! > > > ​Sent with ProtonMail Secure Email.​ > > ‐‐‐ Original Message ‐‐‐ > > On June 5, 2018 2:40 PM, Tim Rühsen wrote: > >> Hi, >> >> in this case you could try it with -X / --exclude-directories. >> >> E.g.

Re: [Bug-wget] Wget follows "button" links

wiki is hosted > doesn't seem to support this, hence wget keeps redownloading the same files. > > Thanks a lot! > ‐‐‐ Original Message ‐‐‐ > > On June 5, 2018 1:57 PM, Tim Rühsen wrote: > >> On 06/05/2018 11:53 AM, CryHard wrote: >> >>> Hey th

Re: [Bug-wget] Retry on host error

Hi Elliot, On 06/25/2018 11:17 PM, Elliot Chandler wrote: > Hello again, > > Thank you for integrating the patch! Thanks for contributing :-) > Regarding licensing, I don't have immediate plans to contribute more > patches to GNU (kind of swamped with things going on), but it's always a >

Re: [Bug-wget] Why does -A not work?

o-19060204-s.jpeg little-nemo-19060211-s.jpeg little-nemo-19060218-s.jpeg little-nemo-19060225-s.jpeg Regards, Tim On 06/20/2018 09:59 PM, Tim Rühsen wrote: > On 20.06.2018 18:20, Nils Gerlach wrote: >> It does not delete any html-file or anything else. Either it is accepted >> and ke

Re: [Bug-wget] Why does -A not work?

ading those files. From a first glimpse: those files are linked from an RSS / Atom file. Those aren't supported by wget, but wget2 does parse them for URLs. Want to give it a try ? https://gitlab.com/gnuwget/wget2 Regards, Tim > > 2018-06-20 16:58 GMT+02:00 Tim Rühsen : > >> Hi Ni

Re: [Bug-wget] Feature request: option to not download rejected files

On 07/03/2018 12:48 PM, Zoe Blade wrote: >> In Wget2 there is an extra option for this, --filter-urls. > > Thank you Tim, this sounds like exactly what I was after! (It's especially > important when you have wget logged in as a user, to be able to tell it not > to go to the logout page.)

Re: [Bug-wget] Newbie help

On 02.05.2018 19:25, Sameeran Joshi wrote: > Hi,I am a newbie to wget,i have tried some interesting stuffs using wget by > reading documentation,and I am curious about how the internal working of > wget [url] works,what may be the logic used.So I downloaded the source code > but I am confused

Re: [Bug-wget] Add support to bind to a local port

On 03.05.2018 19:16, Darshit Shah wrote: > + { "bindport", _port, > cmd_number }, >{ "bodydata", _data, cmd_string }, There are more tabs here. I think that port binding does work with address 'ANY'. So wouldn't it be possible to use

Re: [Bug-wget] cipher_list string when using OpenSSL

On 10/19/2017 11:49 AM, Jeffrey Walton wrote: > On Thu, Oct 19, 2017 at 5:35 AM, Tim Rühsen <tim.rueh...@gmx.de> wrote: >> Hi Jeffrey, >> >> thanks for heads up ! >> >> Does OpenSSL meanwhile have a PFS for their cipher list ? >> >> Currently it

Re: [Bug-wget] one strlen on loop

Hi Rodgger, thanks for your contribution ! Could you please amend a few things !? Not compilable here: > if (tok_len < 12) && (strchr( tok, '-') != NULL)) If you are about to touch the code, please also add a space where it is missing (between tok and +): > *(tok+ (tok_len - 4)) = '\0'; /*

Re: [Bug-wget] nettle-3.4.tar.gz isn't actually compressed

e. I can't reproduce that issue > > Thank you. > > Regards, > Tomas > > On 20.11.2017 17:07, Tim Rühsen wrote: >> There already has been a discussion about that (starting here: >> http://lists.gnu.org/archive/html/bug-wget/2017-11/msg0.html). >> >>

Re: [Bug-wget] Unexpected result with -H and -D

Hi, this is not a PSL matching, so no libpsl is needed. Just sufmatch() has to be fixed to do (sub)domain matching. Attached is a fix. With Best Regards, Tim On 01/17/2018 03:01 PM, Darshit Shah wrote: > Hi, > > This is a bug in Wget, apparently a really old one! Seems like the bug has >

Re: [Bug-wget] nettle-3.4.tar.gz isn't actually compressed

On 01/19/2018 02:23 AM, Thomas Deutschmann wrote: > Hi, > >> sorry, I lost focus on this issue while waiting for >> more examples. > > In Gentoo we hit this several times: > > https://bugs.gentoo.org/640930 > https://bugs.gentoo.org/639752 > https://bugs.gentoo.org/636238 >

Re: [Bug-wget] Testsuite summary for wget 1.19.4: # FAIL: 8

Thanks for the report. This happens because we try to print a numeric IP address: warn "Failed to resolve $$addr, using $srcdir/certs/wgethosts\n"; But then this is printed only if 'localhost' can't be resolved to 127.0.0.1. What is it in your case ? Could you apply this patch and see what

Re: [Bug-wget] wget 1.19.2 fails on Test-metalink-http.py and Test-metalink-http-quoted.py

Thanks, slightly amended and pushed. With Best Regards, Tim On 01/30/2018 09:42 AM, Arkadiusz Miśkiewicz wrote: > On Monday 30 of October 2017, Tim Rühsen wrote: >> On 10/30/2017 12:49 PM, Arkadiusz Miśkiewicz wrote: >>> On Monday 30 of October 2017, Tim Rühsen wrote: >

Re: [Bug-wget] no data downloaded

On 02/05/2018 12:03 PM, jos vaessen wrote: > Hello, > > I am downloading the home-webpage from my solar-power-converter using > WIFI and its IP adress 160.190.0.1 with WGET. No problem beside that the > page is zipped as 7ZIP. After unzipping it shows the content of > index.html like a

Re: [Bug-wget] How to remove a header?

Hi, On 02/09/2018 12:40 AM, Peng Yu wrote: > Hi, > > wget sets some headers. Is there a way to remove some headers, e.g., > Accept-Encoding? Thanks. No, you can change headers and add headers. But it looks like nobody ever needed/asked for an option to remove headers. With Best Regards, Tim

Re: [Bug-wget] no data downloaded

; layout.css for the colors?). > > Always something new to learn when I think I have seen it all, Jos > > > Op 05-Feb-18 om 17:32 schreef Tim Rühsen: >> Either it's missing credentials (do you have to login with Firefox ?) or >> cookies. >> Some servers deliver

Re: [Bug-wget] "make check" fails in "Test-iri-disabled" on Mac OS X, after "./configure" and "make" of wget-1.19.4 complete

The charset conversion has an issue on OSX since a while (OSX changed something). And we have no OSX developer, so there is nothing to do for us. In the past I had several contacts with charset issues on OSX, who found configuration/installation issues after a while and then silenced (problem

Re: [Bug-wget] segment fault while downloading .tar.gz file

Hi Jay, it seems related to https://gitlab.com/gnuwget/wget2/merge_requests/353. Maybe you like to drop in there... In short: the server ignores 'Accept-Encoding: identity' and gives us a 'Content-Type: gzip'. If you build wget2 with libz (just install zlib1-dev and ./configure again), the

Re: [Bug-wget] Why does -A not work?

Hi Nils, On 06/20/2018 06:16 AM, Nils Gerlach wrote: > Hi there, > > in #wget on freenode I was suggested to write this to you: > I tried using wget to get some images: > wget -nd -rH -Dcomicstriplibrary.org -A > "little-nemo*s.jpeg","*html*","*.html.*","*.tmp","*page*","*display*" -p -e >

Re: [Bug-wget] Why does -A not work?

wget ;) > > Thanks for the information on the filename and path, though. > > Greetings > > 2018-06-20 16:13 GMT+02:00 Tim Rühsen : > >> Hi Nils, >> >> On 06/20/2018 06:16 AM, Nils Gerlach wrote: >>> Hi there, >>> >>> in #wget on

Re: [Bug-wget] Deprecate TLS 1.0 and TLS 1.1

On 06/19/2018 12:44 PM, Loganaden Velvindron wrote: > Hi All, > > As per: > https://tools.ietf.org/html/draft-moriarty-tls-oldversions-diediedie-00 > > Attached is a tentative patch to disable TLS 1.0 and TLS 1.1 by > default. No doubt that this will cause some discussions, I'm open to > hearing

Re: [Bug-wget] Any explanation for the '-nc' returned value?

tps://www.gnu.org/software/wget/manual/html_node/Download-Options.html If the server doesn't support it, it simply won't work. All you can do is not to use -N or ask the server's admin to support it. Regards, Tim > > Best Regards, > YX Hao > > -Original Message- > From

Re: [Bug-wget] Any explanation for the '-nc' returned value?

Hi, from the man pages (--no-clobber): When running Wget without -N, -nc, -r, or -p, downloading the same file in the same directory will result in the original copy of file being preserved and the second copy being named file.1. If that file is

Re: [Bug-wget] Async webcrawling

am personally pretty interested in tuning bottlenecks (CPU, memory, bandwidth, ...), so let me know if there is something and I go for it. You can also PM me with more details, if you don't like to post it in public. Regards, Tim > > On Tue, Jul 31, 2018 at 6:22 PM, Tim Rühsen <mai

Re: [Bug-wget] Async webcrawling

On 31.07.2018 18:39, James Read wrote: > Hi, > > how much work would it take to convert wget into a fully fledged > asynchronous webcrawler? > > I was thinking something like using select. Ideally, I want to be able to > supply wget with a list of starting point URLs and then for wget to crawl >

Re: [Bug-wget] Greek translation error

On 05.08.2018 19:54, Anyparktos wrote: > I noticed an error in the greek translation of wget. It reads "Μήκος" which > is greek for "length" instead of "Μέγεθος" which is "size". I attached a > relevant screenshot for clarity. It drives me crazy, please correct it! > In C locale: Length:

Re: [Bug-wget] [PATCH] Don't limit the test suite HTTPS server to TLSv1

Good catch, thanks ! Regards, Tim On 10.08.2018 14:51, Tomas Hozza wrote: > In Fedora, we are implementing crypto policies, in order to enhance the > security of user systems. This is done on the system level by global > configuration. It may happen that due to the active policy, only > TLSv1.2

Re: [Bug-wget] wget wait_ares void value not ignored

Thanks, it has been remove on 8th of March 2018 (commit 7eff94e881b94d119ba22fc0c29edd65f4e6798b) Regards, Tim On 08/24/2018 09:39 AM, Endre Hagelund wrote: > Hi > > I've encountered a bug when compiling wget 1.19.5 from source with the > following command: > > ./configure --with-cares &&

Re: [Bug-wget] [PATCH] Fixes for issues found by Coverity static analysis

On 08/27/2018 11:01 AM, Tomas Hozza wrote: > Hi Darshit. > > On 25.08.2018 08:20, Darshit Shah wrote: >> Hi Tomas, >> >> Thanks for running the scan and the patches you've made! I briefly glanced >> through those and they seem fine. Of course, they will need to be slightly >> modified to apply to

Re: [Bug-wget] info wget: some improvement proposals on the documentation-content

Thank you, Kalle ! We'll go through the docs soon and amend them. On 08/26/2018 10:08 AM, kalle wrote: > hello, > here my proposals: > > chapter 2, part "download all the URLs specified": make it clearer, what > that exactly means in relationship to URLs describing a directory. Is > the whole

Re: [Bug-wget] Planning update to wget. Should I upstream it?

Hi Richard, On 08/22/2018 08:21 PM, Richard Thomas wrote: > Hi, hope this is the correct way to do this. > > I want to be able to download a webpage and all its prerequisites and > turn it into a multipart/related single file. Now, this requires > identifying and changing URLs which, as most

Re: [Bug-wget] Running wget with -O and -q in the background yields a file wget-log

On 8/28/18 10:13 AM, Tomas Korbar wrote: > Hello, > when quiet option is provided and wget runs in background an empty log file > is created. I think this file is obsolete because quiet option prevents any > log to be written into that file. This problem is caused by redirection of > logs. Wget

Re: [Bug-wget] Deprecate TLS 1.0 and TLS 1.1

On 14.07.2018 23:57, Jeffrey Walton wrote: > On Tue, Jun 19, 2018 at 6:44 AM, Loganaden Velvindron > wrote: >> ... >> As per: >> https://tools.ietf.org/html/draft-moriarty-tls-oldversions-diediedie-00 >> >> Attached is a tentative patch to disable TLS 1.0 and TLS 1.1 by >> default. No doubt that

Re: [Bug-wget] Failed 1.19.5 install on Solaris 11.3

On 18.07.2018 14:58, Jeffrey Walton wrote: > On Wed, Jul 18, 2018 at 7:14 AM, Tim Rühsen wrote: >> Maybe it's an bash/sh incompatibility. Anyways - what does 'make >> install' do !? It basically copies the 'wget' executable into a >> directory (e.g. /usr/local/bin/) that i

Re: [Bug-wget] wget in a 'dynamic' pipe

On 19.07.2018 17:24, Paul Wagner wrote: > Dear wgetters, > > apologies if this has been asked before. > > I'm using wget to download DASH media files, i.e. a number of URLs in > the form domain.com/path/segment_1.mp4, domain.com/path/segment_2.mp4, > ..., which represent chunks of audio or

[Bug-wget] TLS1.3 via GnuTLS

FYI GnuTLS 3.6.3 has been released today with TLS1.3 support (latest draft). So if you rebuild/link wget or wget2 with the new GnuTLS version, you can enable TLS1.3 via --ciphers="NORMAL:+VERS-TLS1.3" (wget) resp. --gnutls-options="NORMAL:+VERS-TLS1.3" (wget2). Wget2 seems to get a 0RTT with

Re: [Bug-wget] wget2 Feature Suggestion - Triston Line

Hi Triston, On 13.07.2018 18:52, Triston Line wrote: > Hi Tim, > > Excellent answer thank you very much for this info, "-N" or > "--timestamping" sounds like a much better way to go, however if I'm > converting links, using wget (1) I think I've read somewhere and noticed > that two separate

Re: [Bug-wget] Failed 1.19.5 install on Solaris 11.3

Maybe it's an bash/sh incompatibility. Anyways - what does 'make install' do !? It basically copies the 'wget' executable into a directory (e.g. /usr/local/bin/) that is listed in your PATH env variable. You can do that by hand. If you want the updated man file, copy wget.1 into your man1

Re: [Bug-wget] [PATCH] Add TLS 1.3 support for GnuTLS

Pushed. Thank you, Tomas ! Regards, Tim On 9/4/18 11:22 AM, Tomas Hozza wrote: > Wget currently allows specifying "TLSv1_3" as the parameter for > --secure-protocol option. However it is only implemented for OpenSSL > and in case wget is compiled with GnuTLS, it causes wget to abort with: >

Re: [Bug-wget] wget in a 'dynamic' pipe

On 9/11/18 5:34 AM, Dale R. Worley wrote: > Paul Wagner writes: >> Now I tried >> >>{ i=1; while [[ $i != 100 ]]; do echo >> "http://domain.com/path/segment_$((i++)).mp4"; done } | wget -O foo.mp4 >> -i - >> >> which works like a charm *as long as the 'generator process' is finite*, >>

Re: [Bug-wget] wget2 Feature Suggestion - Triston Line

On 07/12/2018 08:12 PM, Triston Line wrote: > If that's possible that would help immensely. I "review" sites for my > friends at UBC and we look at geographic performance on their apache and > nginx servers, the only problem is they encounter minor errors from time to > time while recursively

Re: [Bug-wget] request to change retry default

On 07/08/2018 02:59 AM, John Roman wrote: > Greetings, > I wish to discuss a formal change of the default retry for wget from 20 > to something more pragmatic such as two or three. > > While I believe 20 retries may have been the correct default many years > ago, it seems overkill for the modern

Re: [Bug-wget] wget2 Feature Suggestion - Triston Line

Hi, On 07/12/2018 08:12 PM, Triston Line wrote: > Hi Wget team, > > I am but a lowly user and linux sysadmin, however, after noticing the wget2 > project I have wondered about a feature that could be added to the new > version. > > I approve of all the excellent new features already being added

Re: [Bug-wget] wget 1.19.4 - possible double free in url.c

On 03/01/2018 03:01 PM, Volkmar Klatt wrote: > Dear wget maintainer, > > 1) in wget 1.19.4 (and probably earlier) > please check carefully > > static char * convert_fname (char *fname) > in > src/url.c > > I run a OpenBSD machine, x86, ABI=32 > and I guess there's a double free when using

Re: [Bug-wget] About GSoC project: Support QUIC Protocol

On 03/09/2018 08:43 AM, Daniel Stenberg wrote: > On Fri, 9 Mar 2018, Gisle Vanem wrote: > >> I agree on ngtcp2. Foremost because it seems to have good support for >> MSVC/Windows. My next contender would be MozQuic. Written in C++, but >> with C interface. A bit of a bummer for Wget2 or libcurl?

Re: [Bug-wget] Miscellaneous thoughts & concerns

Hi Jeffrey, thanks for your feedback ! On 06.04.2018 23:30, Jeffrey Fetterman wrote: > Thanks to the fix that Tim posted on gitlab, I've got wget2 running just > fine in WSL. Unfortunately it means I don't have TCP Fast Open, but given > how fast it's downloading a ton of files at once, it

Re: [Bug-wget] Miscellaneous thoughts & concerns

ions only). So only under certain conditions,e.g. when closing and opening connections to the same IP often. It combines with TLS False Start, so that you can drop connection latency from 3RTT to 1RTT. 0RTT is possible with TLS1.3, which is coming soon (GnuTLS already supports the current draft

Re: [Bug-wget] wget2 hanging, possible I/O issue

ran) I made some tests yesterday without success. --no-tcp-fastopen makes a small difference, write() sets errno to 32 (broken pipe). Removing the gnulib wrapper code didn't make a difference, neither did removal of SO_REUSEADDR. Regards, Tim > > On Wed, Apr 4, 2018 at 3:22 AM, Ti

Re: [Bug-wget] retry_connrefused?

On 10.04.2018 20:37, Jeffrey Fetterman wrote: > with --tries=5 set, Failed to connect (111) will still instantly abort the > operation. As I wrote, not reproducible here (see my debug output). Please append your debug output. Regards, Tim > On Tue, Apr 10, 2018 at 2:45 AM, T

Re: [Bug-wget] retry_connrefused?

On 04/10/2018 03:12 AM, Jeffrey Fetterman wrote: > --retry_connrefused is mentioned in the documentation but it doesn't seem > to be an option anymore. I can't find a replacement for it, either. My VPN > is being a bit fussy today and I keep having to restart my script because > of 111 errors. >

Re: [Bug-wget] make.exe warnings

On 04/06/2018 04:30 AM, Jeffrey Fetterman wrote: > I've successfully built wget2 through msys2 as a Windows binary, and it > appears to be working (granted I've not used it much yet), but I'm > concerned about some of the warnings that occurred during compilation. > > Unsurprisingly they seem to

Re: [Bug-wget] Miscellaneous thoughts & concerns

n a VM in an instant but I feel like that would be a bottleneck > on what's supposed to be a high performance program. Speaking of high > performance, just how much am I missing out on by not being able to > take advantage of tcp fast open? > > > On Fri, Apr 6, 2018

Re: [Bug-wget] Miscellaneous thoughts & concerns

overwrite .wget-session, >>>> plus you've got to remember I'm on WSL and I'd rather have relevant >>> files >>>> kept within my normal folders rather than my WSL filesystem. >>>> >>> I'm not sure if you've understood TLS Session Resume correctly. TLS

Re: [Bug-wget] --http2=off causes Segmentation fault but ./configure --without-libnghttp2 does not

On 04/09/2018 01:04 PM, Jeffrey Fetterman wrote: > So I wanted to see how scraping a large site compared with multiplexing > off. I used the -http2=off parameter, but I got a segfault. Not reproducible here. Could you give me the whole command line ? > So I decided I'd configure wget2 without

Re: [Bug-wget] --http2=off causes Segmentation fault but ./configure --without-libnghttp2 does not

On 04/09/2018 01:18 PM, Jeffrey Fetterman wrote: > God damnit, I just got it to happen with ./configure --without-libnghttp2 > > Now I'm not sure what is triggering it. If you can trigger it in a Linux VM: Install valgrind, build the code with -g and use wget2_noinstall instead of wget2. e.g.

Re: [Bug-wget] wget2: exclude-directories, in documentation but not functional

On 22.04.2018 08:00, Jeffrey Fetterman wrote: > So there's a directory in a site I've been using wget2 on that has a bunch > of files I don't need, but I can't figure out how to filter it out. > > --exclude-directories is in the documentation but it says it's an unknown > option. > > Was it

Re: [Bug-wget] GSOC Project Availability

Hi Eric, IMO the deadline for GSOC student applications is today 18:00 CEST, so you have to hurry. During 27th March and 23rd April the organizations review and decide for/against proposals. Both, http/2 test suite and WARC library hasn't been addressed by an application so far (but that may

Re: [Bug-wget] wget2 hanging, possible I/O issue

Hi Jeffrey, basically wget2 should work on WSL, I just tested it scarcely two weeks ago without issues. I suspect it might have to do with your dependencies (e.g. did you install libnghttp2-dev ?). To find out, please send your config.log. That allows me to see your compiler, CFLAGS and the

Re: [Bug-wget] wget2 hanging, possible I/O issue

libpcre3-dev libpsl-dev libunistring-dev zlib1g-dev > $ sudo update-alternatives --install /usr/bin/python python > /usr/bin/python3.5 1 > > then the commands outlined as per the documentation. config.log attached. > > On Mon, Apr 2, 2018 at 11:53 AM, Tim Rühsen <tim.rueh...@gm

Re: [Bug-wget] wget2 hanging, possible I/O issue

Hi Jeffrey, possibly I can get my hands on a fast Win10 desktop the coming weekend... no promise but I'll try. With Best Regards, Tim On 04/04/2018 09:54 AM, Tim Rühsen wrote: > Hi Jeffrey, > > I can't tell you. Basically because the only WSL I can get my hands on > is on my w

Re: [Bug-wget] wget2 hanging, possible I/O issue

1-dev >> libiconv-hook-dev libidn2-0-dev liblzma-dev libnghttp2-dev >> libmicrohttpd-dev libpcre3-dev libpsl-dev libunistring-dev zlib1g-dev >> $ sudo update-alternatives --install /usr/bin/python python >> /usr/bin/python3.5 1 >> >> then the commands outlined

Re: [Bug-wget] GSoC'18: DNS over HTTPS.

On 03/22/2018 02:01 PM, Aniketh Gireesh wrote: > Further, In my opinion, I think it would be better as a different > library/directory. I think that would be a better refactoring method as > well as it would be easier to work on the codebase at a later point in > time. Further, as far as my

Re: [Bug-wget] GSoC'18: DNS over HTTPS.

On 03/21/2018 07:11 PM, Aniketh Gireesh wrote: > Hello everyone, > > I'm Aniketh Girish, a computer science undergraduate from India. Just > giving a small introduction about myself, I am quite convenient with open > source contribution and I have done GSoC last year with KDE under Krita, a >

Re: [Bug-wget] Feature request: option to not download rejected files

On 06/29/2018 03:20 PM, Zoe Blade wrote: > For anyone else who needs to do this, I adapted Sergey Svishchev's 1.8-era > patch for 19.1 (one of the few versions I managed to get to compile in OS X; > I'm on a Mac, and not the best programmer): > > recur.c:578 > - if (blacklist_contains

Re: [Bug-wget] A C developer

Hi Tom, On 10/4/18 11:13 AM, Tom Mounet wrote: > Hello, > > I saw your needs for a C developer on Savannah home page. I'm a student > in IT I have good basis in Networking (so TCP/IP) and I've been coding > for the last 2 years in C, It's not my main language but I love it, and > I would love to

Re: [Bug-wget] no post-handshake auth under gnutls

Thanks, Nikos. Slightly amended and pushed. Regards, Tim On 10/8/18 10:47 AM, Nikos Mavrogiannopoulos wrote: > Hi, > It seems that wget does not enable/use post-handshake authentication > with gnutls when running under TLS1.3. > > The enabling of TLS1.3 although transparent for all uses

<    1   2   3   4   5   6   7   8   9   >