[Bug-wget] Race condition on downloaded files among multiple wget instances

2013-09-03 Thread Tomas Hozza
Hello. In Fedora I have a bug [1] from guy that is using wget to test web server network load. He runs multiple instances of wget to download some site recursively. Something like this: for i in `seq 20`; do wget -r http://www.makerwise.com/ & done Some of those wget instances are killed wi

Re: [Bug-wget] Race condition on downloaded files among multiple wget instances

2013-09-03 Thread Tim Ruehsen
On Tuesday 03 September 2013 03:17:56 Tomas Hozza wrote: > In Fedora I have a bug [1] from guy that is using wget > to test web server network load. He runs multiple > instances of wget to download some site recursively. > Something like this: > > for i in `seq 20`; do > wget -r http://www.mak

Re: [Bug-wget] Race condition on downloaded files among multiple wget instances

2013-09-03 Thread Tomas Hozza
- Original Message - > On Tuesday 03 September 2013 03:17:56 Tomas Hozza wrote: > > In Fedora I have a bug [1] from guy that is using wget > > to test web server network load. He runs multiple > > instances of wget to download some site recursively. > > Something like this: > > > > for i

Re: [Bug-wget] Race condition on downloaded files among multiple wget instances

2013-09-03 Thread Tim Ruehsen
On Tuesday 03 September 2013 03:48:15 Tomas Hozza wrote: > > BTW, read() instead of mmap() would not help in this case. > > It would eliminate the SIGBUS because the length of the file > is determined from the number of read bytes by read(). This way > we might end up with truncated file, but will

Re: [Bug-wget] Race condition on downloaded files among multiple wget instances

2013-09-03 Thread Daniel Stenberg
On Tue, 3 Sep 2013, Tim Ruehsen wrote: but in general it is a good idea not to suppress errors or misbehavior, just to make people feel better. Then it should return an error and error message etc, it shouldn't crash with a SIGBUS... -- / daniel.haxx.se

Re: [Bug-wget] Race condition on downloaded files among multiple wget instances

2013-09-03 Thread Darshit Shah
> but in general it is a good idea not to suppress errors or misbehavior, >> just to make people feel better. >> > > Then it should return an error and error message etc, it shouldn't crash > with a SIGBUS... > We could add a simple signal handler to catch a SIGBUS and gracefully exit. -- Thank

Re: [Bug-wget] Race condition on downloaded files among multiple wget instances

2013-09-03 Thread Tim Ruehsen
On Tuesday 03 September 2013 10:59:20 Daniel Stenberg wrote: > On Tue, 3 Sep 2013, Tim Ruehsen wrote: > > but in general it is a good idea not to suppress errors or misbehavior, > > just to make people feel better. > > Then it should return an error and error message etc, it shouldn't crash > with

Re: [Bug-wget] Race condition on downloaded files among multiple wget instances

2013-09-03 Thread Daniel Stenberg
On Tue, 3 Sep 2013, Tim Ruehsen wrote: "There was an unexpected signal SIGBUS. It may be a bug or a misuse of Wget or your hardware is broken. Please think about it.". If you think SIGBUS is the ultimate way to inform a user about an error situation, then by all means do that. I wouldn't. -

Re: [Bug-wget] Race condition on downloaded files among multiple wget instances

2013-09-03 Thread Ángel González
On 03/09/13 11:16, Tim Ruehsen wrote: What should it say than ? My ideas are limited to something like "There was an unexpected signal SIGBUS. It may be a bug or a misuse of Wget or your hardware is broken. Please think about it.". This does not give more information than a "SIGBUS". Ideas welco

Re: [Bug-wget] Race condition on downloaded files among multiple wget instances

2013-09-04 Thread Tim Ruehsen
On Tuesday 03 September 2013 23:17:09 Ángel González wrote: > On 03/09/13 11:16, Tim Ruehsen wrote: > > What should it say than ? > > My ideas are limited to something like > > "There was an unexpected signal SIGBUS. It may be a bug or a misuse of > > Wget or your hardware is broken. Please think a

Re: [Bug-wget] Race condition on downloaded files among multiple wget instances

2013-09-04 Thread Tim Ruehsen
On Tuesday 03 September 2013 23:00:57 Daniel Stenberg wrote: > On Tue, 3 Sep 2013, Tim Ruehsen wrote: > > "There was an unexpected signal SIGBUS. It may be a bug or a misuse of > > Wget > > or your hardware is broken. Please think about it.". > > If you think SIGBUS is the ultimate way to inform a

Re: [Bug-wget] Race condition on downloaded files among multiple wget instances

2013-09-04 Thread Ángel González
On 04/09/13 09:38, Tim Ruehsen wrote: Very well, if this would be possible. Right now I have no idea how to print something like the above. I made Tomas Hozza's test with valgrind and wget having debug info. I got 18x (out of 20x) SIGBUS, but on completely different places in the code. Within the

Re: [Bug-wget] Race condition on downloaded files among multiple wget instances

2013-09-09 Thread Tomas Hozza
- Original Message - > Very well, if this would be possible. Right now I have no idea how to print > something like the above. I made Tomas Hozza's test with valgrind and wget > having debug info. I got 18x (out of 20x) SIGBUS, but on completely different > places in the code. Within the mi

Re: [Bug-wget] Race condition on downloaded files among multiple wget instances

2013-09-09 Thread Anthony Bryan
On Mon, Sep 9, 2013 at 4:49 AM, Tomas Hozza wrote: > - Original Message - >> Very well, if this would be possible. Right now I have no idea how to print >> something like the above. I made Tomas Hozza's test with valgrind and wget >> having debug info. I got 18x (out of 20x) SIGBUS, but on

Re: [Bug-wget] Race condition on downloaded files among multiple wget instances

2013-09-10 Thread Giuseppe Scrivano
Tim Ruehsen writes: > And SIGBUS could also occur out of any other reason (e.g. real bugs in Wget). > > As was already said, replacing mmap by read would not crash (wget_read_file() > reads as many bytes as there are without prior checking the length of the > file). But without additional logic