Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-08-04 Thread Ángel González
On 04/08/13 20:47, Micah Cowan wrote: On Sat, Aug 03, 2013 at 11:50:48PM +0200, Ángel González wrote: If stdbuf(1) was installed, wget could use it to disable the std buffering. Adding yet more variation between systems... Thanks for pointing that out; I'd completely forgotten about stdbuf...

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-08-04 Thread Giuseppe Scrivano
Micah Cowan writes: > On Thu, Aug 01, 2013 at 01:24:12PM +0200, Giuseppe Scrivano wrote: >> Tim Ruehsen writes: >> >> > That is basically a good idea. >> > >> > Do you have in mind to keep as close to the standard CGI environment >> > variables >> > as possible ? Or do you think of the CGI en

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-08-04 Thread Micah Cowan
On Sat, Aug 03, 2013 at 11:50:48PM +0200, Ángel González wrote: > On 03/08/13 21:07, Micah Cowan wrote: > >On Sat, Aug 03, 2013 at 04:11:59PM +0200, Tim Rühsen wrote: > >>As a second option, we could introduce (now or later) > >>--name-filter-program="program REGEX" > >> > >>The 'program' answe

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-08-03 Thread Ángel González
On 03/08/13 21:07, Micah Cowan wrote: On Sat, Aug 03, 2013 at 04:11:59PM +0200, Tim Rühsen wrote: As a second option, we could introduce (now or later) --name-filter-program="program REGEX" The 'program' answers each line it gets (the original filename) by excactly one output line (the

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-08-03 Thread Micah Cowan
On Sat, Aug 03, 2013 at 04:11:59PM +0200, Tim Rühsen wrote: > As a second option, we could introduce (now or later) > --name-filter-program="program REGEX" > > The 'program' answers each line it gets (the original filename) by excactly > one > output line (the new filename) as long as Wget

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-08-03 Thread Tim Rühsen
Ok, one step back. We already have - conditionally compilable - PCRE library access plus POSIX regex for --adjust and --reject, switchable by --regex-type. Why not use similar mechanisms for consistency ? --name-filter=REGEX The content of REGEX then depends of the --regex-type (Posix

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-08-02 Thread Ángel González
On 01/08/13 21:57, Dagobert Michelsen wrote: From a packagers perspective this is a nightmare because this feature introduces weak dependencies to programs which need to be in PATH. If PATH includes only tools shipped by the system it may be necessary to explicitly set the path for each of the t

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-08-02 Thread Micah Cowan
On Thu, Aug 01, 2013 at 01:24:12PM +0200, Giuseppe Scrivano wrote: > Tim Ruehsen writes: > > > That is basically a good idea. > > > > Do you have in mind to keep as close to the standard CGI environment > > variables > > as possible ? Or do you think of the CGI environment principle ? > > If th

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-08-02 Thread Micah Cowan
On Fri, Aug 02, 2013 at 11:53:24AM +0200, Tim Ruehsen wrote: > Hi Dagobert, > > > All this added complexity seems highly overengineered for a feature > > that is not in the core functionality of the tool and that only a > > fraction of the users use. Keep in mind: a good tool is one that does > >

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-08-02 Thread Tim Ruehsen
Hi Dagobert, > All this added complexity seems highly overengineered for a feature > that is not in the core functionality of the tool and that only a > fraction of the users use. Keep in mind: a good tool is one that does > a single job right. Andrew already answered you mail with a bunch of arg

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-08-02 Thread Andrew Cady
On Thu, Aug 01, 2013 at 09:57:40PM +0200, Dagobert Michelsen wrote: > Hi Andrew, Hi Dagobert. I have to say, I find your comments perplexing. But first let me state for the record that PCRE is already a dependency of wget. I didn't add it. > Am 01.08.2013 um 21:22 schrieb Andrew Cady : > > > W

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-08-01 Thread Dagobert Michelsen
Hi Andrew, Am 01.08.2013 um 21:22 schrieb Andrew Cady : > On Thu, Aug 01, 2013 at 10:36:00AM -0700, Micah Cowan wrote: >> On Wed, Jul 31, 2013 at 06:32:18PM -0400, Andrew Cady wrote: >>> By that, I assume you mean to execute the option in the shell. So the >>> existing usage: >>> >>> --rename-o

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-08-01 Thread Andrew Cady
On Thu, Aug 01, 2013 at 10:36:00AM -0700, Micah Cowan wrote: > On Wed, Jul 31, 2013 at 06:32:18PM -0400, Andrew Cady wrote: > > By that, I assume you mean to execute the option in the shell. So the > > existing usage: > > > > --rename-output=s/x/y/ > > > > would (almost) become: > > > > --r

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-08-01 Thread Micah Cowan
On Wed, Jul 31, 2013 at 06:32:18PM -0400, Andrew Cady wrote: > By that, I assume you mean to execute the option in the shell. So the > existing usage: > > --rename-output=s/x/y/ > > would (almost) become: > > --rename-output='perl -lpe "BEGIN{\$|++}" -e s/x/y/' It COULD, sure, but why on e

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-08-01 Thread Giuseppe Scrivano
Tim Ruehsen writes: > That is basically a good idea. > > Do you have in mind to keep as close to the standard CGI environment > variables > as possible ? Or do you think of the CGI environment principle ? > If the latter, we should use an own namespace and let environment variables > start wit

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-08-01 Thread Tim Ruehsen
On Wednesday 31 July 2013 13:45:23 Giuseppe Scrivano wrote: > Tim Ruehsen writes: > >> Niwt apparently uses "an HTTP-based protocol" to communicate between > >> plugins. > > > > Any protocol has it's pros and cons. So why not doing it the same/similar > > way as Micah does ? That seems to be intu

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-07-31 Thread Andrew Cady
On Wed, Jul 31, 2013 at 02:26:52PM +0200, Giuseppe Scrivano wrote: > Hi Andrew, > > Andrew Cady writes: > > > * It includes "/usr/include/unistd.h" instead of . > > Otherwise I get compiler warnings about implicitly defined functions. > > I guess the -I flags need to be changed when make c

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-07-31 Thread Giuseppe Scrivano
Hi Andrew, Andrew Cady writes: > This patch adds an option that allows the user to modify wget's output > filenames using a perl expression. It works similarly to perl's > "rename" script, in terms of how perl is used to modify the filename > string. That is, the original filename is stored in

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-07-31 Thread Giuseppe Scrivano
Tim Ruehsen writes: >> Niwt apparently uses "an HTTP-based protocol" to communicate between >> plugins. > > Any protocol has it's pros and cons. So why not doing it the same/similar way > as Micah does ? That seems to be intuitive - dumping the original HTTP > headers > and add your extension

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-07-31 Thread Tim Ruehsen
On Tuesday 30 July 2013 16:19:52 Andrew Cady wrote: > On Tue, Jul 30, 2013 at 11:24:56AM +0200, Tim Ruehsen wrote: > > On Friday 26 July 2013 14:30:00 Andrew Cady wrote: > > > This would be useful and would not be hard to do. For example, instead > > > of printing just hstat.local_file to the pipe

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-07-30 Thread Andrew Cady
On Tue, Jul 30, 2013 at 11:24:56AM +0200, Tim Ruehsen wrote: > On Friday 26 July 2013 14:30:00 Andrew Cady wrote: > > > This would be useful and would not be hard to do. For example, instead > > of printing just hstat.local_file to the pipe, you could serialize the > > whole hstat struct (say as J

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-07-30 Thread Tim Ruehsen
On Friday 26 July 2013 14:30:00 Andrew Cady wrote: > On Fri, Jul 26, 2013 at 11:27:02AM +0200, Tim Rühsen wrote: > > Thank you for your work, Andrew ! > > I appreciate the appreciation :) > > > Call an external program after downloading and saving, not only with > > filename but also with additio

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-07-29 Thread Micah Cowan
On Fri, Jul 26, 2013 at 02:30:00PM -0400, Andrew Cady wrote: > Incidentally, the former maintainer of wget, Micah Cowan, actually > started working on a wget "competitor" (so to speak) based on a plugin > architecture designed around this concept: Thanks for the mention. :) Not plugins; it's base

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-07-26 Thread Andrew Cady
On Fri, Jul 26, 2013 at 11:27:02AM +0200, Tim Rühsen wrote: > Thank you for your work, Andrew ! I appreciate the appreciation :) > Call an external program after downloading and saving, not only with filename > but also with additional information (e.g. HTTP header stuff like > content-type >

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-07-26 Thread Tim Rühsen
Thank you for your work, Andrew ! In general, I like the idea of being able to read and/or modify the filenames. Just for discussion: What about a more slightly extended option: Call an external program after downloading and saving, not only with filename but also with additional information (e

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-07-25 Thread Andrew Cady
On Thu, Jul 25, 2013 at 07:28:07PM +0200, Dagobert Michelsen wrote: > Hi Andrew, > > I see a number of problems with this approach: > - It is not desirable to enable this on all platforms as there may be > no perl in /usr/bin, so it should be possible to disable it during configure > - perl may

Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-07-25 Thread Dagobert Michelsen
Hi Andrew, Am 25.07.2013 um 16:31 schrieb Andrew Cady : > This patch adds an option that allows the user to modify wget's output > filenames using a perl expression. It works similarly to perl's > "rename" script, in terms of how perl is used to modify the filename > string. That is, the origina

[Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

2013-07-25 Thread Andrew Cady
This patch adds an option that allows the user to modify wget's output filenames using a perl expression. It works similarly to perl's "rename" script, in terms of how perl is used to modify the filename string. That is, the original filename is stored in the perl variable $_, which the user-supp

[Bug-wget] Patch: new option --rename-output: modify output filename with perl

2010-01-15 Thread Andrew Cady
This patch adds an option that allows the user to specify a perl expression used to modify the target filenames of a call to wget. It works similarly to perl's "rename" script, in terms of how perl is used to modify the filename string. That is, the original filename is stored in the perl variabl