[Bug-wget] FeatureSpecification - HTTP compression

2012-12-08 Thread 7382846
Hello

I think wget should HTTP compression (Accept-Encoding: gzip, deflate). It
would put less strain on servers being downloading from, and use less of
their bandwidth. Is it okay to add this idea to the
http://wget.addictivecode.org/FeatureSpecifications page? I don't know
where on the page to add it, and thought I should check first before doing
so in case there was a reason it isn't there

Thank you for your time


Re: [Bug-wget] FeatureSpecification - HTTP compression

2012-12-09 Thread Tim Rühsen
Am Samstag, 8. Dezember 2012 schrieb 7382...@gmail.com:
> Hello
> 
> I think wget should HTTP compression (Accept-Encoding: gzip, deflate). It
> would put less strain on servers being downloading from, and use less of
> their bandwidth. Is it okay to add this idea to the
> http://wget.addictivecode.org/FeatureSpecifications page? I don't know
> where on the page to add it, and thought I should check first before doing
> so in case there was a reason it isn't there
> 
> Thank you for your time
> 

The next Wget to come (still called Mget) already has this feature working.

https://github.com/rockdaboot/mget

It still needs some work to do, any help is appreciated.
Mainly ftp and WARC stuff.

Regards, Tim



Re: [Bug-wget] FeatureSpecification - HTTP compression

2012-12-09 Thread Giuseppe Scrivano
7382...@gmail.com writes:

> I think wget should HTTP compression (Accept-Encoding: gzip, deflate). It
> would put less strain on servers being downloading from, and use less of
> their bandwidth. Is it okay to add this idea to the
> http://wget.addictivecode.org/FeatureSpecifications page?

I don't think that page is still used.  Micah, do you think we should
just drop it?

We can have feature requests on Savannah, anyone is against this idea?

Thanks,
Giuseppe



Re: [Bug-wget] FeatureSpecification - HTTP compression

2012-12-10 Thread Micah Cowan
On 12/09/2012 04:11 AM, Giuseppe Scrivano wrote:
> 7382...@gmail.com writes:
> 
>> I think wget should HTTP compression (Accept-Encoding: gzip, deflate). It
>> would put less strain on servers being downloading from, and use less of
>> their bandwidth. Is it okay to add this idea to the
>> http://wget.addictivecode.org/FeatureSpecifications page?
> 
> I don't think that page is still used.  Micah, do you think we should
> just drop it?
> 
> We can have feature requests on Savannah, anyone is against this idea?

I think there should be feature specifications somewhere; don't
particularly care whether they're on Savannah or Wiki, though having
them in both makes it a little easier to organize them, and collaborate
on ideas about them, in addition to tracking them in the bugtracker or
having a maintainer-owned page for them.

A number of the Feature Specifications, last I knew, may no longer be
appropriate for current Wget directions; many of them were basically the
beginning ideas for the Niwt project that I work on sporadically, and
may not be where one would wish to take Wget proper.

This one, of course, looks like something that should still be done for
Wget at some point. But the Wiki is meant for the maintainer, and the
Wget community, so if you want to move them to Savannah, please do feel
free.

-mjc



Re: [Bug-wget] FeatureSpecification - HTTP compression

2012-12-10 Thread Micah Cowan
On 12/09/2012 02:45 AM, Tim Rühsen wrote:
> Am Samstag, 8. Dezember 2012 schrieb 7382...@gmail.com:
>> Hello
>>
>> I think wget should HTTP compression (Accept-Encoding: gzip, deflate). It
>> would put less strain on servers being downloading from, and use less of
>> their bandwidth. Is it okay to add this idea to the
>> http://wget.addictivecode.org/FeatureSpecifications page? I don't know
>> where on the page to add it, and thought I should check first before doing
>> so in case there was a reason it isn't there
>>
>> Thank you for your time
>>
> 
> The next Wget to come (still called Mget) already has this feature working.

I don't think there's been agreement that the next Wget will be called
Mget, or that the current work on Mget represents a step towards
replacing Wget, so I think it's quite misleading to refer to Mget as
"the next Wget to come". Current Wget sources already have concurrent
downloading available, and many other features, both new and
pre-existing, that are not featured in Mget.

Wget proper doesn't have gzip/deflate support currently, but it's not
too hard to add it, and I wouldn't be surprised if it gets added in the
near future; it just hasn't been as high a priority as other
recently-introduced features had been, I believe.

-mjc



Re: [Bug-wget] FeatureSpecification - HTTP compression

2012-12-11 Thread Tim Ruehsen
Am Monday 10 December 2012 schrieb Micah Cowan:
> On 12/09/2012 02:45 AM, Tim Rühsen wrote:
> > Am Samstag, 8. Dezember 2012 schrieb 7382...@gmail.com:
> >> Hello
> >> 
> >> I think wget should HTTP compression (Accept-Encoding: gzip, deflate).
> >> It would put less strain on servers being downloading from, and use
> >> less of their bandwidth. Is it okay to add this idea to the
> >> http://wget.addictivecode.org/FeatureSpecifications page? I don't know
> >> where on the page to add it, and thought I should check first before
> >> doing so in case there was a reason it isn't there
> >> 
> >> Thank you for your time
> > 
> > The next Wget to come (still called Mget) already has this feature
> > working.
> 
> I don't think there's been agreement that the next Wget will be called
> Mget, or that the current work on Mget represents a step towards
> replacing Wget, so I think it's quite misleading to refer to Mget as
> "the next Wget to come".

Sorry, yes. It is a bit misleading.

> Current Wget sources already have concurrent
> downloading available, and many other features, both new and
> pre-existing, that are not featured in Mget.
> 
> Wget proper doesn't have gzip/deflate support currently, but it's not
> too hard to add it, and I wouldn't be surprised if it gets added in the
> near future; it just hasn't been as high a priority as other
> recently-introduced features had been, I believe.

Maybe there is a misunderstanding.
Due to our postings earlier this year, I am *NOT* working on new features 
(regarding Wget) right now. The current goal is to provide most Wget 
features/options to satisfy the Wget test suite. Just as a side effect, there 
are some new features in Mget, but not many.

While doing that "big refactoring" I moved lot's of functionality into a 
library form. As soon as I am done with Wget compatibility, I will create this 
libmget (+ docs & examples), which I will present here for discussion before 
it moves to libwget (or whatever we agree on). That we could merge 
Mget/Wget/libwget and/or even use the library to provide seperate tools.

Doesn't Niwt and Wget have some common code to use ? e.g. HTTP parsing / 
download / handling, HTML pasring, CSS parsing, Cookie handling and much more 
? Maybe we could agree on some common library code instead of having two ?

And after all, I appreciate any helping hands !

> 
> -mjc

Regards,

 Tim Rühsen