Re: [Bug-wget] [Bug-Wget] It's spring cleaning time!
Am Sonntag, 11. Mai 2014, 15:56:15 schrieb Giuseppe Scrivano:
> Darshit Shah writes:
> > Subject: [PATCH] Fix LOTS of compiler warnings
>
> great work! Just some minor comments:
> > * http.c: Fix small memory leak
> >
> > diff --git a/src/css-url.c b/src/css-url.c
> > index f97690d..51e43b4 100644
> > --- a/src/css-url.c
> > +++ b/src/css-url.c
> > @@ -64,8 +64,8 @@ typedef struct yy_buffer_state *YY_BUFFER_STATE;
> >
> > extern YY_BUFFER_STATE yy_scan_bytes (const char *bytes,int len );
> > extern int yylex (void);
> >
> > -#if 1
> > -const char *token_names[] = {
> > +#if 0
> > +static const char *token_names[] = {
> >
> >"CSSEOF",
> >"S",
> >"CDO",
>
> if this is not needed, what about just dropping it? I would prefer we
> keep code that is used somewhere.
Then let's drop it.
Due to version control, the code is not really lost... (if someone minds to
use it again).
>
> > -#define NTLMFLAG_NEGOTIATE_DOMAIN_SUPPLIED (1<<12)
> > -#define NTLMFLAG_NEGOTIATE_WORKSTATION_SUPPLIED (1<<13)
> > -#define NTLMFLAG_NEGOTIATE_LOCAL_CALL(1<<14)
> > -#define NTLMFLAG_NEGOTIATE_ALWAYS_SIGN (1<<15)
> > -#define NTLMFLAG_TARGET_TYPE_DOMAIN (1<<16)
> > -#define NTLMFLAG_TARGET_TYPE_SERVER (1<<17)
> > -#define NTLMFLAG_TARGET_TYPE_SHARE (1<<18)
> > -#define NTLMFLAG_NEGOTIATE_NTLM2_KEY (1<<19)
> > -#define NTLMFLAG_REQUEST_INIT_RESPONSE (1<<20)
> > -#define NTLMFLAG_REQUEST_ACCEPT_RESPONSE (1<<21)
> > -#define NTLMFLAG_REQUEST_NONNT_SESSION_KEY (1<<22)
> > -#define NTLMFLAG_NEGOTIATE_TARGET_INFO (1<<23)
> > +/* #define NTLMFLAG_NEGOTIATE_DOMAIN_SUPPLIED (1<<12) */
> > +/* #define NTLMFLAG_NEGOTIATE_WORKSTATION_SUPPLIED (1<<13) */
> > +/* #define NTLMFLAG_NEGOTIATE_LOCAL_CALL(1<<14) */
> > +/* #define NTLMFLAG_NEGOTIATE_ALWAYS_SIGN (1<<15) */
> > +/* #define NTLMFLAG_TARGET_TYPE_DOMAIN (1<<16) */
> > +/* #define NTLMFLAG_TARGET_TYPE_SERVER (1<<17) */
> > +/* #define NTLMFLAG_TARGET_TYPE_SHARE (1<<18) */
> > +/* #define NTLMFLAG_NEGOTIATE_NTLM2_KEY (1<<19) */
> > +/* #define NTLMFLAG_REQUEST_INIT_RESPONSE (1<<20) */
> > +/* #define NTLMFLAG_REQUEST_ACCEPT_RESPONSE (1<<21) */
> > +/* #define NTLMFLAG_REQUEST_NONNT_SESSION_KEY (1<<22) */
> > +/* #define NTLMFLAG_NEGOTIATE_TARGET_INFO (1<<23) */
>
> same here, what about just dropping them?
Yes, same here.
Darshit, I guess it would be easiest if you change the patch !?
Tim
Re: [Bug-wget] [Bug-Wget] Issues with Metalink support
On 07/05/14 23:46, Jure Grabnar wrote:
Hello,
I wrote two patches regarding this issue based on your suggestions.
The 1st one is crucial for retrieve_from_file() function: it fixes 2
memory corruption bugs.
The 2nd patch is more of an experiment than a real deal and is
intended to get some feedback. It changes parallel download to one
temporary file which is located in the selected directory.
Before download starts it uses posix_fallocate() to allocate space and
mkstemp() for unique name. After download is completed it uses rename()
to rename temporary file to the final file name.
After posix_fallocate() each thread opens file with fopen("wb").
You could use w+b, even though you're not going to read from it.
I opted for fopen() rather than messing around with file descriptors
because I believe it's more portable. I don't know how Windows
would react to file descriptors and I don't have a proper Windows system
to test it out.
It works fine.
On Windows, FILE* are a layer on top of fds, which are themselves a
layer over HANDLEs. To
make things more complex, gnulib provides a different abstraction to wget.
But it should work. The only special bit would be the need to add
O_BINARY, which
gnulib should already be doing for you.
Now, fopen("wb") means file, which was fallocate'd, is
truncated to zero but after first request from the thread, which is
reponsible for the last chunk, it would grow back to at least file_size
- chunk_size. I'm also not sure how devastating that is.
It's up to the filesystem, but I think it would be better to do open (or
dup) + fdopen()
+ fseek rather than the fopen(, "wb"); It also allows you to dispense
with the barrier.
I'm attaching a handmade Metalink file which downloads a 50MB file for
testing purposes. Currently all threads connect to the same server and I
understand we don't support such behaviour but I guess 2-3 threads for
testing purpose don't hurt anyone. :)
I'm open for suggestions.
Regards,
Jure Grabnar
Re: [Bug-wget] Combining domains with directories when filtering
george boukeas writes: > Hello everyone. > > As far as I can tell, when I use wget like this: wget -r -H -D > domain1,domain2 -I dir1,dir2 ... I get everything from domain1/dir1, > domain1/dir2, as well as domain2/dir1 and domain2/dir2. I have not > been able to figure out how to restrain wget to only accept files from > domain1/dir1 and domain2/dir2. Is this possible? have you already had a look at the --accept-regex option of wget? That might help you to solve your problem. Keep it in mind that it is a quite recent feature (added in wget 1.14) so ensure your wget supports it. Cheers, Giuseppe
Re: [Bug-wget] [Bug-Wget] It's spring cleaning time!
Darshit Shah writes:
> Subject: [PATCH] Fix LOTS of compiler warnings
great work! Just some minor comments:
> * http.c: Fix small memory leak
> diff --git a/src/css-url.c b/src/css-url.c
> index f97690d..51e43b4 100644
> --- a/src/css-url.c
> +++ b/src/css-url.c
> @@ -64,8 +64,8 @@ typedef struct yy_buffer_state *YY_BUFFER_STATE;
> extern YY_BUFFER_STATE yy_scan_bytes (const char *bytes,int len );
> extern int yylex (void);
>
> -#if 1
> -const char *token_names[] = {
> +#if 0
> +static const char *token_names[] = {
>"CSSEOF",
>"S",
>"CDO",
if this is not needed, what about just dropping it? I would prefer we
keep code that is used somewhere.
> -#define NTLMFLAG_NEGOTIATE_DOMAIN_SUPPLIED (1<<12)
> -#define NTLMFLAG_NEGOTIATE_WORKSTATION_SUPPLIED (1<<13)
> -#define NTLMFLAG_NEGOTIATE_LOCAL_CALL(1<<14)
> -#define NTLMFLAG_NEGOTIATE_ALWAYS_SIGN (1<<15)
> -#define NTLMFLAG_TARGET_TYPE_DOMAIN (1<<16)
> -#define NTLMFLAG_TARGET_TYPE_SERVER (1<<17)
> -#define NTLMFLAG_TARGET_TYPE_SHARE (1<<18)
> -#define NTLMFLAG_NEGOTIATE_NTLM2_KEY (1<<19)
> -#define NTLMFLAG_REQUEST_INIT_RESPONSE (1<<20)
> -#define NTLMFLAG_REQUEST_ACCEPT_RESPONSE (1<<21)
> -#define NTLMFLAG_REQUEST_NONNT_SESSION_KEY (1<<22)
> -#define NTLMFLAG_NEGOTIATE_TARGET_INFO (1<<23)
> +/* #define NTLMFLAG_NEGOTIATE_DOMAIN_SUPPLIED (1<<12) */
> +/* #define NTLMFLAG_NEGOTIATE_WORKSTATION_SUPPLIED (1<<13) */
> +/* #define NTLMFLAG_NEGOTIATE_LOCAL_CALL(1<<14) */
> +/* #define NTLMFLAG_NEGOTIATE_ALWAYS_SIGN (1<<15) */
> +/* #define NTLMFLAG_TARGET_TYPE_DOMAIN (1<<16) */
> +/* #define NTLMFLAG_TARGET_TYPE_SERVER (1<<17) */
> +/* #define NTLMFLAG_TARGET_TYPE_SHARE (1<<18) */
> +/* #define NTLMFLAG_NEGOTIATE_NTLM2_KEY (1<<19) */
> +/* #define NTLMFLAG_REQUEST_INIT_RESPONSE (1<<20) */
> +/* #define NTLMFLAG_REQUEST_ACCEPT_RESPONSE (1<<21) */
> +/* #define NTLMFLAG_REQUEST_NONNT_SESSION_KEY (1<<22) */
> +/* #define NTLMFLAG_NEGOTIATE_TARGET_INFO (1<<23) */
same here, what about just dropping them?
Giuseppe
