Re: [9fans] another webfs question
It seems I'm hitting this error when sending some GET requests:
In /sys/src/cmd/webfs/url.c:
if(strstr(url, %00)){
werrstr(escaped NUL in URI);
return -1;
}
I haven't fully understood the comment above, especially if it is against
the RFC to have an escaped NUL in an url, but this can actually happen,
at least with queries to a bittorrent tracker. For example when specifying
the info hash of a specific torrent when sending a scrape request:
http://bttracker.debian.org:6969/scrape?info_hash=%F1%AE%D2%E5%15%A0%BD%F1%41%54%9D%44%00%47%AB%97%81%2B%69%16
(13th char in the info hash is a NUL)
I get a reply to that one both with wget on linux or hget on plan 9,
while webfs gives the error from the code above.
So is it webfs that needs fixing for that case, or are the other tools
breaking some RFC with that?
rfc2396 doesn't mention any restrictions; %00 is legal.
- erik
Re: [9fans] another webfs question
2009/3/27 erik quanstrom quans...@quanstro.net:
It seems I'm hitting this error when sending some GET requests:
In /sys/src/cmd/webfs/url.c:
if(strstr(url, %00)){
werrstr(escaped NUL in URI);
return -1;
}
I haven't fully understood the comment above, especially if it is against
the RFC to have an escaped NUL in an url, but this can actually happen,
at least with queries to a bittorrent tracker. For example when specifying
the info hash of a specific torrent when sending a scrape request:
http://bttracker.debian.org:6969/scrape?info_hash=%F1%AE%D2%E5%15%A0%BD%F1%41%54%9D%44%00%47%AB%97%81%2B%69%16
(13th char in the info hash is a NUL)
I get a reply to that one both with wget on linux or hget on plan 9,
while webfs gives the error from the code above.
So is it webfs that needs fixing for that case, or are the other tools
breaking some RFC with that?
rfc2396 doesn't mention any restrictions; %00 is legal.
Yeah, there aren't any. That's the point of URL encoding; NULL bytes
are as acceptable as any other, and your client should be able to
handle them -- so I think that webfs check is just bogus. It should
just encode it as a \0 and pass it through.
--dho
- erik
Re: [9fans] another webfs question
Ok, thanks to both. In the meanwhile, mjl pointed me to
http://www.ietf.org/rfc/rfc3986.txt, sect. 7.3, where this can found:
Note, however, that the %00 percent-encoding
(NUL) may require special handling and should be rejected if the
application is not expecting to receive raw data within a component.
This apparently could be the reason behind the current behaviour of webfs.
I'll try and fix that in webfs unless somebody beats me to it (please
do! ;) ).
Cheers,
Mathieu
---BeginMessage---
2009/3/27 erik quanstrom quans...@quanstro.net:
It seems I'm hitting this error when sending some GET requests:
In /sys/src/cmd/webfs/url.c:
if(strstr(url, %00)){
werrstr(escaped NUL in URI);
return -1;
}
I haven't fully understood the comment above, especially if it is against
the RFC to have an escaped NUL in an url, but this can actually happen,
at least with queries to a bittorrent tracker. For example when specifying
the info hash of a specific torrent when sending a scrape request:
http://bttracker.debian.org:6969/scrape?info_hash=%F1%AE%D2%E5%15%A0%BD%F1%41%54%9D%44%00%47%AB%97%81%2B%69%16
(13th char in the info hash is a NUL)
I get a reply to that one both with wget on linux or hget on plan 9,
while webfs gives the error from the code above.
So is it webfs that needs fixing for that case, or are the other tools
breaking some RFC with that?
rfc2396 doesn't mention any restrictions; %00 is legal.
Yeah, there aren't any. That's the point of URL encoding; NULL bytes
are as acceptable as any other, and your client should be able to
handle them -- so I think that webfs check is just bogus. It should
just encode it as a \0 and pass it through.
--dho
- erik
---End Message---
Re: [9fans] another webfs question
Yeah, there aren't any. That's the point of URL encoding; NULL bytes are as acceptable as any other, and your client should be able to handle them -- so I think that webfs check is just bogus. It should just encode it as a \0 and pass it through. (you do mean %00 should result in a byte with value 0, not two bytes (in c notation) '\\' and '0', right?) assuming that every application that uses webfs is prepared to handle a null byte in the middle of a string. what webfs does — complaining loudly — is much preferrable to programs misbehaving silently. since it's quite likely that plan 9 applications are not going to properly deal with a null in a string, it's probablly a good implementation strategy unless you're willing to test all the programs that use webfs to make sure that this case is properly handled. - erik
Re: [9fans] another webfs question
2009/3/27 erik quanstrom quans...@coraid.com: Yeah, there aren't any. That's the point of URL encoding; NULL bytes are as acceptable as any other, and your client should be able to handle them -- so I think that webfs check is just bogus. It should just encode it as a \0 and pass it through. (you do mean %00 should result in a byte with value 0, not two bytes (in c notation) '\\' and '0', right?) Yes, I meant '\0'. assuming that every application that uses webfs is prepared to handle a null byte in the middle of a string. what webfs does — complaining loudly — is much preferrable to programs misbehaving silently. since it's quite likely that plan 9 applications are not going to properly deal with a null in a string, it's probablly a good implementation strategy unless you're willing to test all the programs that use webfs to make sure that this case is properly handled. Ok, but then valid applications such as this one can't use webfs. I think something needing this could solve the issue by having the application import webfs into its own namespace, and then sending some sort of ctl command telling it to set an option to allow null bytes. --dho - erik
Re: [9fans] another webfs question
assuming that every application that uses webfs is prepared
to handle a null byte in the middle of a string. what webfs does
— complaining loudly — is much preferrable to programs misbehaving
silently. since it's quite likely that plan 9 applications are not
going to properly deal with a null in a string, it's probablly
a good implementation strategy unless you're willing to test
all the programs that use webfs to make sure that this case
is properly handled.
Ok, but then valid applications such as this one can't use webfs. I
think something needing this could solve the issue by having the
application import webfs into its own namespace, and then sending some
sort of ctl command telling it to set an option to allow null bytes.
read to the end:
unless you're willing to test
all the programs that use webfs to make sure that this case
is properly handled.
i think it would be a bad idea to add a control swizzle bit
to avoid testing. testing is not that hard.
grep webfs `{find /sys/src /rc/bin |grep '\.[chy]$'} | grep -v /webfs/
/sys/src/cmd/webcookies.c: * Cookie file system. Allows hget and multiple
webfs's to collaborate.
/sys/src/cmd/webfsget.c:/* Example of how to use webfs */
/sys/src/cmd/webfsget.c:fprint(2, usage: webfsget [-b baseurl] [-m
mtpt] [-p postbody] url\n);
you can search contrib, too. i'm sure that abaco falls on
its face when confronted with a 0 in a url.
- erik
[9fans] another webfs question
Hi all,
It seems I'm hitting this error when sending some GET requests:
In /sys/src/cmd/webfs/url.c:
if(strstr(url, %00)){
werrstr(escaped NUL in URI);
return -1;
}
I haven't fully understood the comment above, especially if it is against
the RFC to have an escaped NUL in an url, but this can actually happen,
at least with queries to a bittorrent tracker. For example when specifying
the info hash of a specific torrent when sending a scrape request:
http://bttracker.debian.org:6969/scrape?info_hash=%F1%AE%D2%E5%15%A0%BD%F1%41%54%9D%44%00%47%AB%97%81%2B%69%16
(13th char in the info hash is a NUL)
I get a reply to that one both with wget on linux or hget on plan 9,
while webfs gives the error from the code above.
So is it webfs that needs fixing for that case, or are the other tools
breaking some RFC with that?
Cheers,
Mathieu
