Re: W3C's version of XMLHttpRequest should be abandoned
Hey all, Like Jungkee, I haven't been active editing the XHR spec. I too feel like XHR L1 is very far from trivial. That is because the snapshot Domenic is talking about is not what XHR L1 is supposed to be. XHR L1 is supposed to be a state of what is actually implemented in released browsers at a specific date, not the state of the living standard at said date. Hence the huge focus on testing as a means to filter out what's not supported. This daunting task is, AFAIK, still done manually (including running tests, collecting results, analyzing them and editing the spec accordingly). Honestly, I don't think we have the proper infrastructure nor the proper spec format for handling this properly and in a timely fashion. With nearly all vendors now gone the auto-update way, anything short of full automation is more and more looking like a fool's errand. But for a fully automated snapshot, specs would need to incorporate unit tests (something not too far fetched when you consider how implementation details centered they are now). That's a lot of parts to move, to agree on and to implement. The community handles the problem with sites like http://caniuse.com/. It doesn't tie support to actual spec content but it seems good (detailed) enough to me (as in, that's how I personally check what I can and cannot use). Do authors actually need more detail that this? Would vendors have a use for something more detailed? In summary, I like the idea of XHR L1 but I think it's an impractical amount of work if not automated and I'm uncertain it would be of more use than what the community already created. Sorry for the wall of text ;) -- Julian On 7 August 2015 at 15:52, Jungkee Song jungkee.s...@samsung.com wrote: Hi Art, Hallvord, Julian, and all, Apologies having not been active on it. My feeling is capturing a snapshot for REC would still be a non-trivial task. Unfortunately, I don't seem to be able to spare much time on this work as of now. Sorry for not being able to help. It's my own stance, not the other editors. Domenic's suggestion sound reasonable to me if we are not coming up with a better plan. Best regards, Jungkee -Original Message- From: Arthur Barstow [mailto:art.bars...@gmail.com] Sent: Friday, August 07, 2015 8:37 PM To: Hallvord Reiar Michaelsen Steen; Jungkee Song; Julian Aubourg Cc: WebApps WG Subject: Re: W3C's version of XMLHttpRequest should be abandoned On 8/6/15 8:07 AM, Hallvord Reiar Michaelsen Steen wrote: On Thu, Aug 6, 2015 at 9:15 AM, Anne van Kesteren ann...@annevk.nl mailto:ann...@annevk.nl wrote: According to Art the plan of record is to still pursue https://dvcs.w3.org/hg/xhr/raw-file/default/xhr-1/Overview.html And you correctly note but that was last updated more than a year ago. Last formally published a year and a half ago. And that is mostly my fault. I intended to keep the W3C fork up to date (at least up to a point), but at some point I attempted to simply apply Git patches from Anne's edits to the WHATWG version, and it turned out Git had problems applying them automatically for whatever reason - apparently the versions were already so distinct that it wasn't possible. Since then I haven't found time for doing the manual cut-and-paste work required, and I therefore think it's probably better to follow Anne's advice and drop the W3C version entirely in favour of the WHATWG version. I still like the idea of having a stable spec documenting the interoperable behaviour of XHR by a given point in time - but I haven't been able to prioritise it and neither, apparently, have the other two editors. Jungkee, Julian - we would like your input, in particular whether or not you can still commit to helping with the tasks required to move XHR Level 1 along the Recommendation track. Others - if you can commit to helping with the main tasks (editing, testing, implementation, etc.) for XHR L1, please let us know. -Thanks, AB
Re: [XHR] Content-Length header for data: URLs
But Content-Length is not the same as the length of the string containing the Data URL. On 20 September 2013 03:39, Boris Zbarsky bzbar...@mit.edu wrote: On 9/19/13 9:31 PM, Anne van Kesteren wrote: Doesn't that depend on how you end up storing it whether getting that information is fast and easy to get ahead of time? I suppose, if you store them somewhere where you don't know how much space they're taking up in the storage... But at some point that data: URL was a string, and you can just save its length in an easy-to-get-to place. -Boris
Re: [XHR] Content-Length header for data: URLs
Sure, what I actually meant is that you'd need to somehow pre-parse the data URL to extract the exact length before storage. Dunno how desirable/desired/common this is. On 20 September 2013 04:58, Boris Zbarsky bzbar...@mit.edu wrote: On 9/19/13 10:51 PM, Julian Aubourg wrote: But Content-Length is not the same as the length of the string containing the Data URL. Sure. It can also be a simple formula computed from that length, in the case of base64-encoded data URLs. And of course you have to subtract out the length of the parts before ','. None of this is all that hard to do. -Boris
Re: [XHR] Content-Length header for data: URLs
Well, it's not rocket-science for sure but we do need to parse the part before the ,. We need to check the encoding, we need to make sure we know how to determine the actual length for this encoding, we need a way to not store length if we dunno the encoding. Simple enough but has some ramifications. What I'm seeing here is that being able to have a Content-Length or not seems implementation dependent. Anne is against not taking a stand yet I'm pretty sure we cannot assume all implementations will be able to provide a Content-Length. So it seems like we should just keep prohibiting Content-Length. Do I make sense? On 20 September 2013 05:24, Jonas Sicking jo...@sicking.cc wrote: On Thu, Sep 19, 2013 at 8:10 PM, Julian Aubourg j...@ubourg.net wrote: Sure, what I actually meant is that you'd need to somehow pre-parse the data URL to extract the exact length before storage. Dunno how desirable/desired/common this is. I would hardly call taking the length subtracting any characters before the , and applying a multiplier parsing. You don't have to look at any characters after the , at all. / Jonas On 20 September 2013 04:58, Boris Zbarsky bzbar...@mit.edu wrote: On 9/19/13 10:51 PM, Julian Aubourg wrote: But Content-Length is not the same as the length of the string containing the Data URL. Sure. It can also be a simple formula computed from that length, in the case of base64-encoded data URLs. And of course you have to subtract out the length of the parts before ','. None of this is all that hard to do. -Boris
Re: [XHR] Content-Length header for data: URLs
It's a malformed data URI for now. What I'm wondering is if we're sure we'll never have an encoding that makes it impossible to determine the length without decoding the entire content (think url-encoded like). I do agree having the Content-Length is useful information, I'm just trying to make sure we're future-proof in our approach. My opinion is that a Content-Length should be provided but I'm not sure it must. And again, since Anne doesn't seem to favour should... well, we're back at must not. On 20 September 2013 05:55, Boris Zbarsky bzbar...@mit.edu wrote: On 9/19/13 11:39 PM, Julian Aubourg wrote: We need to check the encoding You mean the base64 or lack thereof? we need to make sure we know how to determine the actual length for this encoding For base64 you do. Otherwise, it's a malformed data URI. we need a way to not store length if we dunno the encoding In what situation would this ever happen? What I'm seeing here is that being able to have a Content-Length or not seems implementation dependent. No more so than having a Content-Type, as far as I can see. I'm pretty sure we cannot assume all implementations will be able to provide a Content-Length. I'm pretty sure we can in fact assume just that. So it seems like we should just keep prohibiting Content-Length. If it were useless to provide it, I might agree. But it's useful. This is why blob URIs go out of their way to provide it... -Boris
Re: [XHR] test nitpicks: MIME type / charset requirements
Aren't both text/html;charset=windows-1252 and text/html; charset=windows-1252 valid MIME types? Should we make the tests a bit more accepting? Reading http://www.w3.org/Protocols/rfc1341/4_Content-Type.html it's not crystal clear if spaces are accepted, although white spaces and space are clearly cited in the grammar as forbidden in tokens. My understanding is that the intent is for white spaces to be ignored but I could be wrong. Truth is the spec could use some consistency and precision. test script sets charset=utf-8 and charset=UTF-8 on the wire is considered a failure Those tests must ignore case. The type, subtype, and parameter names are not case sensitive. On 6 May 2013 18:31, Hallvord Reiar Michaelsen Steen hallv...@opera.comwrote: Two of the tests in http://w3c-test.org/web-platform-tests/master/XMLHttpRequest/send-content-type-string.htmfails in Firefox just because there is a space before the word charset. Aren't both text/html;charset=windows-1252 and text/html; charset=windows-1252 valid MIME types? Should we make the tests a bit more accepting? Also, there's a test in http://w3c-test.org/web-platform-tests/master/XMLHttpRequest/send-content-type-charset.htmthat fails in Chrome because it asserts charset must be lower case, i.e. test script sets charset=utf-8 and charset=UTF-8 on the wire is considered a failure. Does that make sense? -- Hallvord R. M. Steen Core tester, Opera Software
Re: [XHR] test nitpicks: MIME type / charset requirements
Hey Anne, I don't quite get why you're saying HTTP is irrelevant. As an example, regarding the content-type *request *header, the XHR spec clearly states: If a Content-Type header is in author request headershttp://www.w3.org/TR/XMLHttpRequest/#author-request-headers and its value is a valid MIME typehttp://dev.w3.org/html5/spec/infrastructure.html#valid-mime-type that has a charset parameter whose value is not a case-insensitive match for encoding, and encoding is not null, set all thecharset parameters of that Content-Type header to encoding. So, at least, the encoding in the request content-type is clearly stated as being case-insensitive. BTW, Valid MIME type leads to (HTML 5.1): A string is a valid MIME type if it matches the media-type rule defined in section 3.7 Media Types of RFC 2616. In particular, a valid MIME typehttp://www.w3.org/html/wg/drafts/html/master/infrastructure.html#valid-mime-type may include MIME type parameters. [HTTP]http://www.w3.org/html/wg/drafts/html/master/iana.html#refsHTTP Of course, nothing is explicitely specified regarding the *response *content-type, because it is implicitely covered by HTTP (seeing as the value is generated outside of the client -- except when using overrideMimeType). It's usage as defined by the XHR spec is irrelevant to the fact it is to be considered case-insensitively : any software or hardware along the network path is perfectly entitled to change the case of the Content-Type header because HTTP clearly states case does not matter. So, testing for a response Content-Type case-sensitively is *not *correct. Things are less clear to me when it comes to white spaces. I find HTTP quite evasive on the matter. Please, correct me if I'm wrong and feel free to point me to the exact sentences in the XHR spec that calls for an exception regarding case-insensitivity of MIME types (as defined in HTTP which XHR references through HTML 5.1). I may very well have missed those. Cheers, -- Julian On 6 May 2013 19:22, Anne van Kesteren ann...@annevk.nl wrote: On Mon, May 6, 2013 at 9:31 AM, Hallvord Reiar Michaelsen Steen hallv...@opera.com wrote: ... The reason the tests test that is because the specification requires exactly that. If you want to change the tests, you'd first have to change the specification. (What HTTP says on the matter is not relevant.) -- http://annevankesteren.nl/
Re: [XHR] test nitpicks: MIME type / charset requirements
You made the whole thing a lot clearer to me, thank you :) It seems strange the spec would require a case-sensitive value for Content-Type in certain circumstances. Are these deviations from the case-insensitiveness of the header really necessary ? Are they beneficial for authors ? It seems to me they promote bad practice (case-sensitive testing of Content-Type). On 7 May 2013 01:20, Anne van Kesteren ann...@annevk.nl wrote: On Mon, May 6, 2013 at 3:44 PM, Julian Aubourg j...@ubourg.net wrote: I don't quite get why you're saying HTTP is irrelevant. For the requirements where the XMLHttpRequest says to put a certain byte string as a value of a header, that's what the implementation has to do, and nothing else. We could make the XMLHttpRequest talk about the value in a more abstract manner rather than any particular serialization and leave the serialization undefined, but it's not clear we should do that. As an example, regarding the content-type request header, the XHR spec clearly states: If a Content-Type header is in author request headers and its value is a valid MIME type that has a charset parameter whose value is not a case-insensitive match for encoding, and encoding is not null, set all the charset parameters of that Content-Type header to encoding. Yeah, this part needs to be updated at some point to actually state what should happen in terms of parsing and such, but for now it's clear enough. So, testing for a response Content-Type case-sensitively is not correct. It is if the specification requires a specific byte string as value. Things are less clear to me when it comes to white spaces. I find HTTP quite evasive on the matter. You can have a space there, but not per the requirements in XMLHttpRequest. -- http://annevankesteren.nl/
Re: [XHR] test nitpicks: MIME type / charset requirements
I hear you, but isn't having a case-sensitive value of Content-Type *in certain circumstances* triggering the kind of problem you're talking about (developers to depend on certain things they really should not depend on) ? As I see it, the tests in question here are doing something that is wrong in the general use-case from an author's POW. By requiring the same from every *implementor*, aren't we pushing *authors *in the trap you describe. Case in point : the author of the test is testing Content-Type case-sensitively while it is improper (from an author POW) in any other circumstance. The same code will fail if, say, the server sets a Content-Type. Shouldn't we protect authors from such inconsistencies ? On 7 May 2013 01:39, Anne van Kesteren ann...@annevk.nl wrote: On Mon, May 6, 2013 at 4:33 PM, Julian Aubourg j...@ubourg.net wrote: It seems strange the spec would require a case-sensitive value for Content-Type in certain circumstances. Are these deviations from the case-insensitiveness of the header really necessary ? Are they beneficial for authors ? It seems to me they promote bad practice (case-sensitive testing of Content-Type). There's only two things that seem to work well over a long period of time given multiple implementations and developers coding toward the dominant implementation (this describes the web). 1. Require the same from everyone. 2. Require randomness. Anything else is likely to lead some subset of developers to depend on certain things they really should not depend on and will force everyone to match the conventions of what they depend on (if you're in bad luck you'll get mutual exclusive dependencies; the web has those too). E.g. the ordering of the members of the canvas element is one such thing (trivial bad luck example is User-Agent). -- http://annevankesteren.nl/
Re: Reviewing test submissions
Hi Robin and sorry if this has been asked and/or answered elsewhere (I've been ultra-busy elsewhere as of late), I suppose that there is a hook in place on the github repo so that manifests are auto-magically re-generated whenever a new test is added to a main directory? Or is this still manual? Thanks for the hard work btw, seeing tests in github is very satisfactory and appreciated, -- Julian On 4 April 2013 16:23, Robin Berjon ro...@w3.org wrote: Hi all, after having moved the test suite to the GitHub repository, we've been busy cleaning things up. One item of particular importance is that everything that was in submission directories is now in pull requests (the very last few of those are right now being finalised). The changes being in pull requests makes it very easy for people to review the tests. You can just poke at the code, and if it looks good press a big green button. It's fun, it makes your hair sheen and your fresh bright, and you get to show off at cocktail parties about how you just made a major contribution to building a better web for all of humankind and its descendants to come. All of that with just the press of a big and friendly green button! (And a little bit of code reading.) So if you're ready for the undying adoration of wild throngs of web developers, if you've got the unassumingly humble tone of your I was just doing my job line, and if in you feel a hankering to press glossy green buttons with a motion of such lissom yet muscular grace that bystanders feel like it's happening in slow motion, then waste no time and head straight for: https://github.com/w3c/web-**platform-tests/pullshttps://github.com/w3c/web-platform-tests/pulls Let's party! A big thanks to Ms2ger and Odin for helping a lot with cleaning up the moved repo! -- Robin Berjon - http://berjon.com/ - @robinberjon
Re: [XHR] remove user cancels request
can i ask, how do you use the abort event handler? and error event handler In jQuery 1.x, we don't even use onsuccess, onerror and onabort. Reason being onreadystatechange is the only cross-browser means to handle XMLHttpRequest when you have to support old IEs (and we try and avoid having multiple code paths in jQuery because of size, maintainability and behavioural consistency). In jQuery 2.x, we use onload and onerror, not onabort. We have some short-circuit logic in $.ajax to notify author's initiated aborts as errors with a cancelled status but we ignore all other sources of aborts. The idea is to ignore onunload aborts so that jQuery devs don't have to special case for it in their error handlers. Now, to get back to the subject at hand, like Glenn said, it's probably better to notify those not-author-initiated aborts as errors, because from the author's POW they're hardly aborts. However, if we do so, we have to make sure it's possible to differentiate them from a network error. jQuery, as an author, can then choose to ignore them or bubble them up in $.ajax, not sure what's best for us at this point but that's another story entirely.
Re: [XHR] remove user cancels request
I have the same questions as Jungkee. What is it you want to remove exactly? Why do you think the distinction between an user-initiated abort and a network error is irrelevant? If I am to believe jQuery's bug tracker, our users want and need the distinction. On 25 February 2013 07:49, Jungkee Song jungkee.s...@samsung.com wrote: From: Timmy Willison [mailto:timmywill...@gmail.com] Sent: Monday, February 25, 2013 2:55 AM On Feb 24, 2013, at 11:18 AM, Glenn Maynard gl...@zewt.org wrote: On Sun, Feb 24, 2013 at 8:18 AM, Anne van Kesteren ann...@annevk.nl wrote: Currently the XMLHttpRequest Standard special cases the condition where the end user terminates the request. Given that there's less and less likely to be UI for that kind of feature, does it still make sense to expose this distinction from a network error in the API? I think we should merge them. http://xhr.spec.whatwg.org/ I didn't even know about that behavior. I've always assumed that the only way onabort happens is as a result of my calling abort(). I don't think breaking that assumption would break my code, but it's a rare, untested code path. I doubt other developers test it either. I agree that users killing a network request should look like a network error, and in general the API should guarantee that onabort is only fired as a result of a call to abort(). According to the current spec, it is already the case that onabort() is called only when client.abort() is explicitly called (including CORS requests.) onerror() is getting called in actual network errors such as DNS error, TLS negotiation failure, cross-origin access violation, etc. I am not sure what conditions Anne exactly propose to remove from the spec. I can think of only three scenarios where the end user *terminates* the request: calling open(), calling abort() or explicitly stop in browser chrome. I don't think client.open() and explicit browser stop are what Anne is talking about. Anne, could you elaborate what part of the text are you pointing? If it's the case that you want to merge abort into error, I tend to disagree as there can be use cases explicitly putting cancel button in UI that should be distinguished from network initiated errors. Jungkee +1 -- Glenn Maynard - Timmy
Re: [XHR] remove user cancels request
AFAIK, clicking the stop button of the navigator or clicking on a link in the page will abort outbound requests. That's exactly the kind of aborts authors want to differentiate from network errors. I assume those buttons are UI features that permit request cancellation for users? Or am I completly missing the point again? On 25 February 2013 10:08, Anne van Kesteren ann...@annevk.nl wrote: On Mon, Feb 25, 2013 at 8:20 AM, Julian Aubourg j...@ubourg.net wrote: I have the same questions as Jungkee. What is it you want to remove exactly? Why do you think the distinction between an user-initiated abort and a network error is irrelevant? If I am to believe jQuery's bug tracker, our users want and need the distinction. Euhm. My email said end user (which the spec simply calls users, jQuery's users are called authors/developers). If you search for This is an abort error you'll find the bits I think can be removed due to lack of browser UI for that feature. -- http://annevankesteren.nl/
Re: [XHR] remove user cancels request
I agree with Glenn that these user cancellations would be better notified
as errors rather than aborts. Key here is to make it possible for authors
to know what's going on and onabort seems quite confusing.
Side note: IE not cancelling requests is a real pain, we have to abort
manually on unload in jQuery because, in old IEs when using the
ActiveX-based XHRs those pending requests may end up exhausting the pool of
available XHRs!
Le lundi 25 février 2013, Glenn Maynard a écrit :
On Mon, Feb 25, 2013 at 3:37 AM, Anne van Kesteren
ann...@annevk.nljavascript:_e({}, 'cvml', 'ann...@annevk.nl');
wrote:
Sure, for links (i.e. navigation)... For XMLHttpRequest (fetching)
however those rarely change as that would make applications very
confusing to the user. At least, when I still worked at Opera at some
point the progress bars for fetching after the page had loaded (body
load fired) were disabled as they were just too confusing.
Some data points:
- Firefox already sends onerror if you hit escape while an XHR is in
progress. The onerror handler can just start another XHR request, and
pressing escape again will once again onerror that new request.
- In WebKit, if the document load is still in progress, escape cancels
both the document load (eg. images) and any running XHRs. If this happens,
onabort is sent to the request. However, this only happens during initial
document load; after that, XHR requests can't be cancelled. The onabort
handler can also start another XHR, and since the document load is no
longer in progress, this one can't be cancelled.
- IE9 doesn't seem to abort XHR at all.
WebKit's behavior doesn't make sense.
Firefox's behavior seems to match what you're suggesting, and it's what
I'd expect. Since it doesn't expose any code paths that can't happen
otherwise, it should also result in the least chance of bugs.
IE9's behavior of not cancelling XHR on escape at all seems reasonable,
too.
For navigation,
- Firefox also fires onerror if you navigate while an XHR is in progress.
- WebKit fires onabort.
- IE9 doesn't seem to cancel XHR on navigation. (You can get an XHR
onload after the user has already clicked a link, if the new page hasn't
already begun to load.)
If authors really want to be able to distinguish a user cancellation from
a regular network error, let's stick a flag on onerror. It's much worse to
have to carefully distinguish abort() from a user abort. They're
completely different things.
--
Glenn Maynard
Re: Beacon API
I'd personally be in favour of an optional parameter that would ask the browser to keep on with the request even after the page has been unloaded (which would be the only solution not to block unloading while ensuring data is delivered even for asynchronous requests). I'm not sure how feasibly this would be for implementors (we're talking about a dangling connection outside of the lifecycle of the page). There is also the question of knowing what happens when the browser is closed, not to mention security concerns that will undoubtedly arise. I also have no idea regarding the eventual name of this property. It's difficult to convey the intent with a simple word. Anyway, I agree this is a valid concern. It always bugged me. I too feel like it makes sense to try and address this within the XHR spec. On 13 February 2013 17:03, Jatinder Mann jm...@microsoft.com wrote: The Web Performance working group has been tracking a known poor performance pattern involving XHRs. ** ** We have seen cases where analytics code will block the unloading of the document in order to send data. To guarantee that the data is sent to their servers, analytics will typically register a handler on the unload event,which will make a synchronous XHR call to submit the data. The synchronous XHR forces the browser to delay unloading the document, and makes the next navigation appear to be slower. There is little the next page can do to avoid this perception of poor page load performance. ** ** Frankly, analytics don’t have many good options. Browsers will typically just ignore asynchronous XHR in an unload handler. Sending the data too soon may mean that they miss out on some data gathering opportunities. To solve this problem, the Web Performance WG has included writing a Beacon API in its charter [1]. This API would be an interoperable means for site developers to asynchronously transfer data from the user agent to a web server, with a guarantee from the user agent that the data will be eventually sent. ** ** However, instead of inventing a new way to send data, it may make sense to first explore whether we can update XHR to help in this scenario. This change could be as simple as adding an optional parameter to XHR, a new type of XHR (e.g., BeaconXHLHttpRequest), or just normative text on the expected user agent behavior when a synchronous XHR call is made in the unload event handler. ** ** How interested is this working group in taking on such work in the XHR Level 2 [2] specification? ** ** Thanks, Jatinder ** ** [1] http://www.w3.org/wiki/Web_Performance/Charter#Beacon [2] http://www.w3.org/TR/XMLHttpRequest2/ ** **
Re: Proposal: moving tests to GitHub
I love the idea of moving to github. The one-repo idea, while much simpler from a maintenance point of view, could easily be a burden on users that subscribe to it. Even more so for people who can merge PRs (and thus will receive an email for a PR initiatedfor any spec). Not saying it is blocking but it's something to keep in mind. Mail filters can go a long way here but filtering out specific specs kinda defeats the purpose of having so many eyes looking at everything. Le mardi 22 janvier 2013, Robin Berjon a écrit : On 22/01/2013 17:14 , Tobie Langel wrote: On 1/22/13 4:45 PM, Robin Berjon ro...@w3.org wrote: You *do* need to make the proper commitments for the test suite, but those are much lighter and can easily be extended to all. Moving to GitHub should be an excellent occasion to revisit how the CLA works and provide better integration, e.g.: by using something like CLAHub[1]. FYI we're looking at CLAHub as a possible solution for this (either directly or with a few modifications to tie it into our systems). No promises but it's on the table. That's why we're proposing to ditch per-WG anything here. The way html-testsuite is set up, we already have subdirectories for html, canvas2d, and microdata. Those are all from the HTML WG, but they're just listed as the individual specs. We can keep on adding more specs in there (the Web Crypto people are looking to do that). That sounds good to me. It's the per WG siloing I'm opposed to, not the one repository to rule them all idea. Good! Well, it looks like everyone agrees... If we're forging ahead, I have admin rights to the repo so you know who to prod. -- Robin Berjon - http://berjon.com/ - @robinberjon
Re: CfC: publish WD of XHR; deadline November 29
Hi all, In an ideal world, Anne would be the editor of the W3C version of the spec and that would be the end of it. Such is not the case. Anne is not the editor of the W3C version: he doesn't edit and/or publish anything related to the W3C XHR spec. Current editors do and while it's mostly brain-dead copy/paste, some decisions (especially regarding spec merging) are to be made W3C-side. Current editors also act as first-level reviewers and actually give Anne feedback. To be honest, I hate this situation. As far as I'm concerned, Anne *is* the author of the XHR spec but, AFAIK, there is no standardized way to acknowledge this in W3C documents nor does the WHATWG Licensing makes it mandatory. As a side note, as an open source developper, I can understand why. If the specs are on public repos and accept pull requests (or diffs, or whatever), then the very notion of authorship becomes a bit blurry. Anyway, I'm one of the co-editor of the W3C XHR spec and I don't claim to be the author of anything in the spec. I'm more interested in pushing the spec forward than achieving glory. I accepted the co-editor position to help because help was needed. So while I empathize with the whole W3C plagiarizes WHATWG outrage, could this conversation be held where it belongs? That is far upper the food chain than this WG. Now, that being said and seeing as we cannot put Anne as an editor of the W3C version of the spec (because, technically, he's not). How do you guys suggest we go about acknowledging the WHATWG source? Where in the spec? How? With what kind of wording? -- Julian Aubourg On 23 November 2012 17:36, Adam Barth w...@adambarth.com wrote: On Fri, Nov 23, 2012 at 7:57 AM, Glenn Adams gl...@skynav.com wrote: On Fri, Nov 23, 2012 at 12:09 AM, Adam Barth w...@adambarth.com wrote: On Thu, Nov 22, 2012 at 9:16 AM, Ms2ger ms2...@gmail.com wrote: On 11/22/2012 02:01 PM, Arthur Barstow wrote: TheXHR Editors would like to publish a new WD of XHR and this is a Call for Consensus to do so using the following ED (not yet using the WD template) as the basis http://dvcs.w3.org/hg/xhr/raw-file/tip/Overview.html. Agreement to this proposal: a) indicates support for publishing a new WD; and b) does not necessarily indicate support of the contents of the WD. If you have any comments or concerns about this proposal, please reply to this e-mail by December 29 at the latest. Positive response to this CfC is preferred and encouraged and silence will be assumed to mean agreement with the proposal. I object unless the draft contains a clear pointer to the canonical spec on whatwg.org. I agree. The W3C should not be in the business of plagiarizing the work of others. Are you claiming that the W3C is in the business of plagiarizing? I'm saying that the W3C (and this working group in particular) is taking Anne's work, without his permission, and passing it off as its own. That is plagiarism, and we should not do it. plagiarism. n. The practice of taking someone else's work or ideas and passing them off as one's own. The Status of this Document section should state clearly that this document is not an original work of authorship of the W3C. The SotD section need only refer to the working group that produced the document. Authorship is not noted or tracked in W3C documents. If Anne's work was submitted to and prepared in the context of the WebApps WG, then it is a product of the WG, and there is no obligation to refer to other, prior or variant versions. Referring to an earlier, draft version published outside of the W3C process does not serve any purpose nor is it required by the W3C Process. Legally, we are under no obligation to acknowledge Anne's work. However, we should be honest about the origin of the text and not try to pass off Anne's work as our own. More pointedly: plagiarism is not illegal but that doesn't mean we should do it. If there is a question on the status of the Copyright declaration of the material or its origin, then that should be taken up by the W3C Pubs team. My concern is not about copyright. My concern is about passing off Anne's work as our own. Adam
Re: [XHR] Open issue: allow setting User-Agent?
I tend to agree with Boris on this one.
Couldn't we simply state in the spec that browsers must add the User-Agent
header to the Vary list, all the time? That would instantly solve the
attack-from-the-cache problem, right? No need to sanitize the data, no need
to negotiate anything between both ends.
If that's the only security threat, it actually seems quite simple to
disable.
I'm still more concerned about potentially legitimate use cases of
User-Agent filtering that could lead to security breaches when removing
User-Agent from the non-modifiable list. But if no-one else feels like
there could ever be such a legitimate use-case, then I don't think we
should hold back because of this out-of-cache XSS attack: let's just
specify User-Agent has to be in Vary all the time. It's not like it will
break caching in the general case anyway.
Le mardi 16 octobre 2012, Boris Zbarsky a écrit :
On 10/16/12 8:44 AM, Hallvord Reiar Michaelsen Steen wrote:
xhr=new XMLHttpRequest();
xhr.setRequestHeader('User-**Agent', '--script src=
http://attacker.com/**malice.js http://attacker.com/malice.js
/script!--');
For what it's worth, I would have no problem sanitizing the header value
for User-Agent if we allow script to set it, or throwing on certain values;
I can't think of any legit use cases for values containing '', since no
actual UAs do that.
and then requests /publicdata/index.htm . Once the request reaches
readyState=4, attacker's page does
location.href='http://www.**victim.com/publicdata/index.**htmhttp://www.victim.com/publicdata/index.htm
';
The browser has a fresh copy - loaded with the xhr request just
milliseconds ago - and fetches the page from cache. Voila, the browser is
now rendering a victim.com page with a link to malice.js included and
the attacker can do whatever from the JS now running in the scope of the
site.
Again, Vary: User-Agent is the answer here, from the browser's point of
view. I agree that this would be good to discuss in a security
implications section. The spec could even require that responses to XHR
with custom UA simply not be cached, if we want to play it safe.
so the threat scenario relies on the remote server being stupid enough to
do that and yet be careless about echoing non-sanitised User-Agent strings
back to the page. Which is basically the negotiation scenario Julian
earlier said he would agree to, and the reason I'm still pondering if it
would be worth the extra complexity to allow it for cross-domain requests
only...but this sort of cache poisoning scenario is a concern..
Right, I agree it's a concern. I think it's an easy one to address for
this narrow use case.
-Boris
Re: [XHR] Open issue: allow setting User-Agent?
The point is that a browser can act as if every single server response included Vary: User-Agent. And perhaps should. Intermediary caches _certainly_ should. Yes, that was my point. Do as if User-Agent was part of the Vary response header.
A little introduction
Hi all, My name is Julian Aubourg and I'm one of the new co-editors of the XHR spec (together with Jungkee Song from Samsung and Hallvord R. M. Steen from Opera). I'm a member of jQuery Core and rewrote the lib's ajax module and implemented $.Deferred (and now $.Callbacks). I like everything async, really ;) As for making a living, I co-founded my own company Creative-Area ( http://www.creative-area.net) together with yet again two people: Florent Bourgeois and Xavier Baldacci. We make websites/webapps, from both ends, which means I write far too much PHP and SQL for my own good. -- Julian
Fwd: [XHR] Open issue: allow setting User-Agent?
Sorry, I've been cut by keyboard short cuts :P ... so the burden of proof is on *you*. *You* have to establish the consequences of making a backward incompatible change. Not brush away arguments pro, or cons, to advance your agenda. Did you ask backend devs why they white-listed browsers? Did you try and educate them? Did you ever encounter any sensible use-case for this? Do you really want to break a lot of backends expectations because you don't see the reason? You have to be very careful with breaking backward compatibility. Just look the jQuery's bug tracker for a prime example of what happens when you do. We don't have to prove it is useful. We just have to prove it is used and *you* brought this up yourself. Now you want to bypass this by pretty much hacking client-side. Please make a compelling case for it. I still don't fully understand the scenario(s) you have in mind. You're confusing the script's origin with the site's origin. XHR requests from within a script are issued with the origin of the page that the script is included into. Now, read back your example but suppose the attack is to be pulled against cnn.com. At a given time (say cnn.com's peek usage time), the script issues a gazillions requests. Bye-bye server. That's why I took the ad example. Hack a single point of failure (the ad server, a CDN) and you can DOS a site using the resource from network points all over the net. While the frontend dev is free to use scripts hosted on third-parties, the backend dev is free to add a (silly but effective) means to limit the number of requests accepted from a browser. Simple problem, simple solution and the spec makes it possible. Note that this use-case has nothing to do with filtering out a specific browser btw. Yet you would break this with the change you propose. Maybe it's not the best of examples. But I came up with this in something like 5 minutes. I can't imagine there are no other ways to abuse this. This is a way more interesting (ab)use case. You're presuming that there are web-exposed backend services that are configured to only talk to other backend servers, and use a particular magic token in User-Agent as authentication? If such services exist, does being able to send a server-like UA from a web browser make them significantly more vulnerable than being able to send the same string from a shell script? Same as above: single point of failure. You hack into a server delivering a shared resource and you have as many unwilling agents participating into your attack. So far I see that only Jaredd seems to like the idea (in this thread anyway): I agree with Hallvord, I cannot think of any additional *real* security risk involved with setting the User-Agent header. Particularly in a CORS situation, the server-side will (should) already be authenticating the origin and request headers accordingly. If there truly is a compelling case for a server to only serve to Browser XYZ that is within scope of the open web platform, I'd really like to hear tha By that line of reasoning, I don't see why we need preflight in CORS and specific authorisation from the server-side for content to be delivered cross-domain. It is not *open*. After all since any backend could request the resource without problem, why should browsers be limited? But then again, the problem has nothing to do with CORS but with third-party scripts that effectively steal the origin of the page that includes them and the single point of failure problem that arises. That's why JavaScript is as sandboxed as it is. In all honesty, I'd love to be convinced that the change is without consequences, but the more I think about it, the less likely it seems. -- Forwarded message -- From: Julian Aubourg j...@ubourg.net Date: 11 October 2012 14:47 Subject: Re: [XHR] Open issue: allow setting User-Agent? To: Hallvord R. M. Steen hallv...@opera.com We end up in a philosophical disagreement here :-) I'd say that whatever browser the user decides to use is the user's choice and the server should respect that. I'm sorry but that's complete non-sense. The backend is the provider of the data and has all the right when it comes to its distribution. If it's a mistake on the backend's side (they filter out while they didn't intend to) just contact the backend's maintainer and have them fix this server-side problem... well... server-side. You're trying to circumvent a faulty implementation server-side by breaking a client-side related spec backward compatibility. If you can't see how wrong the whole idea is, I'm afraid you didn't have to suffer the consequences of such drastic changes in the past (I had to with script tag injection and it was a just a pure client-side issue, nothing close to what you're suggesting in term of repercussions). One word: legacy. For example Amazon.com might want to enable CORS for some of its content. The team that will do that won't necessarily have any intention
Re: [XHR] Open issue: allow setting User-Agent?
I personally have contacted hundreds of sites for these types of issues over the past few years. We've done the education, outreach, evangelism, etc. Success rates are very low, the majority are simply ignored. I'm sorry to hear that. I really am. Still trying to have people stop browser sniffing client-side. :( I'm sorry but that's complete non-sense. The backend is the provider of the data and has all the right when it comes to its distribution. If it's a mistake on the backend's side (they filter out while they didn't intend to) just contact the backend's maintainer and have them fix this server-side problem... well... server-side. This isn't feasible. There's a whole web out there filled with legacy content that relies on finding the string Mozilla or Netscape, for example. See also the requirements for navigator.appName, navigator.appVersion, document.all, etc. You can't even get close to cleaning up the mess of legacy code out there, so you work around it. And history repeats itself today with magical strings like Webkit and Chrome. What of new browsers, how do they deal with this legacy content? The same way that current ones do, most likely -- by pretending to be something else. The problem is that the same reasoning can be made regarding CORS. We have backends, today, that do not support it. I'm not convinced they actually want to prevent Cross-Domain requests that come from the browser. Truth is it depends on the backend. So why do we require server opt-in when it comes to CORS? After all, it is just a limitation in the browser itself. Surely there shouldn't be any issue given these URLs are already fetchable from a browser provided the page origin is the same. You can even fetch them using another backend or shell or whatever other means. Problem is backends expect this limitation to be true. So very few actually control anything because browsers on a page from another origin are never supposed to request the backend. There is potential for abuse here. Solution was to add an opt-in system. For backends that are not maintained, behaviour is unchanged. Those that want to support CORS have to say so explicitely. If we had a mechanism to do the same thing for the fact of modifying the UserAgent header, I wouldn't even discuss the issue. Target URL authorizes UserAgent to be changed, browser accepts custom UserAgent, sends the request and filtering that happened between the URL and the browser would be bypassed (solving the problem Hallvord gave with devs working on a part of a site and having to deal with some filtering above their heads). Could work pretty much exactly like CORS custom headers are handled. Hell, it could even be made generic and could potentially solve other issues. What's proposed here is entirely different though: it's an all or nothing approach. Now I'm just trying to see if there is no potential danger here. aside The burden of proof is on you. *You* ha Emphasis with asterisks seems unnecessary aggressive. Perhaps unintentionally so. :) /aside Sorry about that, not my intention at all. I'd love to be convinced and I'd just love it if Hallvord (or anyone really) could actually pull it off. So it's positive excitement, not negative one. I hope my answer above will make my reasonning a bit clearer (just realized it wasn't quite clear before).
Re: [XHR] Open issue: allow setting User-Agent?
Are you really saying that backend developers want to use User-Agent to limit the number of requests accepted from Firefox? (Not one user's Firefox, but all Firefox users, at least of a particular version, combined.) That doesn't make sense at all. If that's not what you mean, then please clarify, because I don't know any other way the User-Agent header could be used to limit requests. A more likely scenario is a URL that only accepts a specific user agent that is not a browser (backend). If user script can change the UserAgent, it can request this URL repeatedly. Given it's in the browser, a shared resource (like an ad provider or a CDN) becomes a very tempting point of failure. AFAIK, you don't have the same problem with PHP libs for instance (you don't request same from a third-party server, making it a potential vector of attack). I'm not saying it's smart (both from the hacker's POW or the backend POW) but I'm just being careful and trying to see if there is potential for abuse. On 11 October 2012 16:22, Glenn Maynard gl...@zewt.org wrote: On Thu, Oct 11, 2012 at 8:09 AM, Julian Aubourg j...@ubourg.net wrote: I still don't fully understand the scenario(s) you have in mind. You're confusing the script's origin with the site's origin. XHR requests from within a script are issued with the origin of the page that the script is included into. Now, read back your example but suppose the attack is to be pulled against cnn.com. At a given time (say cnn.com's peek usage time), the script issues a gazillions requests. Bye-bye server. I'm confused. What does this have to do with unblacklisting the User-Agent header? That's why I took the ad example. Hack a single point of failure (the ad server, a CDN) and you can DOS a site using the resource from network points all over the net. While the frontend dev is free to use scripts hosted on third-parties, the backend dev is free to add a (silly but effective) means to limit the number of requests accepted from a browser. Simple problem, simple solution and the spec makes it possible. Are you really saying that backend developers want to use User-Agent to limit the number of requests accepted from Firefox? (Not one user's Firefox, but all Firefox users, at least of a particular version, combined.) That doesn't make sense at all. If that's not what you mean, then please clarify, because I don't know any other way the User-Agent header could be used to limit requests. -- Glenn Maynard
Re: Shrinking existing libraries as a goal
To me the biggest abomination of all is the XMLHttpRequest object: - the spec is probably one of the most complex I've seen - yet, vast portions are left to interpretations or even not specified at all: - the local filesystem comes to mind, - also every browser has its own specific way of notifying non-applicative errors (like network errors): - specific status, - unhandleable asychronously thrown exception, - exception thrown when accessing a field, - etc... And that's just the tip of the iceberg. It's got to a point where the almighty xhr bleeds through abstractions and makes it impossible to design proper API (at least not if you want to leak memory like crazy). Finally, fixing xhr issues always seem like low priority items in browser bug trackers because there's always some kind of workaround, that libraries like jQuery have to put in their code (provided it can be feature tested which it cannot most of the time). I've been meaning to do a test suite to help provide guidance to implementors (something I figure would be much more useful than yet another round of specs) but I admit I haven't got to it yet. Dunno how people feel about this, but I think providing test suites that browsers could test against as a way to prevent regressions and inconsistencies could help a lot as a starting point. 2012/5/18 Yehuda Katz wyc...@gmail.com I am working on it. I was just getting some feedback on the general idea before I sunk a bunch of time in it. Keep an eye out :D Yehuda Katz (ph) 718.877.1325 On Thu, May 17, 2012 at 3:18 PM, Brian Kardell bkard...@gmail.com wrote: Has anyone compiled an more general and easy to reference list of the stuff jquery has to normalize across browsers new and old? For example, ready, event models in general, query selector differences, etc? On May 17, 2012 3:52 PM, Rick Waldron waldron.r...@gmail.com wrote: On Thu, May 17, 2012 at 3:21 PM, Brian Kardell bkard...@gmail.comwrote: On Thu, May 17, 2012 at 2:47 PM, Rick Waldron waldron.r...@gmail.com wrote: On Thu, May 17, 2012 at 2:35 PM, Brian Kardell bkard...@gmail.com wrote: So, out of curiosity - do you have a list of things? I'm wondering where some efforts fall in all of this - whether they are good or bad on this scale, etc... For example: querySelectorAll - it has a few significant differences from jQuery both in terms of what it will return (jquery uses getElementById in the case that someone does #, for example, but querySelectorAll doesn't do that if there are multiple instances of the same id in the tree) Which is an abomination for for developers to deal with, considering the ID attribute value must be unique amongst all the IDs in the element's home subtree[1] . qSA should've been spec'ed to enforce the definition of an ID by only returning the first match for an ID selector - devs would've learned quickly how that worked; since it doesn't and since getElementById is faster, jQuery must take on the additional code burden, via cover API, in order to make a reasonably usable DOM querying interface. jQuery says you're welcome. and performance (this example illustrates both - since jQuery is doing the simpler thing in all cases, it is actually able to be faster (though technically not correct) I'd argue that qSA, in its own contradictory specification, is not correct. It has been argued in the past - I'm taking no position here, just noting. For posterity (not you specifically, but for the benefit of those who don't follow so closely), the HTML link also references DOM Core, which has stated for some time that getElementById should return the _first_ element with that ID in the document (implying that there could be more than one) [a] and despite whatever CSS has said since day one (ids are unique in a doc) [b] a quick check in your favorite browser will show that CSS doesn't care, it will style all IDs that match. So basically - qSA matches CSS, which does kind of make sense to me... I'd love to see it corrected in CSS too (first element with that ID if there are more than one) but it has been argued that a lot of stuff (more than we'd like to admit) would break. in some very difficult ones. Previously, this was something that the browser APIs just didn't offer at all -- now they offer them, but jQuery has mitigation to do in order to use them effectively since they do not have parity. Yes, we're trying to reduce the amount of mitigation that is required of libraries to implement reasonable apis. This is a multi-view discussion: short and long term. So can someone name specific items? Would qSA / find been pretty high on that list? Is it better for jQuery (specifically) that we have them in their current state or worse? Just curious. TBH, the current state can't get any worse, though I'm sure it will.
