Re: [whatwg] Web Storage: apparent contradiction in spec
On Mon, 31 Aug 2009, Jens Alfke wrote: On Aug 31, 2009, at 3:11 AM, Ian Hickson wrote: We can't treat cookies and persistent storage differently, because otherwise we'll expose users to cookie resurrection attacks. Maintaining the user's expectations of privacy is critical. The fact that local storage can be used as a type of super-cookie doesn't mean the two are the same thing. The fact that local storage can be used for cookie resurrection means we have to make sure that clearing one clears the other. Anything else would be a huge privacy issue (just as Flash has been). That's just one of many reasons why user agents should require user approval for letting a domain access local storage. I disagree that they should, and more importantly, as shipped in multiple browsers, they don't, which likely means we can never make it such that UAs require user approval. That does not mean that the Delete Cookies menu command should also delete local storage. Users often delete cookies to resolve login issues (I've had to do this with Google websites several times). Conflating the two can lead to disasters like I told you to delete my COOKIES! Not my EMAIL DRAFTS that I was trying to log in to send! Certainly the UI issues around this are non-trivial, and one would hope that the UA will help the user handle these cases securely. But authors equally don't want to end up with the more insidious situation of companies tracking them without their knowledge even though they think they are safe due to clearing cookies. So I've removed the text that says that local storage could be user- critical. That's going to come as a shock to developers who were planning to use it for user-created data (whether drafts of content to be pushed to the cloud, or strictly-local documents.) Without this, the safe usage of local storage diminishes to a download cache. I don't see what else we can do. On Mon, 31 Aug 2009, Jeremy Orlow wrote: Yes, this is pretty disconcerting since there's been OVERWHELMING support for LocalStorage being treated as user-critical on this thread. It doesn't matter how much support a proposal gets, only technical merits matter here. On Mon, 31 Aug 2009, Peter Kasting wrote: It seems like you're convinced that UAs won't create UI users can understand, and so you're trying to make the spec mandate what you think will be comprehensible for users. IMO this is not only out-of-scope but pointless, as UAs are going to do what they want anyway. The spec is already pretty clear in telling UAs not to be casual about things, I don't think you're going to change what actually gets implemented by demanding more. Indeed. On Wed, 2 Sep 2009, Peter Kasting wrote: It still seems like you are interpreting this statement as saying that the UA must not allow users to keep/clear cookies separately from Local Storage data. While on the face of it that seems like a possible interpretation, I think it's clear that this would be a lousy user experience and detrimental to developers as well. Therefore I am convinced that the intent of the statement is to say that UAs must give users the same _abilities_ to see and clear Local Storage data as they already have with cookies, not that the two things should always be lumped together and made indistinguishable. Not necessarily indistuingushable, but the point is that the user should have a clear indication that just clearing cookies is pointless if the rest of the site's data isn't cleared also. I've tried to clarify the text a bit here. On Mon, 31 Aug 2009, Mike Shaver wrote: On Mon, Aug 31, 2009 at 6:11 AM, Ian Hicksoni...@hixie.ch wrote: We can't treat cookies and persistent storage differently, because otherwise we'll expose users to cookie resurrection attacks. Maintaining the user's expectations of privacy is critical. By that reasoning we can't treat cookies differently from the HTTP cache (ETag) or history (URIs with session IDs), I think. Indeed; that's an issue for the HTTP spec, though, and is out of scope for Web Storage and HTML5. I don't know of any UAs that expire history/cookie/cache in sync to avoid correlations -- if it's even possible to do so -- and I don't think I've seen any bugs asking Firefox to do so. It's a lot easier to do cookie resurrection with local storage than with the HTTP cache, but I imagine that if anyone ever gets around to doing it with the cache, this will suddenly surge in the public consciousness, so it might be something to deal with in advance anyway. On Tue, 1 Sep 2009, Linus Upson wrote: On Mon, Aug 31, 2009 at 3:11 AM, Ian Hickson i...@hixie.ch wrote: In addition, I'd like to see the pop-up dialogs for the location API removed. I find the Can I know where you are? dialogs on the iPhone very annoying. Mistakes were made. Perhaps we can find a way to make input type=location work well
Re: [whatwg] Web Storage: apparent contradiction in spec
On Thu, Sep 3, 2009 at 7:56 AM, Ian Hicksoni...@hixie.ch wrote: On Mon, 31 Aug 2009, Jens Alfke wrote: On Aug 31, 2009, at 3:11 AM, Ian Hickson wrote: We can't treat cookies and persistent storage differently, because otherwise we'll expose users to cookie resurrection attacks. Maintaining the user's expectations of privacy is critical. The fact that local storage can be used as a type of super-cookie doesn't mean the two are the same thing. The fact that local storage can be used for cookie resurrection means we have to make sure that clearing one clears the other. Anything else would be a huge privacy issue (just as Flash has been). And as Flash will continue to be, forever, in a manner that is generally opaque from the user, especially as more people lean on it for things like a halfway-dependable storage location. That's going to come as a shock to developers who were planning to use it for user-created data (whether drafts of content to be pushed to the cloud, or strictly-local documents.) Without this, the safe usage of local storage diminishes to a download cache. I don't see what else we can do. You could just *not* specify that LocalStorage is worthless for anything but a cache. Is there *anything* that would allow a permanent site-accessible storage solution in your mind, or is cookie resurrection a deal-killer for all time? If the latter, you're not doing anyone any favors, least of all users, as they'll still have their privacy violated but by entities other than their browser which may be more difficult to review and regulate. ~TJ
Re: [whatwg] Web Storage: apparent contradiction in spec
On Thu, Sep 3, 2009 at 8:56 AM, Ian Hicksoni...@hixie.ch wrote: The fact that local storage can be used for cookie resurrection means we have to make sure that clearing one clears the other. Anything else would be a huge privacy issue (just as Flash has been). The *only* reason Flash is a privacy issue is because there's no easy way for users to clear its storage. The issue here isn't the technical details of how the storage works, but the UI. Adobe, for whatever reason, has chosen not to bother with helping Flash users preserve their privacy, and because of lock-in, browser vendors are unable to do anything about it. All major browser vendors have a track record of going to great lengths to ensure that their users' privacy is protected from third-party websites. I think it's safe to say they'll compete to create good UI in this case -- even if technically, the functionality of HTML 5 localStorage is identical to that of Flash local storage. The spec doesn't need to try specifying UI here (especially since it seems like it will be ignored). Not necessarily indistuingushable, but the point is that the user should have a clear indication that just clearing cookies is pointless if the rest of the site's data isn't cleared also. Users might wish to clear their cookies for reasons other than privacy, such as because they're having login problems.
Re: [whatwg] Web Storage: apparent contradiction in spec
On Thu, 3 Sep 2009, Tab Atkins Jr. wrote: On Thu, Sep 3, 2009 at 7:56 AM, Ian Hicksoni...@hixie.ch wrote: On Mon, 31 Aug 2009, Jens Alfke wrote: On Aug 31, 2009, at 3:11 AM, Ian Hickson wrote: We can't treat cookies and persistent storage differently, because otherwise we'll expose users to cookie resurrection attacks. Maintaining the user's expectations of privacy is critical. The fact that local storage can be used as a type of super-cookie doesn't mean the two are the same thing. The fact that local storage can be used for cookie resurrection means we have to make sure that clearing one clears the other. Anything else would be a huge privacy issue (just as Flash has been). And as Flash will continue to be, forever, in a manner that is generally opaque from the user, especially as more people lean on it for things like a halfway-dependable storage location. On Thu, 3 Sep 2009, Aryeh Gregor wrote: The *only* reason Flash is a privacy issue is because there's no easy way for users to clear its storage. The issue here isn't the technical details of how the storage works, but the UI. Adobe, for whatever reason, has chosen not to bother with helping Flash users preserve their privacy, and because of lock-in, browser vendors are unable to do anything about it. All major browser vendors have a track record of going to great lengths to ensure that their users' privacy is protected from third-party websites. I think it's safe to say they'll compete to create good UI in this case -- even if technically, the functionality of HTML 5 localStorage is identical to that of Flash local storage. The spec doesn't need to try specifying UI here (especially since it seems like it will be ignored). Indeed. Flash's privacy problem can be removed by uninstalling Flash. They're not a license to add more privacy problems to the platform. On Thu, 3 Sep 2009, Tab Atkins Jr. wrote: On Thu, Sep 3, 2009 at 7:56 AM, Ian Hicksoni...@hixie.ch wrote: On Mon, 31 Aug 2009, Jens Alfke wrote: That's going to come as a shock to developers who were planning to use it for user-created data (whether drafts of content to be pushed to the cloud, or strictly-local documents.) Without this, the safe usage of local storage diminishes to a download cache. I don't see what else we can do. You could just *not* specify that LocalStorage is worthless for anything but a cache. Is there *anything* that would allow a permanent site-accessible storage solution in your mind, or is cookie resurrection a deal-killer for all time? The latter. If the latter, you're not doing anyone any favors, least of all users, as they'll still have their privacy violated but by entities other than their browser which may be more difficult to review and regulate. If Adobe wants to violate their users' privacy, that's their prerogative. But I'm not speccing something that so blatently allows users to be tracked without their consent -- and worse -- despite their attempts to stop it. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Web Storage: apparent contradiction in spec
On Thu, Sep 3, 2009 at 6:49 AM, Tab Atkins Jr. jackalm...@gmail.com wrote: You could just *not* specify that LocalStorage is worthless for anything but a cache. This seems like a severe overstatement given the current spec. It's not worthless. It won't be guaranteed to be thrown away all the time automatically like cookies are (due to browsers having what turn out to be low global and per-site limits). Users will have the ability to delete it, but it won't be silently lumped in with cookies. All the spec really says is that UAs should note to their users that sites can keep data about them in Local Storage. This isn't grounds for a tantrum. PK
Re: [whatwg] Web Storage: apparent contradiction in spec
On Thu, Sep 3, 2009 at 5:33 PM, Ian Hicksoni...@hixie.ch wrote: On Thu, 3 Sep 2009, Tab Atkins Jr. wrote: On Thu, Sep 3, 2009 at 7:56 AM, Ian Hicksoni...@hixie.ch wrote: On Mon, 31 Aug 2009, Jens Alfke wrote: On Aug 31, 2009, at 3:11 AM, Ian Hickson wrote: We can't treat cookies and persistent storage differently, because otherwise we'll expose users to cookie resurrection attacks. Maintaining the user's expectations of privacy is critical. The fact that local storage can be used as a type of super-cookie doesn't mean the two are the same thing. The fact that local storage can be used for cookie resurrection means we have to make sure that clearing one clears the other. Anything else would be a huge privacy issue (just as Flash has been). And as Flash will continue to be, forever, in a manner that is generally opaque from the user, especially as more people lean on it for things like a halfway-dependable storage location. On Thu, 3 Sep 2009, Aryeh Gregor wrote: The *only* reason Flash is a privacy issue is because there's no easy way for users to clear its storage. The issue here isn't the technical details of how the storage works, but the UI. Adobe, for whatever reason, has chosen not to bother with helping Flash users preserve their privacy, and because of lock-in, browser vendors are unable to do anything about it. All major browser vendors have a track record of going to great lengths to ensure that their users' privacy is protected from third-party websites. I think it's safe to say they'll compete to create good UI in this case -- even if technically, the functionality of HTML 5 localStorage is identical to that of Flash local storage. The spec doesn't need to try specifying UI here (especially since it seems like it will be ignored). Indeed. Flash's privacy problem can be removed by uninstalling Flash. They're not a license to add more privacy problems to the platform. And more-than-a-cache-Storage can be explicitly turned off or have its quota dropped to zero. If that's important, the browsers will make it easy. And more importantly, they'll make it *consistent* (within the browser), rather than the user having to figure out how to do it within Flash, then possibly within the next technology that hacks around this lack in browser technology, and the next one... On Thu, 3 Sep 2009, Tab Atkins Jr. wrote: On Thu, Sep 3, 2009 at 7:56 AM, Ian Hicksoni...@hixie.ch wrote: On Mon, 31 Aug 2009, Jens Alfke wrote: That's going to come as a shock to developers who were planning to use it for user-created data (whether drafts of content to be pushed to the cloud, or strictly-local documents.) Without this, the safe usage of local storage diminishes to a download cache. I don't see what else we can do. You could just *not* specify that LocalStorage is worthless for anything but a cache. Is there *anything* that would allow a permanent site-accessible storage solution in your mind, or is cookie resurrection a deal-killer for all time? The latter. If the latter, you're not doing anyone any favors, least of all users, as they'll still have their privacy violated but by entities other than their browser which may be more difficult to review and regulate. If Adobe wants to violate their users' privacy, that's their prerogative. But I'm not speccing something that so blatently allows users to be tracked without their consent -- and worse -- despite their attempts to stop it. It doesn't 'allow users to be tracked without their consent' any more than cookies by themselves do. If this is important, browsers will expose the ability to blow away all of a site's storages at once. There's nothing to resurrect then. On the other hand, if someone wants a site to keep its permanent Storage, then cookie resurrection isn't a big deal. You're seem to be assuming that either permanent Storage is *really* permanent, or that browsers will never expose a way to delete that data to the user (which amounts to the same). That's silly. The whole *point* of specifying a permanent Storage in HTML is so browsers can produce something that *they* control the UI for, rather than leaving the user's privacy to unknown plugins and other hacky means. Would an approach like the input type=save be okay, where the user has to explicitly take an action to enable permanent Storage for a site the first time? That makes simple navigation exactly as safe as it is today, where cookies can leak privacy but they can be wiped. You have to actively opt in to reducing your privacy. I *want* to be able to reduce my privacy for sites that I trust. ~TJ
Re: [whatwg] Web Storage: apparent contradiction in spec
On Thu, Sep 3, 2009 at 3:44 PM, Tab Atkins Jr. jackalm...@gmail.com wrote: And more-than-a-cache-Storage can be explicitly turned off or have its quota dropped to zero. If that's important, the browsers will make it easy. And more importantly, they'll make it *consistent* (within the browser), rather than the user having to figure out how to do it within Flash, then possibly within the next technology that hacks around this lack in browser technology, and the next one... As a UA author I see nothing in the spec that prevents this *now*. If this is important, browsers will expose the ability to blow away all of a site's storages at once. There's nothing to resurrect then. On the other hand, if someone wants a site to keep its permanent Storage, then cookie resurrection isn't a big deal. You're seem to be assuming that either permanent Storage is *really* permanent, or that browsers will never expose a way to delete that data to the user (which amounts to the same). That's silly. The whole *point* of specifying a permanent Storage in HTML is so browsers can produce something that *they* control the UI for, rather than leaving the user's privacy to unknown plugins and other hacky means. Again, this is precisely what we as UA authors can do now, with the current spec. I'm not sure what you're arguing. Our job is to make sure users whose philosophy is like Ian's are as well-served as users whose philosophy is like yours, and our hands are not tied. PK
Re: [whatwg] Web Storage: apparent contradiction in spec
On Thu, Sep 3, 2009 at 5:48 PM, Peter Kastingpkast...@google.com wrote: On Thu, Sep 3, 2009 at 3:44 PM, Tab Atkins Jr. jackalm...@gmail.com wrote: And more-than-a-cache-Storage can be explicitly turned off or have its quota dropped to zero. If that's important, the browsers will make it easy. And more importantly, they'll make it *consistent* (within the browser), rather than the user having to figure out how to do it within Flash, then possibly within the next technology that hacks around this lack in browser technology, and the next one... As a UA author I see nothing in the spec that prevents this *now*. Indeed, I don't either. I'm not sure that Ian's considering that, as an informed user who highly values his privacy, he would have such controls available to him. If this is important, browsers will expose the ability to blow away all of a site's storages at once. There's nothing to resurrect then. On the other hand, if someone wants a site to keep its permanent Storage, then cookie resurrection isn't a big deal. You're seem to be assuming that either permanent Storage is *really* permanent, or that browsers will never expose a way to delete that data to the user (which amounts to the same). That's silly. The whole *point* of specifying a permanent Storage in HTML is so browsers can produce something that *they* control the UI for, rather than leaving the user's privacy to unknown plugins and other hacky means. Again, this is precisely what we as UA authors can do now, with the current spec. I'm not sure what you're arguing. Our job is to make sure users whose philosophy is like Ian's are as well-served as users whose philosophy is like yours, and our hands are not tied. You may have missed the part where Ian said that, to protect their user's privacy, browsers *must* clear cookies and LocalStorage at the same time. This means that LocalStorage is exactly as ephemeral as cookies, and is far too easy to blow away for authors to rely on. I regularly have semi-technical users of my intranet apps precede bug reports with I cleared my cookies, but that didn't do anything I store next to nothing in cookies, so this is never useful, but people have still internalized that action as a troubleshooting step when a website is broken. People also clear cookies globally fairly often (you have to dig down in FF to delete cookies on a site-specific basis). This *will* make people unintentionally blow away information they want to keep for sites that are unrelated to the one they're having trouble with. ~TJ
Re: [whatwg] Web Storage: apparent contradiction in spec
On Thu, Sep 3, 2009 at 3:55 PM, Tab Atkins Jr. jackalm...@gmail.com wrote: Again, this is precisely what we as UA authors can do now, with the current spec. I'm not sure what you're arguing. Our job is to make sure users whose philosophy is like Ian's are as well-served as users whose philosophy is like yours, and our hands are not tied. You may have missed the part where Ian said that, to protect their user's privacy, browsers *must* clear cookies and LocalStorage at the same time. Wrong. This is precisely what I argued that the spec didn't say, and (in agreement) Ian changed the spec so people wouldn't misinterpret it by thinking it said this. This is the precise reason why I don't understand what you're arguing about. PK
Re: [whatwg] Web Storage: apparent contradiction in spec
On Thu, Sep 3, 2009 at 5:59 PM, Peter Kastingpkast...@google.com wrote: On Thu, Sep 3, 2009 at 3:55 PM, Tab Atkins Jr. jackalm...@gmail.com wrote: You may have missed the part where Ian said that, to protect their user's privacy, browsers *must* clear cookies and LocalStorage at the same time. Wrong. This is precisely what I argued that the spec didn't say, and (in agreement) Ian changed the spec so people wouldn't misinterpret it by thinking it said this. This is the precise reason why I don't understand what you're arguing about. 10 hours ago, Hixie said: The fact that local storage can be used for cookie resurrection means we have to make sure that clearing one clears the other. Anything else would be a huge privacy issue (just as Flash has been). There are a few other remarks in line with this from that email and a subsequent one. If this doesn't imply that LocalStorage in practice will thus become an ephemeral storage mechanism that is easy for users to accidentally blow away, and thus will be too unreliable for anything beyond a supercookie, then I'd like clear statements to that effect. As it is, theoretical assurances that it's all going to be okay aren't adequate - if any browser *actually* made cookie-clearing *automatically* also clear LocalStorage, we authors can't rely on it. Providing the *option* to blow away everything at once is fine, if notated properly in the UI. *Automatically* doing so, as Hixie strongly appears to be suggesting browsers should do, is not. ~TJ
Re: [whatwg] Web Storage: apparent contradiction in spec
On Fri, Sep 4, 2009 at 12:33 AM, Ian Hicksoni...@hixie.ch wrote: Flash's privacy problem can be removed by uninstalling Flash. They're not a license to add more privacy problems to the platform. And a permanent's storage potential privacy problems could also be removed by having separate Delete cookies from this site and Delete cookies and all other data from this site buttons side by side. On Fri, Sep 4, 2009 at 12:36 AM, Peter Kastingpkast...@google.com wrote: All the spec really says is that UAs should note to their users that sites can keep data about them in Local Storage. This isn't grounds for a tantrum. The problem is not what the spec says, or is supposed to say, but how does it say it. This long discussion seems to be mostly around the point that the current wording is too likely to be miss-interpreted as The delete cookies button (or any equivalent UI element) should also delete all other data stored by the site. Now, this question is mostly addressed to Ian, as the only one who can provide a 100% accurate answer: based on the spec text intent, would the idea of having separate Delete cookies and Delete everything buttons side by side be conformant? If it would (and a lot of people here seem to be arguing that it would), then this discussion could be easily be put to an end by tweaking the wording in a way that makes this more clearer. Extra: more mails are flowing as I'm writing this, more messages are flowing into my inbox. Trying to reply to all of them would get me on an endless loop, but the discussion seems to be more about what the spec text is supposed to mean rather than what it would say. So, please, Ian, whatever it is supposed to say, could you word it in a way that is clear enough for everybody? Discussion about what the spec should say is generally productive but, IMO, discussion about what it's supposed to mean is counter-productive: the efforts put by all participants into this debate would be more useful on other aspects of the language. Regards, Eduard Pascual
Re: [whatwg] Web Storage: apparent contradiction in spec
On Thu, Sep 3, 2009 at 4:08 PM, Tab Atkins Jr. jackalm...@gmail.com wrote: 10 hours ago, Hixie said: The fact that local storage can be used for cookie resurrection means we have to make sure that clearing one clears the other. Anything else would be a huge privacy issue (just as Flash has been). There are a few other remarks in line with this from that email and a subsequent one. Statements in an email are not equivalent to statements in the spec. Ian is rightly concerned about user privacy, and the spec reflects that concern by advising that user agents should try to make this clear to users. But what it does not say is that UAs must clear them at the same time, and the fact that the spec language was changed SPECIFICALLY to try and make this more clear is pretty solid evidence. As I have said before, in the end it's irrelevant what you and Ian want to argue about by email: what ultimately matters is what UA vendors decide to do, which the spec will then reflect. And I find it extraordinarily unlikely that any UA vendor would not implement granular control over local storage. If this doesn't imply that LocalStorage in practice will thus become an ephemeral storage mechanism that is easy for users to accidentally blow away, and thus will be too unreliable for anything beyond a supercookie, then I'd like clear statements to that effect. As it is, theoretical assurances that it's all going to be okay aren't adequate - if any browser *actually* made cookie-clearing *automatically* also clear LocalStorage, we authors can't rely on it. Clear statements aren't going to make you any safer than unclear ones will make you unsafe. A UA vendor can easily decide to violate a clear statement (and this has been done countless times with numerous specs in the past). That said, I have heard nothing from *any* UA vendor that suggests anyone will do what you fear. PK
Re: [whatwg] Web Storage: apparent contradiction in spec
On Fri, 4 Sep 2009, Eduard Pascual wrote: Now, this question is mostly addressed to Ian, as the only one who can provide a 100% accurate answer: based on the spec text intent, would the idea of having separate Delete cookies and Delete everything buttons side by side be conformant? Yes. If it would (and a lot of people here seem to be arguing that it would), then this discussion could be easily be put to an end by tweaking the wording in a way that makes this more clearer. The wording was tweaked very recently. Is it still not clear? -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Web Storage: apparent contradiction in spec
On Thu, Sep 3, 2009 at 7:12 PM, Eduard Pascualherenva...@gmail.com wrote: The problem is not what the spec says, or is supposed to say, but how does it say it. This long discussion seems to be mostly around the point that the current wording is too likely to be miss-interpreted as The delete cookies button (or any equivalent UI element) should also delete all other data stored by the site. I think the new wording can't possibly be construed as saying that. User agents should present the persistent storage feature to the user in a way that associates them strongly with HTTP session cookies. [COOKIES] This might encourage users to view such storage with healthy suspicion. I think this is too specific -- it should say something more like User agents should make it clear to the user that to ensure privacy from sites, he must delete persistent storage as well as HTTP session cookies. But the current wording doesn't require or even encourage anything unreasonable like having cookies cleared with the same button as persistent storage.
Re: [whatwg] Web Storage: apparent contradiction in spec
On Thu, 3 Sep 2009, Aryeh Gregor wrote: I think this is too specific -- it should say something more like User agents should make it clear to the user that to ensure privacy from sites, he must delete persistent storage as well as HTTP session cookies. But the current wording doesn't require or even encourage anything unreasonable like having cookies cleared with the same button as persistent storage. There's more wording in a later section on cookie resurrection which gives more background. Does that satisfy your request? -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Web Storage: apparent contradiction in spec
On Thu, Sep 3, 2009 at 7:26 PM, Ian Hicksoni...@hixie.ch wrote: There's more wording in a later section on cookie resurrection which gives more background. Does that satisfy your request? Not really. You still have the sentence User agents should present the persistent storage feature to the user in a way that associates them strongly with HTTP session cookies., which IMO recommends an overly specific and rather questionable UI. It would be okay as an example of how UAs might present it, but should seems too strong to me. But it doesn't really matter.
Re: [whatwg] Web Storage: apparent contradiction in spec
On Thu, Sep 3, 2009 at 4:26 PM, Ian Hickson i...@hixie.ch wrote: There's more wording in a later section on cookie resurrection which gives more background. Does that satisfy your request? I think that later section actually muddies the waters. Something like this would be more clear: If users attempt to protect their privacy by clearing cookies without also clearing persistent storage data, sites can defeat those attempts by using the two features as redundant backup for each other. User agents should present the interfaces for clearing these in a way that helps users to understand this possibility and enables them to delete data in both simultaneously. IMO this achieves what you're trying for while leaving the actual UI design as open as possible. PK
Re: [whatwg] Web Storage: apparent contradiction in spec
On Fri, Sep 4, 2009 at 1:23 AM, Ian Hicksoni...@hixie.ch wrote: On Fri, 4 Sep 2009, Eduard Pascual wrote: If it would (and a lot of people here seem to be arguing that it would), then this discussion could be easily be put to an end by tweaking the wording in a way that makes this more clearer. The wording was tweaked very recently. Is it still not clear? Sorry, I wasn't aware of the update. The text is definitelly clearer now, even if I don't really like it too much. I wonder if the people discussing about what the spec says or how it says it are aware of the new wording.
Re: [whatwg] Web Storage: apparent contradiction in spec
On Thu, Sep 3, 2009 at 6:31 PM, Peter Kastingpkast...@google.com wrote: On Thu, Sep 3, 2009 at 4:26 PM, Ian Hickson i...@hixie.ch wrote: There's more wording in a later section on cookie resurrection which gives more background. Does that satisfy your request? I think that later section actually muddies the waters. Something like this would be more clear: If users attempt to protect their privacy by clearing cookies without also clearing persistent storage data, sites can defeat those attempts by using the two features as redundant backup for each other. User agents should present the interfaces for clearing these in a way that helps users to understand this possibility and enables them to delete data in both simultaneously. IMO this achieves what you're trying for while leaving the actual UI design as open as possible. I would love language to this effect. ~TJ
Re: [whatwg] Web Storage: apparent contradiction in spec
On Thu, 3 Sep 2009, Aryeh Gregor wrote: On Thu, Sep 3, 2009 at 7:26 PM, Ian Hicksoni...@hixie.ch wrote: There's more wording in a later section on cookie resurrection which gives more background. Does that satisfy your request? Not really. You still have the sentence User agents should present the persistent storage feature to the user in a way that associates them strongly with HTTP session cookies., which IMO recommends an overly specific and rather questionable UI. It would be okay as an example of how UAs might present it, but should seems too strong to me. That really isn't a specific UI. It covers a massive range of UIs including almost every suggestion made on this thread. The only thing it really discourages is having the two features be unrelated in the UI such that the user can empty one without even thinking about the other. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Web Storage: apparent contradiction in spec
On Thu, 3 Sep 2009, Peter Kasting wrote: On Thu, Sep 3, 2009 at 4:26 PM, Ian Hickson i...@hixie.ch wrote: There's more wording in a later section on cookie resurrection which gives more background. Does that satisfy your request? I think that later section actually muddies the waters. Something like this would be more clear: If users attempt to protect their privacy by clearing cookies without also clearing persistent storage data, sites can defeat those attempts by using the two features as redundant backup for each other. User agents should present the interfaces for clearing these in a way that helps users to understand this possibility and enables them to delete data in both simultaneously. IMO this achieves what you're trying for while leaving the actual UI design as open as possible. Do you mean this as a repalcement or in addition to what's in the spec now? For the Cookie Resurrection section or the User Tracking section? I don't understand the difference between what you suggest and what the spec says. What is wrong with what the spec says, that is fixed by the above? -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Web Storage: apparent contradiction in spec
On Thu, Sep 3, 2009 at 5:17 PM, Ian Hickson i...@hixie.ch wrote: On Thu, 3 Sep 2009, Peter Kasting wrote: On Thu, Sep 3, 2009 at 4:26 PM, Ian Hickson i...@hixie.ch wrote: There's more wording in a later section on cookie resurrection which gives more background. Does that satisfy your request? I think that later section actually muddies the waters. Something like this would be more clear: If users attempt to protect their privacy by clearing cookies without also clearing persistent storage data, sites can defeat those attempts by using the two features as redundant backup for each other. User agents should present the interfaces for clearing these in a way that helps users to understand this possibility and enables them to delete data in both simultaneously. IMO this achieves what you're trying for while leaving the actual UI design as open as possible. Do you mean this as a repalcement or in addition to what's in the spec now? Replacement. For the Cookie Resurrection section or the User Tracking section? Cookie resurrection section. Although because the comments in both sections are so similar, I'm not sure I see value in having two sections. Just having one, which has this text, seems fine. I don't understand the difference between what you suggest and what the spec says. What is wrong with what the spec says, that is fixed by the above? The key objectionable phrase in the cookie resurrection section is presents data in the persistent storage features ... separately from data in HTTP session cookies. This can be construed to mean that the UA should not _ever_ separate presentation of the stored data in the two features (lest it risk user hazard), when in practice a UA may want UI with some amount of presentation separation (e.g. side-by-side buttons that call up separate dialogs for the data stored in each feature) while still trying to make it clear and easy for users to manage their privacy. I don't believe the spec intends to imply this level of detail about UAs' UI, so I am trying to remove any potential for ambiguity. IMO my suggested text cannot be construed as mandating fine details of the presentation of the data. FWIW, the text in the User Tracking section that says associates them strongly is perhaps also still going too far, for similar reasons. It's not clear precisely what this means, and for some use cases (e.g. gmail storing document drafts for offline editing) the use of persistent storage is not equivalent to an HTTP session cookie; if a UA has some sort of detailed knowledge of the distinctions here, it should be allowed to present data in whatever way is most clear and helpful to the user. The text currently in the spec, while much better than before, still goes beyond noting a risk to users that UAs should highlight, and ventures into the realm of prescribing specific solutions for that risk, which may not always be appropriate. PK
Re: [whatwg] Web Storage: apparent contradiction in spec
On Aug 31, 2009, at 12:04 PM, Peter Kasting wrote: If you combine that statement with section 6.1's User agents should present the persistent storage feature to the user in a way that does not distinguish them from HTTP session cookies, then the result is that, when the user requests to delete cookies from a site, the UA will also delete that site's local storage. That is exactly the behavior I am concerned about. That's not true. You're misinterpreting a statement about the granularity of control users should have as one about what terminology a UA should use. The [lack of] granularity of control actually is a serious concern, whatever the terminology. The spec already recommends a bunch of things about what users should be shown w.r.t. Local Storage, such as how much space a site is using, so it's clear that a UA that wants to comply with this should is going to need to construct UI that doesn't just use the word cookies everywhere but actually presents the data as here's your locally stored data for this site with local storage content enumerated. Users won't be given a prompt that says clear cookies that, confusingly, clears more than cookies; they'll be given a prompt like clear all locally stored data. The command will have to say something about cookies or it'll confuse anyone but an HTML5 expert. It'd have to be more like Clear cookies and other locally stored data. The fundamental problem here is that some uses of local storage are nothing at all like cookies, for the same reason that ~/Documents is not the same as ~/Library/Caches. In the example I gave, the user needs to delete cookies for a site, but absolutely should not delete local storage. For the spec to tell browser developers to present the two as being the same thing makes no sense here. —Jens
Re: [whatwg] Web Storage: apparent contradiction in spec
On Wed, Sep 2, 2009 at 11:08 AM, Jens Alfke s...@google.com wrote: On Aug 31, 2009, at 12:04 PM, Peter Kasting wrote: If you combine that statement with section 6.1's User agents should present the persistent storage feature to the user in a way that does not distinguish them from HTTP session cookies, then the result is that, when the user requests to delete cookies from a site, the UA will also delete that site's local storage. That is *exactly* the behavior I am concerned about. That's not true. You're misinterpreting a statement about the granularity of control users should have as one about what terminology a UA should use. The [lack of] granularity of control actually is a serious concern, whatever the terminology. It still seems like you are interpreting this statement as saying that the UA must not allow users to keep/clear cookies separately from Local Storage data. While on the face of it that seems like a possible interpretation, I think it's clear that this would be a lousy user experience and detrimental to developers as well. Therefore I am convinced that the intent of the statement is to say that UAs must give users the same _abilities_ to see and clear Local Storage data as they already have with cookies, not that the two things should always be lumped together and made indistinguishable. Of course, Hixie could step in here and clarify what he means. But if he really means what you think he means (that users must not be able to tell a difference or control the two separately) then that seems like obvious grounds for a revolt. The spec already recommends a bunch of things about what users should be shown w.r.t. Local Storage, such as how much space a site is using, so it's clear that a UA that wants to comply with this should is going to need to construct UI that doesn't just use the word cookies everywhere but actually presents the data as here's your locally stored data for this site with local storage content enumerated. Users won't be given a prompt that says clear cookies that, confusingly, clears more than cookies; they'll be given a prompt like clear all locally stored data. The command will have to say something about cookies or it'll confuse anyone but an HTML5 expert. It'd have to be more like Clear cookies and other locally stored data. In general UAs should have separate checkboxes for these on their clear private data UIs. The fundamental problem here is that *some uses of local storage are nothing at all like cookies*, for the same reason that ~/Documents is not the same as ~/Library/Caches. Yes, this is precisely why UAs should present them separately. In the example I gave, the user needs to delete cookies for a site, but absolutely should not delete local storage. For the spec to tell browser developers to present the two as being the same thing makes no sense here. See comment above. PK
Re: [whatwg] Web Storage: apparent contradiction in spec
On Sep 2, 2009, at 11:36 AM, Peter Kasting wrote: It still seems like you are interpreting this statement as saying that the UA must not allow users to keep/clear cookies separately from Local Storage data. Yes; that seemed the most direct interpretation of its language. While on the face of it that seems like a possible interpretation, I think it's clear that this would be a lousy user experience and detrimental to developers as well. Therefore I am convinced that the intent of the statement is to say that UAs must give users the same _abilities_ to see and clear Local Storage data as they already have with cookies, not that the two things should always be lumped together and made indistinguishable. Then we are in violent agreement :) Good. As a newbie here I was less inclined to give the spec the benefit of the doubt than you were. So do the rest of us agree about what the statement should mean? And if so, how could it be rephrased to say that more unambiguously? I think Peter's quote above is a good start. —Jens
Re: [whatwg] Web Storage: apparent contradiction in spec
On Thu, Sep 3, 2009 at 4:50 AM, Jens Alfke s...@google.com wrote: On Sep 2, 2009, at 11:36 AM, Peter Kasting wrote: It still seems like you are interpreting this statement as saying that the UA must not allow users to keep/clear cookies separately from Local Storage data. Yes; that seemed the most direct interpretation of its language. While on the face of it that seems like a possible interpretation, I think it's clear that this would be a lousy user experience and detrimental to developers as well. Therefore I am convinced that the intent of the statement is to say that UAs must give users the same _abilities_ to see and clear Local Storage data as they already have with cookies, not that the two things should always be lumped together and made indistinguishable. Then we are in violent agreement :) Good. As a newbie here I was less inclined to give the spec the benefit of the doubt than you were. So do the rest of us agree about what the statement should mean? And if so, how could it be rephrased to say that more unambiguously? I think Peter's quote above is a good start. The fact that we talked about this so long demonstrates how unclear it is. I agree that Peter's reading is one possible interpretation, but I worry that not everyone will read and implement it that way. An extra sentence or two would go a long way.
Re: [whatwg] Web Storage: apparent contradiction in spec
On 01/09/2009 00:14, Tab Atkins Jr. jackalm...@gmail.com wrote: Sure, the ones using it for tracking that care *that much* will use other solutions anyway. But people who just want some persistent storage as part of their app, because it's useful to their users, will use the browser-native solution if it works. If LocalStorage is explicitly supposed to be as ephemeral of cookies, though, that will push people towards stuff like Flash LocalStorage instead. No one in their right mind would use flash LocalStorage for user critical data. It's great for tracking because most users don't know how to clear it, but because user's don't know about it they also don't back it up or transfer it to new computers/browsers etc. Besides which, there are already very popular UAs that have no support for Flash and thus no Flash LocalStorage. It would be nice to not create the same privacy hole on those platforms. Regards, Adrian Sutton. __ Adrian Sutton, CTO UK: +44 1 628 200 182 x481 US: +1 (650) 292 9659 x717 Ephox http://www.ephox.com/ Ephox Blogs http://planet.ephox.com/, Personal Blog http://www.symphonious.net/
Re: [whatwg] Web Storage: apparent contradiction in spec
On Sep 1, 2009, at 12:11 AM, Adrian Sutton wrote: On 01/09/2009 00:14, Tab Atkins Jr. jackalm...@gmail.com wrote: Sure, the ones using it for tracking that care *that much* will use other solutions anyway. But people who just want some persistent storage as part of their app, because it's useful to their users, will use the browser-native solution if it works. If LocalStorage is explicitly supposed to be as ephemeral of cookies, though, that will push people towards stuff like Flash LocalStorage instead. No one in their right mind would use flash LocalStorage for user critical data. This is wrong. That developers use Flash LocalStorage for this is not hypothetical. It's the best option they have, so they've been doing it - even though it has its own horrible flaws. It's great for tracking because most users don't know how to clear it, but because user's don't know about it they also don't back it up or transfer it to new computers/browsers etc. Tracking aside, Flash LocalStorage *is* also used for storage of user data. It is flawed for this, but the fact is: Flash LocalStorage is currently the best way to store data on the client machine and have a reasonable expectation that it will be there in the future. If HTML5 LocalStorage isn't *at least as reliable*, then developers will keep using Flash. That users don't know about it and don't know to back-up or transfer this data is something that user agents have an interest to change, but plug-in developers probably don't. Besides which, there are already very popular UAs that have no support for Flash and thus no Flash LocalStorage. It would be nice to not create the same privacy hole on those platforms. Equating HTML5 LocalStorage with a privacy hole seems to be a bit of a hyperbole, and a bit unfounded. The fact that we're still having this discussion is reflective of how much browser developers have learned about the security of the web and our users data, and how little we want to repeat past mistakes. Flash LocalStorage is the *current* privacy hole, and we won't move the web forward and bring this type of data into the light until we can at least match the expectations developers already have. ~Brady Regards, Adrian Sutton. __ Adrian Sutton, CTO UK: +44 1 628 200 182 x481 US: +1 (650) 292 9659 x717 Ephox http://www.ephox.com/ Ephox Blogs http://planet.ephox.com/, Personal Blog http://www.symphonious.net/
Re: [whatwg] Web Storage: apparent contradiction in spec
On Tue, Sep 1, 2009 at 3:11 AM, Adrian Suttonadrian.sut...@ephox.com wrote: Besides which, there are already very popular UAs that have no support for Flash and thus no Flash LocalStorage. It would be nice to not create the same privacy hole on those platforms. What's the privacy hole, if users are given an option to clear localStorage in the standard clear private data dialog? It should just be a separate option from cookies, since users might legitimately want to clear cookies (typically login info) but not localStorage.
Re: [whatwg] Web Storage: apparent contradiction in spec
Please take in to account that email is asynchronous but spam is still annoying. Linus On Mon, Aug 31, 2009 at 3:11 AM, Ian Hickson i...@hixie.ch wrote: In addition, I'd like to see the pop-up dialogs for the location API removed. I find the Can I know where you are? dialogs on the iPhone very annoying. Mistakes were made. Perhaps we can find a way to make input type=location work well instead. The geolocation model is asynchronous, which gets around this neatly (the UI can slide in and the user can ignore it until he's ready to give his location).
Re: [whatwg] Web Storage: apparent contradiction in spec
On Tue, 25 Aug 2009, Jens Alfke wrote: I've just noticed an apparent self-contradiction in the Web Storage spec (24 August draft). Section 4.3 states: Data stored in local storage areas should be considered potentially user-critical. It is expected that Web applications will use the local storage areas for storing user-written documents. Section 6.1 states: User agents should present the persistent storage feature to the user in a way that does not distinguish them from HTTP session cookies. These statements are contradictory, because cookies don't store user-critical data such as documents. The user model of cookies is that they're conveniences (at best) for keeping you logged into a site or remembering preferences like font-size, so deleting them is no more than an inconvenience. If local storage is presented to the user as being cookies, then a user may delete it without understanding the consequences. Potential result: I was having trouble logging into FooDocs.com, so my friend suggested I delete the cookies for that site. After that I could log in, but now the document I was working on this morning has lost all the changes I made! How do I get them back? I suggest that the sub-section Treating persistent storage as cookies of section 6.1 be removed. We can't treat cookies and persistent storage differently, because otherwise we'll expose users to cookie resurrection attacks. Maintaining the user's expectations of privacy is critical. So I've removed the text that says that local storage could be user-critical. On Tue, 25 Aug 2009, Brady Eidson wrote: One key advantage of LocalStorage and Databases over cookies is that they *do* have a predictable, persistent lifetime, and the browser is *not* allowed to prune them at will. I haven't changed this. On Tue, 25 Aug 2009, Brady Eidson wrote: LocalStorage is quite clearly modeled after Flash's LocalStorage Actually I don't think I'd heard of Flash's LocalStorage at the time that localStorage (then globalStorage) was first specced. On Wed, 26 Aug 2009, Michael Nordman wrote: * Can an individual key,value pair for an origin be removed from local storage while leaving other local data in place? * Can an individual Database be deleted for an origin while leaving other local data in place? * Can an individual Manifest be deleted for an origin while leaving other local data in place? * Or should an origin's data be subject to all-or-none eviction. I would prefer to see the spec clarify questions along those lines. That would be useful. The spec leaves all of this up to the UA. It discourages unprompted eviction, but at the end of the day, as has been pointed out, there are limits to what can be guaranteed. On Wed, 26 Aug 2009, Peter Kasting wrote: That's not the proposed mandate. The proposed mandate is thou shalt not discard successfully-written data without explicit user action, which seems implementable to me. Note that this doesn't make claims like the hard drive will not fail, and it doesn't claim that the UA is required to allow the app to write whatever data it wants in the first place. That's more or less what the spec says, but it uses should which allows UAs to violate this in extreme cases (like abuse being detected, or running out of disk space, or whatnot). On Wed, 26 Aug 2009, Michael Nordman wrote: What seems inevitable are vista-like prompts to allow something (or prods to delete something) seemingly unrelated to a user's interaction with a site... please, oh please, lets avoid making that part of the web platform. I'm assuming that UA will have out-of-band mechanisms to 'bless' certain sites which should not be subject to automated eviction. If push comes to shove, the system could suggest cleaning up one of these 'blessed' sites if inactivity for an extended period was noticed. But for the overwhelming number of sites in a users browsing history, its a different matter. If the storage APIs are just available for use, no questions asked making the storage just go away, no questions asked, is symmetrical. Blessing involves asking questions... making it go away does too. UAs can do this, yes. On Wed, 26 Aug 2009, Jens Alfke wrote: If this is out-of-spec and browser-dependent, there won't be a good way for an app to request that blessing; it'll be something the user has to know to do, otherwise their data can get lost. That seems dangerous. In most systems user data loss is just about the worst-case scenario of what could go wrong, and you try to prevent it at all costs. In-band systems can be abused, out-of-band systems can be hard to discover. I don't know which is worse. We had bb type=makeapp for a while (an in-band system), but we removed it due to implementor feedback. My suggestion to have separate 'important' and 'cache' local storage areas would provide such a mechanism in a
Re: [whatwg] Web Storage: apparent contradiction in spec
On Aug 31, 2009, at 3:11 AM, Ian Hickson wrote: We can't treat cookies and persistent storage differently, because otherwise we'll expose users to cookie resurrection attacks. Maintaining the user's expectations of privacy is critical. The fact that local storage can be used as a type of super-cookie doesn't mean the two are the same thing. Yes, obviously if I give a website permission to put 50MB of stuff on my disk, it can use 1k of that as a type of cookie if it wants. That's just one of many reasons why user agents should require user approval for letting a domain access local storage. That does not mean that the Delete Cookies menu command should also delete local storage. Users often delete cookies to resolve login issues (I've had to do this with Google websites several times). Conflating the two can lead to disasters like I told you to delete my COOKIES! Not my EMAIL DRAFTS that I was trying to log in to send! So I've removed the text that says that local storage could be user- critical. That's going to come as a shock to developers who were planning to use it for user-created data (whether drafts of content to be pushed to the cloud, or strictly-local documents.) Without this, the safe usage of local storage diminishes to a download cache. —Jens
Re: [whatwg] Web Storage: apparent contradiction in spec
On Mon, Aug 31, 2009 at 11:12 AM, Jeremy Orlow jor...@chromium.org wrote: Yes, this is pretty disconcerting since there's been OVERWHELMING support for LocalStorage being treated as user-critical on this thread. The spec says basically what you want except that it uses should. It seems like UAs and authors would both be satisfied with this; I don't expect any UA vendor to wantonly discard local storage data. PK
Re: [whatwg] Web Storage: apparent contradiction in spec
On Mon, Aug 31, 2009 at 11:18 AM, Peter Kasting pkast...@google.com wrote: On Mon, Aug 31, 2009 at 11:12 AM, Jeremy Orlow jor...@chromium.orgwrote: Yes, this is pretty disconcerting since there's been OVERWHELMING support for LocalStorage being treated as user-critical on this thread. The spec says basically what you want except that it uses should. It seems like UAs and authors would both be satisfied with this; I don't expect any UA vendor to wantonly discard local storage data. If some platforms don't treat local storage as sacred, it's unlikely web developers are going to program one version for flash/whatever and one version for local storage on browsers that meet their requirements. Especially since the only way you could treat these differently is by user agent detection which has historically been a bad thing for the web.
Re: [whatwg] Web Storage: apparent contradiction in spec
On Aug 31, 2009, at 11:18 AM, Peter Kasting wrote: The spec says basically what you want except that it uses should. It seems like UAs and authors would both be satisfied with this; I don't expect any UA vendor to wantonly discard local storage data. By encouraging the UI to treat local storage as a type of cookie, the spec is contributing to user confusion that can lead to the type of data loss that I described in my previous message. Since user data loss is one of the very worst things any program can do to a user, I'd like the spec to encourage UAs to take a bit more care with local storage. —Jens
Re: [whatwg] Web Storage: apparent contradiction in spec
On Aug 31, 2009, at 11:35 AM, Peter Kasting wrote: Again, the spec now says in 4.3: User agents should expire data from the local storage areas only for security reasons or when requested to do so by the user. The only stronger statement you could get would be by changing this to a must. It's not clear to me that that is going to result in any practical difference on the part of implementations or author perception. If you combine that statement with section 6.1's User agents should present the persistent storage feature to the user in a way that does not distinguish them from HTTP session cookies, then the result is that, when the user requests to delete cookies from a site, the UA will also delete that site's local storage. That is exactly the behavior I am concerned about. This sounds like you are either completely ignoring, or disagreeing with, my claim that UAs aren't going to be flippant about this data. If UA's shouldn't treat the data lightly, then I would prefer to see a statement to that effect in the spec, such as the one that was just deleted. Local storage is a significant change from the browser's current data model, and I think that (no offense) browser developers are not used to taking care of user-critical data for longer than the duration of a DOM tree or POST request. It's a change in perspective. Coming as I do from a client-software world, it's actually an eye-opener to me that this is even controversial. —Jens
Re: [whatwg] Web Storage: apparent contradiction in spec
Jens Alfke wrote: Local storage is a significant change from the browser's current data model, and I think that (no offense) browser developers are not used to taking care of user-critical data for longer than the duration of a DOM tree or POST request. It's a change in perspective. Coming as I do from a client-software world, it's actually an eye-opener to me that this is even controversial. It's controversial because, no offense, browser developers don't trust the website author, nor should the users. At least to a first approximation. This is a critical difference from client software, for better or worse. We could restrict local storage to explicitly trusted sites and then not treat it as cookies; would that be preferable? It might be. Alternately, it could be treated like cookies except for explicitly trusted sites... or something. -Boris
Re: [whatwg] Web Storage: apparent contradiction in spec
On Mon, Aug 31, 2009 at 11:50 AM, Jens Alfke s...@google.com wrote: On Aug 31, 2009, at 11:35 AM, Peter Kasting wrote: Again, the spec now says in 4.3: User agents should expire data from the local storage areas only for security reasons or when requested to do so by the user. The only stronger statement you could get would be by changing this to a must. It's not clear to me that that is going to result in any practical difference on the part of implementations or author perception. If you combine that statement with section 6.1's User agents should present the persistent storage feature to the user in a way that does not distinguish them from HTTP session cookies, then the result is that, when the user requests to delete cookies from a site, the UA will also delete that site's local storage. That is *exactly* the behavior I am concerned about. That's not true. You're misinterpreting a statement about the granularity of control users should have as one about what terminology a UA should use. The spec already recommends a bunch of things about what users should be shown w.r.t. Local Storage, such as how much space a site is using, so it's clear that a UA that wants to comply with this should is going to need to construct UI that doesn't just use the word cookies everywhere but actually presents the data as here's your locally stored data for this site with local storage content enumerated. Users won't be given a prompt that says clear cookies that, confusingly, clears more than cookies; they'll be given a prompt like clear all locally stored data. It seems like you're convinced that UAs won't create UI users can understand, and so you're trying to make the spec mandate what you think will be comprehensible for users. IMO this is not only out-of-scope but pointless, as UAs are going to do what they want anyway. The spec is already pretty clear in telling UAs not to be casual about things, I don't think you're going to change what actually gets implemented by demanding more. This sounds like you are either completely ignoring, or disagreeing with, my claim that UAs aren't going to be flippant about this data. If UA's shouldn't treat the data lightly, then I would prefer to see a statement to that effect in the spec, such as the one that was just deleted. The sentence I quoted in 4.3 says _exactly_ that UAs should not treat data lightly. I think that (no offense) browser developers are not used to taking care of user-critical data for longer than the duration of a DOM tree or POST request. This kind of generalization is just silly. See e.g. saved passwords, extensions, stored browsing history, persistent settings, etc. PK
Re: [whatwg] Web Storage: apparent contradiction in spec
On Mon, Aug 31, 2009 at 6:11 AM, Ian Hicksoni...@hixie.ch wrote: We can't treat cookies and persistent storage differently, because otherwise we'll expose users to cookie resurrection attacks. Maintaining the user's expectations of privacy is critical. By that reasoning we can't treat cookies differently from the HTTP cache (ETag) or history (URIs with session IDs), I think. I don't know of any UAs that expire history/cookie/cache in sync to avoid correlations -- if it's even possible to do so -- and I don't think I've seen any bugs asking Firefox to do so. Mike
Re: [whatwg] Web Storage: apparent contradiction in spec
On Aug 31, 2009, at 11:58 AM, Boris Zbarsky wrote: It's controversial because, no offense, browser developers don't trust the website author, nor should the users. At least to a first approximation. Over on another thread of this list we've already been talking about the need to get the user's permission before a site can use [more than a certain minimum of] local storage. So that implies the user expressing a degree of trust in the site, at least enough trust to let it use a sliver of her hard disk. I agree that if an app is just storing a few kbytes of local storage without the user's informed consent, that's just the moral equivalent of a cookie and ought to be treated as such. We could restrict local storage to explicitly trusted sites and then not treat it as cookies; would that be preferable? It might be. That would be fine. The problem is that this seems to require an API change to allow the site to distinguish between persistent storage I'm just using quietly as a cookie, and persistent storage I want to be able to store larger amounts of possibly user-critical data in. —Jens
Re: [whatwg] Web Storage: apparent contradiction in spec
Quoting Ian Hickson i...@hixie.ch: On Tue, 25 Aug 2009, Jens Alfke wrote: Potential result: I was having trouble logging into FooDocs.com, so my friend suggested I delete the cookies for that site. After that I could log in, but now the document I was working on this morning has lost all the changes I made! How do I get them back? I suggest that the sub-section Treating persistent storage as cookies of section 6.1 be removed. We can't treat cookies and persistent storage differently, because otherwise we'll expose users to cookie resurrection attacks. Maintaining the user's expectations of privacy is critical. I think the paragraph under treating persistent storage as cookies should simply be removed. The remainder of that section already does an adequate job of explaining the privacy implications of persistent storage. The UI should be entirely at the discretion of the browser vendor since it involves a variety of tradeoffs, with the optimum solution depending on the anticipated user base of the browser. Placing spec requirements simply limits the abilities of browser vendors to find innovative solutions to the problem. In addition, since there is no interoperability requirement here, using RFC 2119 language seems inappropriate; especially since the justification given is rather weak (this might encourage users?) and not supported by any evidence. As to what browser vendors should actually _do_, it seems to me that the user's expectations of privacy is actually an illusion in this case; all the bad stuff that can be done with persistent storage can already be done using a variety of techniques. Trying to fix up this one case seems like closing the stable door after the horse has bolted. Therefore the delete local storage when you delete cookies model seems flawed, particularly as it can lead to the type of problem that Jens described above. On a slightly different topic, it is unclear what the relationship between the statement in section 4.3 User agents should expire data from the local storage areas only for security reasons or when requested to do so by the user and the statement in section 6.1 User agents may automatically delete stored data after a period of time. is supposed to be. Does the latter count as a security reason?
Re: [whatwg] Web Storage: apparent contradiction in spec
On Mon, Aug 31, 2009 at 2:01 PM, Jens Alfkes...@google.com wrote: The fact that local storage can be used as a type of super-cookie doesn't mean the two are the same thing. Yes, obviously if I give a website permission to put 50MB of stuff on my disk, it can use 1k of that as a type of cookie if it wants. That's just one of many reasons why user agents should require user approval for letting a domain access local storage. I was under the impression that the idea was that this wouldn't be necessary. Sites should be free to use localStorage as they use cookies, transparently without annoying the user with spammed pop-ups. Does any browser currently require user approval for localStorage by default, or plan to? That does not mean that the Delete Cookies menu command should also delete local storage. Users often delete cookies to resolve login issues (I've had to do this with Google websites several times). Conflating the two can lead to disasters like I told you to delete my COOKIES! Not my EMAIL DRAFTS that I was trying to log in to send! I entirely agree with this, as a user. Browsers should expose the option to clear cookies and localStorage separately. Currently Chrome's Clear Browsing Data has a separate checkboxes for browsing history, download history, cache, cookies, saved passwords, and saved form data. I'd expect it to add another checkbox for localStorage, not change cookies to cookies and localStorage. Although in any event, I'm not sure what good UI would be to say This might destroy valuable data, but probably not most of it, and whether it destroys the particular bit of data you're thinking of really depends on if the app happens to store it locally or on its servers, and if on its servers, whether it's pushed out this data yet or not. On Mon, Aug 31, 2009 at 2:36 PM, Tab Atkins Jr.jackalm...@gmail.com wrote: Outlawing persistent storage in HTML5 as a privacy mechanism does *nothing* for privacy. There are numerous methods, Flash LocalStorage in particular, that can and will be used to achieve what we developers want. These methods will be *harder* for the end-user to monitor and control, and result in privacy violations being *easier*. What you see as a reasonable step to protect privacy, we see as an admonition that we'd better get used to Flash, because it's here to stay. I don't think that the ones using Flash LocalStorage to track users will ever switch to a standards-based solution. Anything implemented by the browser vendors will allow users to easily clear it, which defeats their goal. What HTML 5 needs to do is supplant Flash's *site-critical* functions. If Flash is relegated to only things like tracking methods, it would be feasible for it to just not be installed, and users wouldn't care. Do site-critical Flash apps (i.e., ones where the user would say the site is broken if they didn't work) often rely heavily on localStorage persistence?
Re: [whatwg] Web Storage: apparent contradiction in spec
On Mon, Aug 31, 2009 at 6:08 PM, Aryeh Gregorsimetrical+...@gmail.com wrote: On Mon, Aug 31, 2009 at 2:36 PM, Tab Atkins Jr.jackalm...@gmail.com wrote: Outlawing persistent storage in HTML5 as a privacy mechanism does *nothing* for privacy. There are numerous methods, Flash LocalStorage in particular, that can and will be used to achieve what we developers want. These methods will be *harder* for the end-user to monitor and control, and result in privacy violations being *easier*. What you see as a reasonable step to protect privacy, we see as an admonition that we'd better get used to Flash, because it's here to stay. I don't think that the ones using Flash LocalStorage to track users will ever switch to a standards-based solution. Anything implemented by the browser vendors will allow users to easily clear it, which defeats their goal. What HTML 5 needs to do is supplant Flash's *site-critical* functions. If Flash is relegated to only things like tracking methods, it would be feasible for it to just not be installed, and users wouldn't care. Do site-critical Flash apps (i.e., ones where the user would say the site is broken if they didn't work) often rely heavily on localStorage persistence? Sure, the ones using it for tracking that care *that much* will use other solutions anyway. But people who just want some persistent storage as part of their app, because it's useful to their users, will use the browser-native solution if it works. If LocalStorage is explicitly supposed to be as ephemeral of cookies, though, that will push people towards stuff like Flash LocalStorage instead. I think a lot of people weren't that aware of Flash LocalStorage, but more will be now that we're talking about it. ~TJ
Re: [whatwg] Web Storage: apparent contradiction in spec
Michael Nordman wrote: And to confound the problem further, UAs dont have meta-data on hand with which to relate various pieces of local data together and attribute them to a specific user-identifiable 'application'. Everything is bound to a security-origin, but that doesn't clearly identify or label an 'application'. Agreed. It would be nice if data was bound to namespaced contexts on a more fine-grained model than origin/hostname. This could potentially be done using the path-based mechanism I just posted about in the thread titled origin+path namespacing and security. It would be good with a single mechanism that can be extended to confine applications within their own namespaces (to not interfer with each other), be used for security, and for assisting user decisions on what to keep or delete. Best regards Mike
Re: [whatwg] Web Storage: apparent contradiction in spec
On Aug 27, 2009, at 5:53 PM, Dirk Pranke wrote: Would you hold a different opinion for different types of HTML5-based applications (like the TiddlyWiki app that gets copied locally)? Or would you require that all apps go through an install process that acquires quota, permissions, etc.? Locally-stored apps are a different story. But I don't necessarily see TiddlyWiki (or similar apps) as working that way. It currently has to be saved to a local HTML file because its only mechanism for saving content is to self-modify its source. With HTML5 persistent storage, there's no reason it couldn't live on a server like tiddlywiki.com. With app-caching support too, it would work offline just as well as the current version does. —Jens
Re: [whatwg] Web Storage: apparent contradiction in spec
Speaking up as an application developer ;-) here, I think the evict data at browser's choice route is fatal for new inventions in app development. I've been hoping that WebStorage and Offline together with other new APIs could provide a platform that allows us to build applications free from dependencies to specific platforms/OSs (a no-brainer for the web) and also freeing us from the tyranny of the 50-year old file concept (information doesn't necessarily want to be contained in files, and most of the time users don't want to do the additional task of administering file names and folder structures on a local disk). These new apps might not have a server at all, maybe using peer-to-peer technologies to exchange parts of its data with the outside world. If, like many say here, a user file is considered more precious than data in localStorage, and the latter may have an automatic eviction scheme, then these new apps cannot use WebStorage. Instead I would look forward to new File APIs, allowing me to store data in local files. And then we are back to having our users handling and managing files and folders on their local disk... To back up a bit, offline data can be used to improve applications in different ways, and here's a quick list of scenarios I come to think of: * the browser makes an automatic decision to cache certain server data * the application decides to cache certain server data without asking the user * the application caches certain server data in response to user command (I need to have all my mail for offline access while I'm on the plane) * the application allows creation of new data and modification of existing cached server data while offline * the application allows creation and modification of data without relying on that there is a server or a copy of the data on that server If localStorage data is not considered precious I can only see the first two types of caching possible in professional applications. Even the third point would not be acceptable if the user has specifically asked to have a certain subset of his data available while travelling the Amazonas, just to discover that it was evicted because the browser decided to. As someone said, I think it is important that the spec clarifies which of these scenarios WebStorage is aiming to fulfill. Best regards Mike Wilson Drew Wilson wrote: This is one of those times when I *really* wish that the application developer community was more active on this list. I absolutely understand Linus' point of view, but I also feel like we are really hamstringing applications when we make choices like this and I wish that those developers were more vocally represented in these types of discussions. Going down this path would basically kill the ability to have offline web applications, because there would be no guarantees that the data would persist until the user comes back online. But since that point's already been made several times, I guess it's not a compelling argument. -atw On Wed, Aug 26, 2009 at 8:23 PM, Linus Upson li...@google.com wrote: I simply want clicking on links to be safe. In a previous thread I wrote safe and stateless but I'm coming to the opinion that stateless is a corollary of safe. Clicking on links shouldn't, either by filling my disk or hitting my global quota, someday lead to a dialog which reads, Please choose what to delete so that web sites will continue to work. The candidate delete list will be thousands long and hidden in that haystack will be a few precious needles. I also want to avoid any [Yes] [No] dialogs. Can I do something scary [Yes] [No]? Can I do something innocuous [Yes] [No]? Users shouldn't be forced to make those kinds of safety judgements. I'm guilty of instigating at least one of those dialogs. As shamed politicians do I'll retreat to the passive voice: Mistakes were made. I'm not opposed to web apps manipulating files on the user's computer, but the user should be in explicit control. I'd support input type=open and input type=save that worked similarly to input type=file. User agents are already registering for file types so that double clicking a file with a certain extension can be automatically sent to an URL, perhaps residing in an AppCache. In addition, I'd like to see the pop-up dialogs for the location API removed. I find the Can I know where you are? dialogs on the iPhone very annoying. Mistakes were made. Perhaps we can find a way to make input type=location work well instead. Linus On Wed, Aug 26, 2009 at 5:14 PM, Brady Eidson beid...@apple.com wrote: I started writing a detailed rebuttal to Linus's reply, but by the time I was finished, many others had already delivered more targetted replies. So I'll cut the rebuttal format and make a few specific points. - Many apps act as a shoebox for managing specific types of data, and users are used to using these apps to
Re: [whatwg] Web Storage: apparent contradiction in spec
On Aug 26, 2009, at 4:51 PM, Jens Alfke wrote: To repeat what I said up above: Maybe the local storage API needs a way to distinguish between cached data that can be silently thrown away, and important data that can't. That makes sense to me. There might even be more than two categories: - Cached for convenience - discarding this will affect performance but not functionality. - Useful for offline use - discarding this will prevent some data from being accessed when offline. - Critical for offline use - discarding this will prevent the app storing this data from working offline at all. - Critical user data - discarding this will lead to permanent user data loss. Regards, Maciej
Re: [whatwg] Web Storage: apparent contradiction in spec
On 27/08/2009 15:47, Maciej Stachowiak m...@apple.com wrote: - Cached for convenience - discarding this will affect performance but not functionality. - Useful for offline use - discarding this will prevent some data from being accessed when offline. - Critical for offline use - discarding this will prevent the app storing this data from working offline at all. - Critical user data - discarding this will lead to permanent user data loss. The only catch being that if the web app decides this for itself, a malicious script or tracking cookie will be marked as critical user data when in fact the user would disagree. On the plus side, it would mean a browser could default to not allowing storage in the critical user data by default and then let users whitelist just the sites they want. This could be through an evil dialog, or just a less intrusive indicator somewhere the website itself would be able to detect that it couldn¹t save and warn the user in whatever way is most appropriate. Regards, Adrian Sutton. __ Adrian Sutton, CTO UK: +44 1 628 200 182 x481 US: +1 (650) 292 9659 x717 Ephox http://www.ephox.com/ Ephox Blogs http://planet.ephox.com/, Personal Blog http://www.symphonious.net/
Re: [whatwg] Web Storage: apparent contradiction in spec
Adrian Sutton said: On 27/08/2009 15:47, Maciej Stachowiak m...@apple.com wrote: - Cached for convenience - discarding this will affect performance but not functionality. - Useful for offline use - discarding this will prevent some data from being accessed when offline. - Critical for offline use - discarding this will prevent the app storing this data from working offline at all. - Critical user data - discarding this will lead to permanent user data loss. The only catch being that if the web app decides this for itself, a malicious script or tracking cookie will be marked as critical user data when in fact the user would disagree. On the plus side, it would mean a browser could default to not allowing storage in the critical user data by default and then let users whitelist just the sites they want. This could be through an evil dialog, or just a less intrusive indicator somewhere - the website itself would be able to detect that it couldn't save and warn the user in whatever way is most appropriate. This seems to me a better idea than having multiple storage areas (SessionStorage, CachedStorage and FileStorage as suggested by Brady). However this could lead to even more evil dialogs: Do you want to save this data? Is it important? How important is it? The user - and for that matter, the app or UA - doesn't necessarily know how critical a piece of data is. The user doesn't know because without some form of notification they won't know what the lifetime of that data is (and even if they do they will have to know how that lifetime impacts on app functionality). The UA doesn't know because it doesn't understand the nature of the data without the user telling it. The app doesn't necessarily know because it can't see the wider implications of saving the data - storage space on the machine etc. Catch 22. So, to what extent do people think that automatic decisions could be made by the UA and app regarding the criticality of a particular piece of data? The more the saving of data can be automated - with the right level of importance attached to it - the better, saving obtrusive and potentially confusing dialogs, and (hopefully) saving the right data in the right way. Perhaps UAs could notify apps of the storage space available and user preferences on the saving of data up front, helping the app and UA to make reasonable decisions, only asking for user confirmation where an reasonable automatic decision can't be made. It's a head-twister, this one. Chris This message has been scanned for malware by SurfControl plc. www.surfcontrol.com
Re: [whatwg] Web Storage: apparent contradiction in spec
I'm one of the lurking web application developers. This discussion has gone mostly how I was hoping, but since 'offline apps' has come up again, I might bootstrap that a bit. Below are some use cases, but I'll summarize my perspective briefly. 1. To run Doom requiring 500M of localStorage sounds like an 'application'--both users and developers currently have the expectation that users have to approve something like that before being installed to their computers to consume possibly valuable resources. A preliminary prompt to the user sounds reasonable and expected here. 2. Once, I've approved of an application to store data locally, that data could very easily be incredibly precious to both user and developer. Neither would want that data to be swept away by a garbage-collecting browser without user/developer action. 3. If the developer is presented with an unprompted option to store data permanently or just have it 'cached'--I'm not sure why the developer (both the nefarious and the good) would ever opt for the only-cached version. 4. We'd like either (OffTopic) a. localStorage to work for file:// AND/OR b. there to be some way for someone with file access to change the manifest file cache in offline applications (while preserving SOME USE CASES -- Where I work (http://ccnmtl.columbia.edu), we have two main use cases for offline applications, which would be great if they were accomodated in the spec and even more importantly, browsers :-) 1. Static deployment as an 'offline app' http://ccnmtl.columbia.edu/portfolio/medicine_and_health/masivukeni.html Requirements/Constraints: * We were most familiar with programming and deploying web applications, so we preferred to use html/js (and python for backend to content development) * The application was to be run on a laptop which could not (and should not) connect to the Internet while being used. * Data (including user accounts) is then maintained using localStorage. * Rare updates (for critical bugs, e.g.) would not require online access again. Experience/Lessons: * We deployed this with Firefox's legacy globalStorage (before ffox3.5 came out) and static html files accessed at file:// * As written, it no longer works in ffox 3.5 or even with updates to ffox 3.0, because whereas file://localhost/ once worked (and thus globalStorage['localhost']), but seems to have been removed. Thankfully, as deployed, it's not connected to the Internet to receive updates :-) * If we implemented it again (which we will!) for HTML5, we would have the laptop download the files as an offline app with a manifest. Then localStorage would (hopefully) still work, and work upon visiting the domain--even offline. However, this might cause problems in updating the cached files without bringing the computer back online. 2. Deprecating experience for unprivileged users. * We have some services which allow annotation of primary sources (video/audio/images). * Some of these sites have public content, but we do not have the resources or interest in storing annotation/analysis from other users. Nonetheless, we've considered offline storage as a way to allow non-privileged users to analyze the content and store their data locally--to be saved for their own benefit and possibly later to be uploaded or shared elsewhere. In concluding, many thanks to all those that are working on HTML5 (esp. the browser vendors). It's an exciting time to be a web developer. cheers, Schuyler Message: 1 Date: Wed, 26 Aug 2009 20:54:12 -0700 From: Drew Wilson atwil...@google.com To: Linus Upson li...@google.com Cc: Brady Eidson beid...@apple.com, WHATWG List wha...@whatwg.org, Jeremy Orlow jor...@chromium.org, Jens Alfke s...@google.com, Aaron Boodman a...@google.com Subject: Re: [whatwg] Web Storage: apparent contradiction in spec Message-ID: f965ae410908262054y7d405cbdq2f7b115c7b694...@mail.gmail.com Content-Type: text/plain; charset=iso-8859-1 This is one of those times when I *really* wish that the application developer community was more active on this list. I absolutely understand Linus' point of view, but I also feel like we are really hamstringing applications when we make choices like this and I wish that those developers were more vocally represented in these types of discussions. Going down this path would basically kill the ability to have offline web applications, because there would be no guarantees that the data would persist until the user comes back online. But since that point's already been made several times, I guess it's not a compelling argument. -atw On Wed, Aug 26, 2009 at 8:23 PM, Linus Upson li...@google.com wrote: I simply want clicking on links to be safe. In a previous thread I wrote safe and stateless but I'm coming to the opinion that stateless is a corollary of safe. Clicking on links shouldn't, either by filling my disk or hitting my global quota, someday lead to a dialog
Re: [whatwg] Web Storage: apparent contradiction in spec
Adrian Sutton wrote: On 27/08/2009 15:47, Maciej Stachowiak m...@apple.com wrote: - Cached for convenience - discarding this will affect performance but not functionality. - Useful for offline use - discarding this will prevent some data from being accessed when offline. - Critical for offline use - discarding this will prevent the app storing this data from working offline at all. - Critical user data - discarding this will lead to permanent user data loss. The only catch being that if the web app decides this for itself, a malicious script or tracking cookie will be marked as critical user data when in fact the user would disagree. On the plus side, it would mean a browser could default to not allowing storage in the critical user data by default and then let users whitelist just the sites they want. This could be through an evil dialog, or just a less intrusive indicator somewhere the website itself would be able to detect that it couldn¹t save and warn the user in whatever way is most appropriate. I don't fancy having to explain to my Mum that she has to go through some complex (to her) sequence of operations to see if a site is storing her important data somewhere where it might be deleted or in some secure area. Nor do I fancy explaining the procedure for changing between one and the other. I don't really see how the site could help either. I guess it might be possible for it to put up a your data is stored in a non-persistent way message, but instructions to change to persistent storage would have to be per-browser and possibly per browser version; no good for the people who don't know the difference between the browser, the internet and google. I can't imagine how to make this simple enough for end users without all data being persistent by default. Even then, knowing how to clear out data once the quota is hit is likely to be difficult and confusing.
Re: [whatwg] Web Storage: apparent contradiction in spec
And to confound the problem further, UAs dont have meta-data on hand with which to relate various pieces of local data together and attribute them to a specific user-identifiable 'application'. Everything is bound to a security-origin, but that doesn't clearly identify or label an 'application'. On Thu, Aug 27, 2009 at 8:10 AM, Chris Taylor chris.tay...@figureout.comwrote: Adrian Sutton said: On 27/08/2009 15:47, Maciej Stachowiak m...@apple.com wrote: - Cached for convenience - discarding this will affect performance but not functionality. - Useful for offline use - discarding this will prevent some data from being accessed when offline. - Critical for offline use - discarding this will prevent the app storing this data from working offline at all. - Critical user data - discarding this will lead to permanent user data loss. The only catch being that if the web app decides this for itself, a malicious script or tracking cookie will be marked as critical user data when in fact the user would disagree. On the plus side, it would mean a browser could default to not allowing storage in the critical user data by default and then let users whitelist just the sites they want. This could be through an evil dialog, or just a less intrusive indicator somewhere - the website itself would be able to detect that it couldn't save and warn the user in whatever way is most appropriate. This seems to me a better idea than having multiple storage areas (SessionStorage, CachedStorage and FileStorage as suggested by Brady). However this could lead to even more evil dialogs: Do you want to save this data? Is it important? How important is it? The user - and for that matter, the app or UA - doesn't necessarily know how critical a piece of data is. The user doesn't know because without some form of notification they won't know what the lifetime of that data is (and even if they do they will have to know how that lifetime impacts on app functionality). The UA doesn't know because it doesn't understand the nature of the data without the user telling it. The app doesn't necessarily know because it can't see the wider implications of saving the data - storage space on the machine etc. Catch 22. So, to what extent do people think that automatic decisions could be made by the UA and app regarding the criticality of a particular piece of data? The more the saving of data can be automated - with the right level of importance attached to it - the better, saving obtrusive and potentially confusing dialogs, and (hopefully) saving the right data in the right way. Perhaps UAs could notify apps of the storage space available and user preferences on the saving of data up front, helping the app and UA to make reasonable decisions, only asking for user confirmation where an reasonable automatic decision can't be made. It's a head-twister, this one. Chris This message has been scanned for malware by SurfControl plc. www.surfcontrol.com
Re: [whatwg] Web Storage: apparent contradiction in spec
On Thu, Aug 27, 2009 at 10:47 AM, Maciej Stachowiakm...@apple.com wrote: That makes sense to me. There might even be more than two categories: - Cached for convenience - discarding this will affect performance but not functionality. - Useful for offline use - discarding this will prevent some data from being accessed when offline. - Critical for offline use - discarding this will prevent the app storing this data from working offline at all. - Critical user data - discarding this will lead to permanent user data loss. I don't see the utility in this level of granularity. Two levels seem fine: delete at will as long as the app isn't actually running, and never delete once stored without explicit user action. On Thu, Aug 27, 2009 at 10:52 AM, Adrian Suttonadrian.sut...@ephox.com wrote: The only catch being that if the web app decides this for itself, a malicious script or tracking cookie will be marked as critical user data when in fact the user would disagree. Dispose when you feel like it storage doesn't provide any better protection for the user -- the site can always reset it regularly when the user visits. Cookies are already widely used for tracking, and work fine. Flash localStorage is used for tracking mainly because users don't know how to clear it, I suspect, not because it's more persistent than cookies in practice. (One of the many reasons HTML 5 needs to kill Flash.) On Thu, Aug 27, 2009 at 11:10 AM, Chris Taylorchris.tay...@figureout.com wrote: This seems to me a better idea than having multiple storage areas (SessionStorage, CachedStorage and FileStorage as suggested by Brady). However this could lead to even more evil dialogs: Do you want to save this data? Is it important? How important is it? The user - and for that matter, the app or UA - doesn't necessarily know how critical a piece of data is. The decision needs to be made by the app. The app knows whether it's just doing caching, or needs persistence. The UA doesn't know, and nor does the user. I expect that as with cookies, the default in most browsers would be to let the app decide how long it wants to store the data with no user interaction, but privacy-aware users would be able to get more control. I also think it's clear that localStorage would not persist after things like exiting privacy mode, or logging out of a public terminal (if correctly configured), or clicking a clear private data button (if that also cleared bookmarks/history/etc.). Nobody's saying store forever, just don't delete except by explicit user request. So, to what extent do people think that automatic decisions could be made by the UA and app regarding the criticality of a particular piece of data? I think criticality is the wrong word to use. Persistence is better. Persistent data may be unimportant, and transient data may be critical.
Re: [whatwg] Web Storage: apparent contradiction in spec
On Thu, Aug 27, 2009 at 11:14 AM, Schuyler Duveenwha...@graffitiweb.org wrote: 1. To run Doom requiring 500M of localStorage sounds like an 'application'--both users and developers currently have the expectation that users have to approve something like that before being installed to their computers to consume possibly valuable resources. A preliminary prompt to the user sounds reasonable and expected here. I expect 500 MB will require a prompt by anyone's standards, yes, unless maybe the user has a terabyte of free disk space. 3. If the developer is presented with an unprompted option to store data permanently or just have it 'cached'--I'm not sure why the developer (both the nefarious and the good) would ever opt for the only-cached version. For the same reason developers write files in /tmp. In concluding, many thanks to all those that are working on HTML5 (esp. the browser vendors). It's an exciting time to be a web developer. Indeed!
Re: [whatwg] Web Storage: apparent contradiction in spec
Aryeh Gregor wrote: On Thu, Aug 27, 2009 at 11:14 AM, Schuyler Duveenwha...@graffitiweb.org wrote: 1. To run Doom requiring 500M of localStorage sounds like an 'application'--both users and developers currently have the expectation that users have to approve something like that before being installed to their computers to consume possibly valuable resources. A preliminary prompt to the user sounds reasonable and expected here. I expect 500 MB will require a prompt by anyone's standards, yes, unless maybe the user has a terabyte of free disk space. 3. If the developer is presented with an unprompted option to store data permanently or just have it 'cached'--I'm not sure why the developer (both the nefarious and the good) would ever opt for the only-cached version. For the same reason developers write files in /tmp. If it's user-specific, then why not just use sessionStorage? If it's non-user specific (e.g. defaults) or cachable user-data, why not have it cached with normal HTTP as a file, with an XHR (with allow caching headers)? cheers, Schuyler In concluding, many thanks to all those that are working on HTML5 (esp. the browser vendors). It's an exciting time to be a web developer. Indeed!
Re: [whatwg] Web Storage: apparent contradiction in spec
On Thu, Aug 27, 2009 at 11:53 AM, Schuyler Duveenwha...@graffitiweb.org wrote: If it's user-specific, then why not just use sessionStorage? Hmm. You're right, there's a lot of overlap there. sessionStorage might be sufficient for caching, for most applications. If not, localStorage could be used and managed manually. I retract my opinion on the usefulness of two different types of localStorage; I'll defer to the developers of actual web apps here.
Re: [whatwg] Web Storage: apparent contradiction in spec
On Aug 27, 2009, at 7:47 AM, Maciej Stachowiak wrote: On Aug 26, 2009, at 4:51 PM, Jens Alfke wrote: To repeat what I said up above: Maybe the local storage API needs a way to distinguish between cached data that can be silently thrown away, and important data that can't. That makes sense to me. There might even be more than two categories: - Cached for convenience - discarding this will affect performance but not functionality. - Useful for offline use - discarding this will prevent some data from being accessed when offline. - Critical for offline use - discarding this will prevent the app storing this data from working offline at all. - Critical user data - discarding this will lead to permanent user data loss. I agree with Maciej's 4-level distinction on philosophical grounds, and think it's a fine list of use cases. But I think there's been a reasonable amount of agreement on this list that it is unnecessarily fine grained. A developer who is consciously choosing a cache will always choose the most aggressive cache, and a developer who is consciously choose file storage will always choose the most sacred file storage. So we're left with the cache vs file distinction once more. All browser vendors who have implemented LocalStorage are willing to implement the cache, because what they've done either meets or exceeds the cache use-case. The remaining question is the file storage. How do we implement this distinction? I don't like the idea of having different modes on LocalStorage. How would the different mode be triggered? How would it be managed? What happens when two applications from the same security origin try to mix modes? Different modes just makes what is already a dirt simple API more complex, makes implementation more difficult for browser vendors, and confuses web developers. So I resubmit my three-Storage-object solution: SessionStorage, CacheStorage, and FileStorage. From this discussion, it appears that FileStorage is something Google might not be willing to implement. That's fine! They can have the object available to scripts but just give it a zero quota. To be more friendly to developers and not force them into checking abilities by catching exceptions we could add one more property to the storage interface so they can check ahead of time whether their attempt to store data will fail. Web developers would then have the ability to make the conscious decision of Is a cache good enough? and fallback to CacheStorage, or decide No, I really need persistent data and fallback to Flash or some other plug-in. The interfaces are all so similar as to be pretty painless for the developer. Thoughts? ~Brady
Re: [whatwg] Web Storage: apparent contradiction in spec
I don't think there is consensus at Google yet. I'm not saying that UAs shouldn't provide file-like lifetime semantics for storage. I'm just saying the user should decide, not the web page. Here's one way such a thing could be achieved: input type=storage src=button.png quota=20GB / When the user clicks the button they see a dialog that mail.google.com would like to use 20GB of storage. You have 50GB of free space. [Yes] [No]. Script can't cause the dialog to appear, only a user action. There would also be some affordance in that dialog to allow the user to manage persistent storage from other domains. A small Other sites are using 2GB of storage link perhaps. AppCache, LocalStrorage, and all other persistent bits for that domain would live within this quota. UAs would take this user action as a strong signal that the data is valuable and would act accordingly. If web sites use LocalStorage, AppCache, et. al. without the user clicking on and accepting a storage input button, then the UA would be free to garbage collect as it sees fit. Good UAs would do a good job of not throwing away things that are important to the user, just as they do today with cookies. Linus On Thu, Aug 27, 2009 at 9:42 AM, Brady Eidson beid...@apple.com wrote: On Aug 27, 2009, at 7:47 AM, Maciej Stachowiak wrote: On Aug 26, 2009, at 4:51 PM, Jens Alfke wrote: To repeat what I said up above: *Maybe the local storage API needs a way to distinguish between cached data that can be silently thrown away, and important data that can't.* That makes sense to me. There might even be more than two categories: - Cached for convenience - discarding this will affect performance but not functionality. - Useful for offline use - discarding this will prevent some data from being accessed when offline. - Critical for offline use - discarding this will prevent the app storing this data from working offline at all. - Critical user data - discarding this will lead to permanent user data loss. I agree with Maciej's 4-level distinction on philosophical grounds, and think it's a fine list of use cases. But I think there's been a reasonable amount of agreement on this list that it is unnecessarily fine grained. A developer who is consciously choosing a cache will always choose the most aggressive cache, and a developer who is consciously choose file storage will always choose the most sacred file storage. So we're left with the cache vs file distinction once more. All browser vendors who have implemented LocalStorage are willing to implement the cache, because what they've done either meets or exceeds the cache use-case. The remaining question is the file storage. How do we implement this distinction? I don't like the idea of having different modes on LocalStorage. How would the different mode be triggered? How would it be managed? What happens when two applications from the same security origin try to mix modes? Different modes just makes what is already a dirt simple API more complex, makes implementation more difficult for browser vendors, and confuses web developers. So I resubmit my three-Storage-object solution: SessionStorage, CacheStorage, and FileStorage. From this discussion, it appears that FileStorage is something Google might not be willing to implement. That's fine! They can have the object available to scripts but just give it a zero quota. To be more friendly to developers and not force them into checking abilities by catching exceptions we could add one more property to the storage interface so they can check ahead of time whether their attempt to store data will fail. Web developers would then have the ability to make the conscious decision of Is a cache good enough? and fallback to CacheStorage, or decide No, I really need persistent data and fallback to Flash or some other plug-in. The interfaces are all so similar as to be pretty painless for the developer. Thoughts? ~Brady
Re: [whatwg] Web Storage: apparent contradiction in spec
On Wed, Aug 26, 2009 at 8:23 PM, Linus Upsonli...@google.com wrote: I simply want clicking on links to be safe. In a previous thread I wrote safe and stateless but I'm coming to the opinion that stateless is a corollary of safe. Clicking on links shouldn't, either by filling my disk or hitting my global quota, someday lead to a dialog which reads, Please choose what to delete so that web sites will continue to work. The candidate delete list will be thousands long and hidden in that haystack will be a few precious needles. I see what you're saying. If the web is able to use local storage and UAs aren't allowed to prune it, then the storage used by web browsers tends to grow forever. If you want the web to continue to be completely stateless, there is no way around this. On Thu, Aug 27, 2009 at 10:18 AM, Erik Kayerik...@google.com wrote: I also think this thread is jumping to hyperbole about how the UA would be deleting data arbitrarily. Reasonable cache eviction algorithms won't cause much of a problem here. Do you use offline email all of the time? Guess what? It won't be evicted. Did you recently make change to an offline document that you didn't sync? It won't be evicted. The things that are problematic are the things which are accessed infrequently yet only exist in this local storage. See my argument above for why I think local storage isn't a great solution to offline only data anyway. I agree you could get 95% of the way there. But I can prove that this problem will happen in the real world. All I have to do is have a user with a small enough disk (an iPhone) and enough applications storing precious data. No amount of heuristics will stop the UA from either accidentally deleting something the user wanted, or asking the user to choose something to delete in this case. On Thu, Aug 27, 2009 at 10:18 AM, Erik Kayerik...@google.com wrote: What kinds of applications do people write against non-prompted Flash storage? As far as I can tell, it's used as a way to store cookies that are less likely to be deleted. It's not for data that the user would define as precious. How do I get data out of Flash storage? How do I back it up? Nobody does because they don't care and there isn't a use case driving this need today. The data isn't precious. Sure, but we're only having this conversation at all because we believe that the existing technology isn't meeting some use cases. People want to build offline applications based on web technology, and those apps tend to want to store data persistently and have it be treated as precious. If you accept this use case exists, and you accept that it is not possible for a UA to automatically make this decision, then we're just left with how to get the user involved. I think we are seeing, again, that there needs to be a difference between installed apps built on the web platform and the web as it is today. The web we know and love is stateless and should stay that way. Installed apps have state, and have to be actively managed by users. In that world, I think that the spec should be silent on what promises the UA makes wrt the data. It will be client-type-dependent. - a
Re: [whatwg] Web Storage: apparent contradiction in spec
There should also be a way to ask for more quota (from the user) without losing user data. The API via a form element is a little odd--generally forms are for submitting information to the site. Historically, all of these kinds of things are done via javascript: * cookies * opensearch additions * extension additions This would also have the benefit of allowing a Worker() to request quota without something on the page. cheers, Schuyler Linus Upson wrote: I don't think there is consensus at Google yet. I'm not saying that UAs shouldn't provide file-like lifetime semantics for storage. I'm just saying the user should decide, not the web page. Here's one way such a thing could be achieved: input type=storage src=button.png quota=20GB / When the user clicks the button they see a dialog that mail.google.com http://mail.google.com would like to use 20GB of storage. You have 50GB of free space. [Yes] [No]. Script can't cause the dialog to appear, only a user action. There would also be some affordance in that dialog to allow the user to manage persistent storage from other domains. A small Other sites are using 2GB of storage link perhaps. AppCache, LocalStrorage, and all other persistent bits for that domain would live within this quota. UAs would take this user action as a strong signal that the data is valuable and would act accordingly. If web sites use LocalStorage, AppCache, et. al. without the user clicking on and accepting a storage input button, then the UA would be free to garbage collect as it sees fit. Good UAs would do a good job of not throwing away things that are important to the user, just as they do today with cookies. Linus On Thu, Aug 27, 2009 at 9:42 AM, Brady Eidson beid...@apple.com mailto:beid...@apple.com wrote: On Aug 27, 2009, at 7:47 AM, Maciej Stachowiak wrote: On Aug 26, 2009, at 4:51 PM, Jens Alfke wrote: To repeat what I said up above: *Maybe the local storage API needs a way to distinguish between cached data that can be silently thrown away, and important data that can't.* That makes sense to me. There might even be more than two categories: - Cached for convenience - discarding this will affect performance but not functionality. - Useful for offline use - discarding this will prevent some data from being accessed when offline. - Critical for offline use - discarding this will prevent the app storing this data from working offline at all. - Critical user data - discarding this will lead to permanent user data loss. I agree with Maciej's 4-level distinction on philosophical grounds, and think it's a fine list of use cases. But I think there's been a reasonable amount of agreement on this list that it is unnecessarily fine grained. A developer who is consciously choosing a cache will always choose the most aggressive cache, and a developer who is consciously choose file storage will always choose the most sacred file storage. So we're left with the cache vs file distinction once more. All browser vendors who have implemented LocalStorage are willing to implement the cache, because what they've done either meets or exceeds the cache use-case. The remaining question is the file storage. How do we implement this distinction? I don't like the idea of having different modes on LocalStorage. How would the different mode be triggered? How would it be managed? What happens when two applications from the same security origin try to mix modes? Different modes just makes what is already a dirt simple API more complex, makes implementation more difficult for browser vendors, and confuses web developers. So I resubmit my three-Storage-object solution: SessionStorage, CacheStorage, and FileStorage. From this discussion, it appears that FileStorage is something Google might not be willing to implement. That's fine! They can have the object available to scripts but just give it a zero quota. To be more friendly to developers and not force them into checking abilities by catching exceptions we could add one more property to the storage interface so they can check ahead of time whether their attempt to store data will fail. Web developers would then have the ability to make the conscious decision of Is a cache good enough? and fallback to CacheStorage, or decide No, I really need persistent data and fallback to Flash or some other plug-in. The interfaces are all so similar as to be pretty painless for the developer. Thoughts? ~Brady
Re: [whatwg] Web Storage: apparent contradiction in spec
On Wed, Aug 26, 2009 at 8:23 PM, Linus Upsonli...@google.com wrote: The candidate delete list will be thousands long and hidden in that haystack will be a few precious needles. While that is certainly one of the outcomes, and I agree a bad one, I am not sure that the user experience needs to be that bleak. Further, I expect that most UAs will need some dialog like this to let users manage the data in some cases anyway -- if users can grant permissions, they'll need to be able to revoke them, etc. I think that there is a lot of data that UAs can use to present likely targets for the user, as infrequently used desktop icons or similar endeavour to do. Bookmarked sites would be advantaged, as would sites that have had their stored data read recently (or have an active cookie, history frequency data, etc.). I think highly of myself and my team, to be sure, but I don't think that we are really going to always know better than the user -- even a technically naive user -- what they consider to be important data. Mike
Re: [whatwg] Web Storage: apparent contradiction in spec
On Thu, Aug 27, 2009 at 10:33 AM, Linus Upsonli...@google.com wrote: I don't think there is consensus at Google yet. I'm not saying that UAs shouldn't provide file-like lifetime semantics for storage. I'm just saying the user should decide, not the web page. Linus, are you only considering traditional in-browser web apps that don't go through an install process of any sort (i.e., I go to a URL in my web browser)? Would you hold a different opinion for different types of HTML5-based applications (like the TiddlyWiki app that gets copied locally)? Or would you require that all apps go through an install process that acquires quota, permissions, etc.? It seems like others are objecting to your position based on different application models than the ones you have in mind. -- Dirk
Re: [whatwg] Web Storage: apparent contradiction in spec
My concerns are around browser UAs. AIR, Dashboard, XULRunner, Extensions, etc. can have different policies. I simply want clicking on links in my browser to be safe. Linus On Thu, Aug 27, 2009 at 5:53 PM, Dirk Pranke dpra...@chromium.org wrote: On Thu, Aug 27, 2009 at 10:33 AM, Linus Upsonli...@google.com wrote: I don't think there is consensus at Google yet. I'm not saying that UAs shouldn't provide file-like lifetime semantics for storage. I'm just saying the user should decide, not the web page. Linus, are you only considering traditional in-browser web apps that don't go through an install process of any sort (i.e., I go to a URL in my web browser)? Would you hold a different opinion for different types of HTML5-based applications (like the TiddlyWiki app that gets copied locally)? Or would you require that all apps go through an install process that acquires quota, permissions, etc.? It seems like others are objecting to your position based on different application models than the ones you have in mind. -- Dirk
Re: [whatwg] Web Storage: apparent contradiction in spec
On Wed, 26 Aug 2009 00:59:31 +0200, Aaron Boodman a...@google.com wrote: On Tue, Aug 25, 2009 at 3:51 PM, Jeremy Orlowjor...@chromium.org wrote: I still don't understand what use local storage has outside of 'cloud storage'. Even in the extensions use case (which I think is out of scope for this spec), there's no reason you can't sync user preferences and such to the cloud. The use case, though, is local storage, not cloud storage. Requiring cloud storage here kind of defeats the purpose and makes the API a lot harder to use. Indeed. It would be nice to be able to write simple applications that do not require the cloud at all and basically consist of a set of static documents distributed over HTTP. -- Anne van Kesteren http://annevankesteren.nl/
Re: [whatwg] Web Storage: apparent contradiction in spec
On Aug 26, 2009, at 3:10 AM, Anne van Kesteren wrote: Indeed. It would be nice to be able to write simple applications that do not require the cloud at all and basically consist of a set of static documents distributed over HTTP. TiddlyWiki is a perfect example of this, if anyone's looking for real- world examples. It's a single-user local wiki distributed as an HTML file. It's always had the problem of how to save changes persistently — the basic mechanism is really awkward, requiring you to manually export a new HTML file including the content as embedded JS data. It would be great for it to use local storage (and maybe they've added that since I last looked at it.) Needless to say, some GTD acolytes accumulate tons of crucial personal data in local wikis like this (or VoodooPad) and would be furious if the browser deleted it without their consent. I think there are lots of other use cases for web apps that would use local storage without ever syncing it to the cloud, simply because it's vastly easier to do. I can write such an app and host it on my cheapo personal website without having to worry about user account registration, storage and bandwidth costs for an unknown number of documents, tech support or legal liability if my server loses people's data, privacy requirements, DMCA takedown requests, etc. etc. —Jens
Re: [whatwg] Web Storage: apparent contradiction in spec
As an offline cloud-app developer, I've got these three kinds of data:1) Data from the cloud that's cached locally on a best-effort basis, that can be thrown out if we start running out of space. 2) Data from the cloud that the app needs in order to keep functioning at all. 3) Locally created data that may not have been synced to the cloud yet, like emails written on an airplane. Even if the app stops working, this data is still valuable and recovering it is important. Once it is synced, it can usually be downgraded to 'best-effort'. In order for apps to make the best use of the space they have available, especially if space is tight, it'd be great to have an API that indicates how big the app's quota is, and how much space it's taking up now. Then the app code can decide what data to keep and what data to drop, in case it approaches the max. Apps that sync a portion of a larger cloud-based datastore to local disk can also use the local quota info to help decide what to sync. If the UA lets the user adjust the quotas, it'd be nice to expose a quota-max-changed event, so polling won't be necessary. On Tue, Aug 25, 2009 at 3:59 PM, Aaron Boodman a...@google.com wrote: On Tue, Aug 25, 2009 at 3:51 PM, Jeremy Orlowjor...@chromium.org wrote: I still don't understand what use local storage has outside of 'cloud storage'. Even in the extensions use case (which I think is out of scope for this spec), there's no reason you can't sync user preferences and such to the cloud. The use case, though, is local storage, not cloud storage. Requiring cloud storage here kind of defeats the purpose and makes the API a lot harder to use. - a
Re: [whatwg] Web Storage: apparent contradiction in spec
On 26/08/2009 18:36, Jens Alfke s...@google.com wrote: I think there are lots of other use cases for web apps that would use local storage without ever syncing it to the cloud, simply because it's vastly easier to do. I can write such an app and host it on my cheapo personal website without having to worry about user account registration, storage and bandwidth costs for an unknown number of documents, tech support or legal liability if my server loses people's data, privacy requirements, DMCA takedown requests, etc. etc. On the other hand, look at it from a user's point of view - where is the data on disk? How do I back it up? The browser's local storage is very opaque and hidden to the user so it's not a safe place to save important data since it's unlikely to be thought of when it comes to backing up. We do clearly have two types of local storage data though - one set is a local cache and should be garbage collected automatically if I don't use the site for a while or if space is tight. As a user, I really want that functionality - otherwise I have to either disable local storage and have permission dialogs popping up all the time to make exceptions, or have an ever growing database of rubbish on my drive. On the other hand, there will be a handful of sites where I explicitly decide that the data I create is important. The natural tendency would be to select a location on my disk where the local storage for that webapp would be saved. Then it becomes a visible, manageable file on my disk. Devices like the iPhone where there's no file management, would probably just have a flag to say whether it is permanent or not. I'm not sure how this fits into what the spec says of course, but from a user's perspective, I don't want to lose important data, but I also don't want to keep every bit of local storage forever. The spec really needs to allow both, but have a flag to tell the application which mode is being used. Regards, Adrian Sutton. __ Adrian Sutton, CTO UK: +44 1 628 200 182 x481 US: +1 (650) 292 9659 x717 Ephox http://www.ephox.com/ Ephox Blogs http://planet.ephox.com/, Personal Blog http://www.symphonious.net/
Re: [whatwg] Web Storage: apparent contradiction in spec
On Wed, Aug 26, 2009 at 4:01 PM, Linus Upson li...@google.com wrote: The analogy was made comparing a user agent that purges local storage to an OS throwing out files without explicit user action. This is misleading since most files arrive on your computer's disk via explicit user action. You copy files to your disk by downloading them from the internet, copying from a network drive, from a floppy, your camera, etc. You put them on your disk and you are responsible for removing them to reclaim space. There are apps that create files in hidden places such as: C:\Documents and Settings\linus\Local Settings\Application Data\Google\Chrome\User Data If those apps do not manage their space carefully, users get annoyed. If such an app filled the user's disk they would have no idea what consumed the space or how to reclaim it. They didn't put the files there. How are they supposed to know to remove them? Most users have no idea that Local Settings exists (it is hidden), much less how to correctly manage any files they find. This seems like an argument for ensuring web apps have as much ability to take reasonable steps to control their space usage as local apps do, not an argument that the UA should be able to discard those files. After all, you are not arguing that Windows should be able to throw away those non-user-visible files in Local Storage. Without automatic space management the local storage consumed will grow without bound. I'm concerned that even without an intentional DOS attack users are going to be unhappy about their shrinking disks and not know what to do about it. The problem is worse on phones. I don't think anyone is suggesting UAs should not have the ability to control the total space usage, e.g. by presetting per-app and global quotas. That's not the same as saying that the UA can throw away data after the fact. Things get worse still if a griefer wants to make a point about the importance of keeping web browsers logically stateless. Here's how such an attack could be carried out: 2a. Acquire a bunch of unrelated domains from a bunch of registrars using stolen credit cards. Skip this step if UAs don't group subdomains under the same storage quota. For extra credit pick names that are similar to legitimate sites that use local storage. 2b. Start up some web hosting accounts. Host your attack code here. If they aren't free, use stolen credit cards. 2c. Buy ads from a network that subsyndicates from a network that subsyndicates from a major ad network that allows 3rd party ad serving. There are lots to choose from. No money? Stolen credit cards. Serve the ads from your previously acquired hosting accounts. 2d. Giggle. The user will be faced with the choice of writing off the space, deleting everything including their precious data, or carefully picking though tens of thousands of entries to find the few domains that hold precious content. User gets really unhappy if the attack managed to fill the disk. I'm not sure why this is more compelling for a griefer than the existing attack (along similar lines) they can already make against the cookie store to blow away 100% of the user's cookies, and keep doing it, such that the user can never log in anywhere. In fact, to some degree that's a testimony that treating things like cookies doesn't mean users will be free from griefing. In practice I don't foresee either of these happening unless doing so allows attackers monetary gain. Chrome's Incognito mode creates a temporary, in-memory profile. Local storage operations will work, but nothing will be saved after the Incognito window is closed. Safari takes a different approach and causes local storage operations to fail when in Private Browsing mode. Some sites won't work in Private Browsing. I don't recall what Firefox or IE do. Pick your poison. This is a problem that has to be solved regardless, and it doesn't seem like a bad one. If the purpose of section 6.1 is to state that UAs must give users the ability to see and clean up their Local Storage data (which seems to me like a good idea but outside the scope of what HTML5 should be specifying), then users have the ability to manually delete this data anyway, and live with the consequences. Chrome's behavior is akin to a user manually clearing his Local Storage data, and Safari's is akin to an app hitting its quota. Apps have to be able to deal with both anyway, perhaps with a (possibly large) reduction in functionality. While that may sound like an argument for your position (since I am saying that apps need to deal with disappearing Local Storage data), the critical difference is that the user is in control of this, either by cleaning up the data manually or by electing to use a private browsing mode. Thus I don't think it justifies some UA behavior when the user is _not_ in explicit control. If the spec requires UAs to maintain local storage as 'precious' it will be the first such
Re: [whatwg] Web Storage: apparent contradiction in spec
At a minimum the HTML 5 spec should be silent on how user agents implement local storage policies. +1 linus The sort of 'policies' being discussed are feeling 'out-of-scope' for the HTML5 spec. The practical realities are that eviction needs to be there in some form w/o an explicit user 'installation' step. Something that could be appropriately specified is the grain size of local data eviction... * Can an individual key,value pair for an origin be removed from local storage while leaving other local data in place? * Can an individual Database be deleted for an origin while leaving other local data in place? * Can an individual Manifest be deleted for an origin while leaving other local data in place? * Or should an origin's data be subject to all-or-none eviction. I would prefer to see the spec clarify questions along those lines. That would be useful. A mandate from on high that says 'shall store forever and ever' will be promptly ignored because its impossible to make that guarantee. On Wed, Aug 26, 2009 at 4:01 PM, Linus Upson li...@google.com wrote: Not convinced. :) 1. Analogies The analogy was made comparing a user agent that purges local storage to an OS throwing out files without explicit user action. This is misleading since most files arrive on your computer's disk via explicit user action. You copy files to your disk by downloading them from the internet, copying from a network drive, from a floppy, your camera, etc. You put them on your disk and you are responsible for removing them to reclaim space. There are apps that create files in hidden places such as: C:\Documents and Settings\linus\Local Settings\Application Data\Google\Chrome\User Data If those apps do not manage their space carefully, users get annoyed. If such an app filled the user's disk they would have no idea what consumed the space or how to reclaim it. They didn't put the files there. How are they supposed to know to remove them? Most users have no idea that Local Settings exists (it is hidden), much less how to correctly manage any files they find. A better analogy would be, What if watching TV caused 0-5MB size files to silently be created from time to time in a hidden folder on your computer, and when your disk filled up both your TV and computer stopped working? Lengthy discussion on cleaning up hidden resources (persistent background content) here: http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2009-July/021421.html 2. Attack Without automatic space management the local storage consumed will grow without bound. I'm concerned that even without an intentional DOS attack users are going to be unhappy about their shrinking disks and not know what to do about it. The problem is worse on phones. Things get worse still if a griefer wants to make a point about the importance of keeping web browsers logically stateless. Here's how such an attack could be carried out: 2a. Acquire a bunch of unrelated domains from a bunch of registrars using stolen credit cards. Skip this step if UAs don't group subdomains under the same storage quota. For extra credit pick names that are similar to legitimate sites that use local storage. 2b. Start up some web hosting accounts. Host your attack code here. If they aren't free, use stolen credit cards. 2c. Buy ads from a network that subsyndicates from a network that subsyndicates from a major ad network that allows 3rd party ad serving. There are lots to choose from. No money? Stolen credit cards. Serve the ads from your previously acquired hosting accounts. 2d. Giggle. The user will be faced with the choice of writing off the space, deleting everything including their precious data, or carefully picking though tens of thousands of entries to find the few domains that hold precious content. User gets really unhappy if the attack managed to fill the disk. 3. Ingcognito / Private Browsing Chrome's Incognito mode creates a temporary, in-memory profile. Local storage operations will work, but nothing will be saved after the Incognito window is closed. Safari takes a different approach and causes local storage operations to fail when in Private Browsing mode. Some sites won't work in Private Browsing. I don't recall what Firefox or IE do. Pick your poison. Lengthy discussion here: http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2009-April/019238.html 4. Cache Eviction Algorithms At a minimum the HTML 5 spec should be silent on how user agents implement local storage policies. I would prefer the spec to make it clear that local storage is a cache, domains can use up to 5MB of space without interrupting the user, and that UAs were free to implement varying cache eviction algorithms. Some browsers may provide interface to allow users to specify precious local storage, some may not. Eviction policies for installed extensions may be different than those for web pages. Quotas for extensions may be
Re: [whatwg] Web Storage: apparent contradiction in spec
On Wed, Aug 26, 2009 at 4:31 PM, Michael Nordman micha...@google.comwrote: A mandate from on high that says 'shall store forever and ever' will be promptly ignored because its impossible to make that guarantee. That's not the proposed mandate. The proposed mandate is thou shalt not discard successfully-written data without explicit user action, which seems implementable to me. Note that this doesn't make claims like the hard drive will not fail, and it doesn't claim that the UA is required to allow the app to write whatever data it wants in the first place. PK
Re: [whatwg] Web Storage: apparent contradiction in spec
On Aug 26, 2009, at 4:01 PM, Linus Upson wrote: The analogy was made comparing a user agent that purges local storage to an OS throwing out files without explicit user action. This is misleading since most files arrive on your computer's disk via explicit user action. You copy files to your disk by downloading them from the internet, copying from a network drive, from a floppy, your camera, etc. A web app would also be pretty likely to put stuff in local storage as a result of explicit user action. The use cases seem pretty similar. Also, you're not counting files that you create locally. After all, files have to come from somewhere :) Those are the most precious since they're yours and they may not live anywhere else if you haven't backed them up or copied them elsewhere. There's no reason web-apps can't create the same kind of content, and it would look very similar to a user: I go to the word processor [website], click New Document, type some stuff, and click Save. Even if the save process involves migrating the local data up to the cloud, that transition is not instantaneous: it can take arbitrarily large amounts of time if there are network/server problems or the user is offline. During that time, the local storage represents the only copy of the data. There is therefore a serious race condition where, if the browser decides to purge local data before the app has uploaded it, the data is gone forever. A better analogy would be, What if watching TV caused 0-5MB size files to silently be created from time to time in a hidden folder on your computer, and when your disk filled up both your TV and computer stopped working? This is a cache — that isn't the kind of usage I'm concerned about. Maybe the local storage API needs a way to distinguish between cached data that can be silently thrown away, and important data that can't. (For example, the Mac OS has separate 'Caches' and 'Application Support' subfolders of ~/Library/.) First, this is what quotas are for. The TV web-app would have a limited quota of space to cache stuff. Second, the browser should definitely help you delete stuff like this if disk space does get low; I'm just saying it shouldn't delete it silently or as part of some misleading command like Empty Cache or Delete Cookies. At a minimum the HTML 5 spec should be silent on how user agents implement local storage policies. I would prefer the spec to make it clear that local storage is a cache, domains can use up to 5MB of space without interrupting the user, and that UAs were free to implement varying cache eviction algorithms. That will have the effect of making an interesting category of new applications fail, with user data loss, on some browsers. That sounds like a really bad idea to me. To repeat what I said up above: Maybe the local storage API needs a way to distinguish between cached data that can be silently thrown away, and important data that can't. —Jens
Re: [whatwg] Web Storage: apparent contradiction in spec
Ok... I overstated things ;) What seems inevitable are vista-like prompts to allow something (or prods to delete something) seemingly unrelated to a user's interaction with a site... please, oh please, lets avoid making that part of the web platform. I'm assuming that UA will have out-of-band mechanisms to 'bless' certain sites which should not be subject to automated eviction. If push comes to shove, the system could suggest cleaning up one of these 'blessed' sites if inactivity for an extended period was noticed. But for the overwhelming number of sites in a users browsing history, its a different matter. If the storage APIs are just available for use, no questions asked making the storage just go away, no questions asked, is symmetrical. Blessing involves asking questions... making it go away does too. On Wed, Aug 26, 2009 at 4:35 PM, Peter Kasting pkast...@google.com wrote: On Wed, Aug 26, 2009 at 4:31 PM, Michael Nordman micha...@google.comwrote: A mandate from on high that says 'shall store forever and ever' will be promptly ignored because its impossible to make that guarantee. That's not the proposed mandate. The proposed mandate is thou shalt not discard successfully-written data without explicit user action, which seems implementable to me. Note that this doesn't make claims like the hard drive will not fail, and it doesn't claim that the UA is required to allow the app to write whatever data it wants in the first place. PK
Re: [whatwg] Web Storage: apparent contradiction in spec
Maybe the local storage API needs a way to distinguish between cached data that can be silently thrown away, and important data that can't.* * *What if instead of the storage APIs providing a way to distinguish things, UA's provide a way for users to indicate which applications are important, and UA's provide a way for applications guide a user towards making that indication.* *Seems like permissioning, blessing, could happen out-of-band of the existing storage APIs.* * * On Wed, Aug 26, 2009 at 4:51 PM, Jens Alfke s...@google.com wrote: On Aug 26, 2009, at 4:01 PM, Linus Upson wrote: The analogy was made comparing a user agent that purges local storage to an OS throwing out files without explicit user action. This is misleading since most files arrive on your computer's disk via explicit user action. You copy files to your disk by downloading them from the internet, copying from a network drive, from a floppy, your camera, etc. A web app would also be pretty likely to put stuff in local storage as a result of explicit user action. The use cases seem pretty similar. Also, you're not counting files that you *create* locally. After all, files have to come from somewhere :) Those are the most precious since they're yours and they may not live anywhere else if you haven't backed them up or copied them elsewhere. There's no reason web-apps can't create the same kind of content, and it would look very similar to a user: I go to the word processor [website], click New Document, type some stuff, and click Save. Even if the save process involves migrating the local data up to the cloud, that transition is not instantaneous: it can take arbitrarily large amounts of time if there are network/server problems or the user is offline. During that time, *the local storage represents the only copy of the data*. There is therefore a serious race condition where, if the browser decides to purge local data before the app has uploaded it, the data is gone forever. A better analogy would be, What if watching TV caused 0-5MB size files to silently be created from time to time in a hidden folder on your computer, and when your disk filled up both your TV and computer stopped working? This is a cache — that isn't the kind of usage I'm concerned about. Maybe the local storage API needs a way to distinguish between cached data that can be silently thrown away, and important data that can't. (For example, the Mac OS has separate 'Caches' and 'Application Support' subfolders of ~/Library/.) First, this is what quotas are for. The TV web-app would have a limited quota of space to cache stuff. Second, the browser should definitely help you delete stuff like this if disk space does get low; I'm just saying it shouldn't delete it silently or as part of some misleading command like Empty Cache or Delete Cookies. At a minimum the HTML 5 spec should be silent on how user agents implement local storage policies. I would prefer the spec to make it clear that local storage is a cache, domains can use up to 5MB of space without interrupting the user, and that UAs were free to implement varying cache eviction algorithms. That will have the effect of making an interesting category of new applications fail, with user data loss, on some browsers. That sounds like a really bad idea to me. To repeat what I said up above: *Maybe the local storage API needs a way to distinguish between cached data that can be silently thrown away, and important data that can't.* —Jens
Re: [whatwg] Web Storage: apparent contradiction in spec
On Aug 26, 2009, at 4:55 PM, Michael Nordman wrote: What seems inevitable are vista-like prompts to allow something (or prods to delete something) seemingly unrelated to a user's interaction with a site... please, oh please, lets avoid making that part of the web platform. Doesn't Gears already do this? If I do something like enabling local draft storage in WordPress, I get a prompt asking me if I want to allow myblog.com to store some local data on my disk, and I click OK because after all that's what I asked the site to do. I'm assuming that UA will have out-of-band mechanisms to 'bless' certain sites which should not be subject to automated eviction. If this is out-of-spec and browser-dependent, there won't be a good way for an app to request that blessing; it'll be something the user has to know to do, otherwise their data can get lost. That seems dangerous. In most systems user data loss is just about the worst-case scenario of what could go wrong, and you try to prevent it at all costs. My suggestion to have separate 'important' and 'cache' local storage areas would provide such a mechanism in a standard way. The first time an app tried to put stuff in the 'important' area, you'd be asked for approval. And 'important' stores wouldn't be deleted without your consent. —Jens
Re: [whatwg] Web Storage: apparent contradiction in spec
On Thu, Aug 27, 2009 at 1:55 AM, Michael Nordmanmicha...@google.com wrote: Ok... I overstated things ;) What seems inevitable are vista-like prompts to allow something (or prods to delete something) seemingly unrelated to a user's interaction with a site... please, oh please, lets avoid making that part of the web platform. As far as I know, cookies work the same way as the proposed local storage policy: once a cookie is created, the browser won't delete it when space becomes a problem. The site controls the expiration date of the cookie, and it can fill up the entire drive with cookies if it wants to do so. This is all without user interaction. I don't think this has ever been a problem. -- Remco
Re: [whatwg] Web Storage: apparent contradiction in spec
On Wed, Aug 26, 2009 at 5:08 PM, Remco remc...@gmail.com wrote: As far as I know, cookies work the same way as the proposed local storage policy: once a cookie is created, the browser won't delete it when space becomes a problem. The site controls the expiration date of the cookie, and it can fill up the entire drive with cookies if it wants to do so. This is all without user interaction. I don't think this has ever been a problem. This is not at all how cookies work. All UAs have various limits (e.g. a per-host and global limit) and purge cookies silently when those limits are reached. It is exactly this model which Linus is proposing for Local Storage. Cookies disappearing causes problems like users not being logged in when they return to a site, sites forgetting user preferences, and (importantly to publishers) ad tracking not working well. All of these are reasons why various sites now use Flash to store cookies instead/in addition. PK
Re: [whatwg] Web Storage: apparent contradiction in spec
I started writing a detailed rebuttal to Linus's reply, but by the time I was finished, many others had already delivered more targetted replies. So I'll cut the rebuttal format and make a few specific points. - Many apps act as a shoebox for managing specific types of data, and users are used to using these apps to manage that data directly. See iTunes, Windows Media Player, iPhoto, and desktop mail clients as examples. This trend is growing, not waning. Browsers are already a shoebox for history, bookmarks, and other types of data. Claiming that this data is hidden from users who are used to handling obscure file management scenarios and therefore we shouldn't fully respect it is trying to fit in with the past, not trying to make the future better. - No one is suggesting that UAs not have whatever freedom they want in deciding *what* or *how much* to store. We're only suggesting that once the UA has committed to storing it, it *not* be allowed to arbitrarily purge it. - One use of LocalStorage is as a cache for data that is transient and non-critical in nature, or that will live on a server. But another, just-as-valid use of LocalStorage is for persistent, predictable storage in the client UA that will never rely on anything in the cloud. - And on that note, if developers don't have faith that data in LocalStorage is actually persistent and safe, they won't use it. I've given talks on this point 4 times in the last year, and I am stating this as a fact, based on real-world feedback from actual, real- world web developers: If LocalStorage is defined in the standard to be a purgable cache, developers will continue to use what they're already comfortable with, which is Flash's LocalStorage. When a developer is willing to instantiate a plug-in just to reliably store simple nuggets of data - like user preferences and settings - because they don't trust the browser, then I think we've failed in moving the web forward. I truly hope we can sway the LocalStorage is a cache crowd. But if we can't, then I would have to suggest something crazy - that we add a third Storage object. (Note that Jens - from Google - has already loosely suggested this) So we'd have something like: -SessionStorage - That fills the per browsing context role and whose optionally transient nature is already well spec'ed -CachedStorage - That fills Google's interpretation of the LocalStorage role in that it's global, and will probably be around on the disk in the future, maybe -FileStorage - That fills Apple's interpretation of the LocalStorage role in that it's global, and is as sacred as a file on the disk (or a song in your media library, or a photo in your photo library, or a bookmark, or...) The names are just suggestions at this point. ~Brady
Re: [whatwg] Web Storage: apparent contradiction in spec
On Wed, Aug 26, 2009 at 5:08 PM, Remco remc...@gmail.com wrote: On Thu, Aug 27, 2009 at 1:55 AM, Michael Nordmanmicha...@google.com wrote: Ok... I overstated things ;) What seems inevitable are vista-like prompts to allow something (or prods to delete something) seemingly unrelated to a user's interaction with a site... please, oh please, lets avoid making that part of the web platform. As far as I know, cookies work the same way as the proposed local storage policy: once a cookie is created, the browser won't delete it when space becomes a problem. The site controls the expiration date of the cookie, and it can fill up the entire drive with cookies if it wants to do so. This is all without user interaction. I don't think this has ever been a problem. a. cookies are compartively small... size constraints built in b. UA's actually do evict cookies if need be (hasn't ever been a problem) c. they have expriration dates, these new pieces of info don't -- Remco
Re: [whatwg] Web Storage: apparent contradiction in spec
My suggestion to have separate 'important' and 'cache' local storage areas would provide such a mechanism in a standard way. The first time an app tried to put stuff in the 'important' area, you'd be asked for approval. And 'important' stores wouldn't be deleted without your consent. I think you just described a way to 'bless' things. On Wed, Aug 26, 2009 at 5:06 PM, Jens Alfke s...@google.com wrote: On Aug 26, 2009, at 4:55 PM, Michael Nordman wrote: What seems inevitable are vista-like prompts to allow something (or prods to delete something) seemingly unrelated to a user's interaction with a site... please, oh please, lets avoid making that part of the web platform. Doesn't Gears already do this? If I do something like enabling local draft storage in WordPress, I get a prompt asking me if I want to allow myblog.com to store some local data on my disk, and I click OK because after all that's what I asked the site to do. Yes, but we hate those prompts, don't you? I'm assuming that UA will have out-of-band mechanisms to 'bless' certain sites which should not be subject to automated eviction. If this is out-of-spec and browser-dependent, there won't be a good way for an app to request that blessing; it'll be something the user has to know to do, otherwise their data can get lost. That seems dangerous. In most systems user data loss is just about the worst-case scenario of what could go wrong, and you try to prevent it at all costs. I'd love it if HTML5 took on a standard means of 'blessing' things. Thus far every time we come even close to the notion of 'installing' something... it doesn't go well in this forum. My suggestion to have separate 'important' and 'cache' local storage areas would provide such a mechanism in a standard way. The first time an app tried to put stuff in the 'important' area, you'd be asked for approval. And 'important' stores wouldn't be deleted without your consent. Interesting... I think you just described a way to 'bless' things by alluding to 'important' repositories and trying to put something in them. There are probably other ways to express this too that wouldn't involve altering the storage APIs... i'd prefer to pursue things in a more modular fashion... jmho. —Jens
Re: [whatwg] Web Storage: apparent contradiction in spec
On Wed, Aug 26, 2009 at 4:55 PM, Michael Nordman micha...@google.comwrote: What seems inevitable are vista-like prompts to allow something (or prods to delete something) seemingly unrelated to a user's interaction with a site... please, oh please, lets avoid making that part of the web platform. I hate prompts as much as you. Flash uses a model where a site can silently store small amounts of data with no prompts. Because devices have wildly different storage amounts, one could imagine a UA on a desktop machine allowing a site to store, say, 2 MB without prompting, while on a phone the site might only get 20 KB, or maybe none at all. This would mean users would be prompted sooner or more often on a phone, which seems like a reasonable outcome to me given that a phone may have so little storage that serious use of Local Storage may be difficult to impossible anyway. In this world, the hard quotas I suggested become soft quotas which result in some kind of user elevation. A UA could elect not to elevate and just deny the additional space if its authors felt that prompts were evil :) I'm assuming that UA will have out-of-band mechanisms to 'bless' certain sites which should not be subject to automated eviction. If push comes to shove, the system could suggest cleaning up one of these 'blessed' sites if inactivity for an extended period was noticed. But for the overwhelming number of sites in a users browsing history, its a different matter. If the storage APIs are just available for use, no questions asked making the storage just go away, no questions asked, is symmetrical. Blessing involves asking questions... making it go away does too. If we suggest that the user be prompted before anything be written persistently, there are a couple bad outcomes (note that these are problems with Gears today): * The user is asked to make a choice _before_ using the app's functionality, at which point he is ill-prepared to decide how much he likes the app or what it should be able to do * The app author is less-likely to bother to use Local Storage since prompts drive users away, and just uses Flash I think the overall UX from requiring blessing on all persistent data (as opposed to on large data sets) is poorer. PK
Re: [whatwg] Web Storage: apparent contradiction in spec
On Thu, Aug 27, 2009 at 2:26 AM, Peter Kastingpkast...@google.com wrote: On Wed, Aug 26, 2009 at 4:55 PM, Michael Nordman micha...@google.com wrote: What seems inevitable are vista-like prompts to allow something (or prods to delete something) seemingly unrelated to a user's interaction with a site... please, oh please, lets avoid making that part of the web platform. I hate prompts as much as you. Flash uses a model where a site can silently store small amounts of data with no prompts. Because devices have wildly different storage amounts, one could imagine a UA on a desktop machine allowing a site to store, say, 2 MB without prompting, while on a phone the site might only get 20 KB, or maybe none at all. This would mean users would be prompted sooner or more often on a phone, which seems like a reasonable outcome to me given that a phone may have so little storage that serious use of Local Storage may be difficult to impossible anyway. In this world, the hard quotas I suggested become soft quotas which result in some kind of user elevation. A UA could elect not to elevate and just deny the additional space if its authors felt that prompts were evil :) This makes me think about applications that need to store huge amounts of data in order to work. Imagine that Quake Live is ported to WebGL and uses Local Storage so it can work without any plugin. The app will want to store some 500MB of game data. If local storage were to work like a cache, then everything else would likely get purged each time you play the game. -- Remco
Re: [whatwg] Web Storage: apparent contradiction in spec
On Wed, Aug 26, 2009 at 5:14 PM, Brady Eidson beid...@apple.com wrote: I started writing a detailed rebuttal to Linus's reply, but by the time I was finished, many others had already delivered more targetted replies. So I'll cut the rebuttal format and make a few specific points. - Many apps act as a shoebox for managing specific types of data, and users are used to using these apps to manage that data directly. See iTunes, Windows Media Player, iPhoto, and desktop mail clients as examples. This trend is growing, not waning. Browsers are already a shoebox for history, bookmarks, and other types of data. Claiming that this data is hidden from users who are used to handling obscure file management scenarios and therefore we shouldn't fully respect it is trying to fit in with the past, not trying to make the future better. - No one is suggesting that UAs not have whatever freedom they want in deciding *what* or *how much* to store. We're only suggesting that once the UA has committed to storing it, it *not* be allowed to arbitrarily purge it. - One use of LocalStorage is as a cache for data that is transient and non-critical in nature, or that will live on a server. But another, just-as-valid use of LocalStorage is for persistent, predictable storage in the client UA that will never rely on anything in the cloud. - And on that note, if developers don't have faith that data in LocalStorage is actually persistent and safe, they won't use it. I've given talks on this point 4 times in the last year, and I am stating this as a fact, based on real-world feedback from actual, real-world web developers: If LocalStorage is defined in the standard to be a purgable cache, developers will continue to use what they're already comfortable with, which is Flash's LocalStorage. When a developer is willing to instantiate a plug-in just to reliably store simple nuggets of data - like user preferences and settings - because they don't trust the browser, then I think we've failed in moving the web forward. I truly hope we can sway the LocalStorage is a cache crowd. But if we can't, then I would have to suggest something crazy - that we add a third Storage object. (Note that Jens - from Google - has already loosely suggested this) So we'd have something like: -SessionStorage - That fills the per browsing context role and whose optionally transient nature is already well spec'ed -CachedStorage - That fills Google's interpretation of the LocalStorage role in that it's global, and will probably be around on the disk in the future, maybe -FileStorage - That fills Apple's interpretation of the LocalStorage role in that it's global, and is as sacred as a file on the disk (or a song in your media library, or a photo in your photo library, or a bookmark, or...) The names are just suggestions at this point. Just for the record, I (now) agree with everything you said here.
Re: [whatwg] Web Storage: apparent contradiction in spec
On Wed, Aug 26, 2009 at 5:14 PM, Brady Eidson beid...@apple.com wrote: I started writing a detailed rebuttal to Linus's reply, but by the time I was finished, many others had already delivered more targetted replies. So I'll cut the rebuttal format and make a few specific points. - Many apps act as a shoebox for managing specific types of data, and users are used to using these apps to manage that data directly. See iTunes, Windows Media Player, iPhoto, and desktop mail clients as examples. This trend is growing, not waning. Browsers are already a shoebox for history, bookmarks, and other types of data. Claiming that this data is hidden from users who are used to handling obscure file management scenarios and therefore we shouldn't fully respect it is trying to fit in with the past, not trying to make the future better. - No one is suggesting that UAs not have whatever freedom they want in deciding *what* or *how much* to store. We're only suggesting that once the UA has committed to storing it, it *not* be allowed to arbitrarily purge it. - One use of LocalStorage is as a cache for data that is transient and non-critical in nature, or that will live on a server. But another, just-as-valid use of LocalStorage is for persistent, predictable storage in the client UA that will never rely on anything in the cloud. - And on that note, if developers don't have faith that data in LocalStorage is actually persistent and safe, they won't use it. I've given talks on this point 4 times in the last year, and I am stating this as a fact, based on real-world feedback from actual, real-world web developers: If LocalStorage is defined in the standard to be a purgable cache, developers will continue to use what they're already comfortable with, which is Flash's LocalStorage. When a developer is willing to instantiate a plug-in just to reliably store simple nuggets of data - like user preferences and settings - because they don't trust the browser, then I think we've failed in moving the web forward. I truly hope we can sway the LocalStorage is a cache crowd. But if we can't, then I would have to suggest something crazy - that we add a third Storage object. (Note that Jens - from Google - has already loosely suggested this) So we'd have something like: -SessionStorage - That fills the per browsing context role and whose optionally transient nature is already well spec'ed -CachedStorage - That fills Google's interpretation of the LocalStorage role in that it's global, and will probably be around on the disk in the future, maybe -FileStorage - That fills Apple's interpretation of the LocalStorage role in that it's global, and is as sacred as a file on the disk (or a song in your media library, or a photo in your photo library, or a bookmark, or...) In addition to the key/value pair storage apis, i think we'd need to make this distinction for databases and appcaches too. This distinction may be better handled in a way not tied to a particular flavor on storage. Or a similar distinction could be expressible within the database and appcache interfaces. window.openPermanentDatabase() / openPurgeableDatabase() manifest file syntax games: PURGEABLE or PERMANENT keyword in there somewhere. The names are just suggestions at this point. ~Brady
Re: [whatwg] Web Storage: apparent contradiction in spec
On Wed, Aug 26, 2009 at 5:42 PM, Michael Nordman micha...@google.comwrote: In addition to the key/value pair storage apis, i think we'd need to make this distinction for databases and appcaches too. This distinction may be better handled in a way not tied to a particular flavor on storage. Or a similar distinction could be expressible within the database and appcache interfaces. window.openPermanentDatabase() / openPurgeableDatabase() manifest file syntax games: PURGEABLE or PERMANENT keyword in there somewhere. I think having authors choose between permanent and purgeable storage types adds complexity to the implementation and usage that is not desirable from either an authoring or a UX perspective. I continue to support Brady et al.'s vision of Local Storage. PK
Re: [whatwg] Web Storage: apparent contradiction in spec
On Wed, Aug 26, 2009 at 8:58 PM, Peter Kastingpkast...@google.com wrote: I think having authors choose between permanent and purgeable storage types adds complexity to the implementation and usage that is not desirable from either an authoring or a UX perspective. This complexity already exists for real apps, and developers seem fine with it. If you want persistent storage, you write to ~ or C:\Documents and Settings\ or such; if you want temporary storage, you write to /tmp or C:\temp\ or such. The distinction is useful to both users and developers, not burdensome. If authors can't say this can be thrown out if I haven't used it for a while, poorly-written apps will ignore the issue and store data forever. If they're storing small amounts of data, they probably won't run into any limits in testing, so they might not bother with handling out-of-space errors properly. So then eventually the garbage will accumulate until they run out of room and die, or bother the user pointlessly. A properly-written app, on the other hand, will be burdened with managing its own temporary storage. Authors will have to manually keep track of which of their resources are temporary and figure out when to remove them. Authors can't do this as well as the browser can. Authors don't know how much disk space is free, or how much space other apps are using, and they're in no position to decide how much disk space *should* be free. They might use too much disk space, or slow down their app by caching less for fear of using too much disk space. The browser should manage cache purging for the same reason the OS manages the page cache instead of applications.
Re: [whatwg] Web Storage: apparent contradiction in spec
I simply want clicking on links to be safe. In a previous thread I wrote safe and stateless but I'm coming to the opinion that stateless is a corollary of safe. Clicking on links shouldn't, either by filling my disk or hitting my global quota, someday lead to a dialog which reads, Please choose what to delete so that web sites will continue to work. The candidate delete list will be thousands long and hidden in that haystack will be a few precious needles. I also want to avoid any [Yes] [No] dialogs. Can I do something scary [Yes] [No]? Can I do something innocuous [Yes] [No]? Users shouldn't be forced to make those kinds of safety judgements. I'm guilty of instigating at least one of those dialogs. As shamed politicians do I'll retreat to the passive voice: Mistakes were made. I'm not opposed to web apps manipulating files on the user's computer, but the user should be in explicit control. I'd support input type=open and input type=save that worked similarly to input type=file. User agents are already registering for file types so that double clicking a file with a certain extension can be automatically sent to an URL, perhaps residing in an AppCache. In addition, I'd like to see the pop-up dialogs for the location API removed. I find the Can I know where you are? dialogs on the iPhone very annoying. Mistakes were made. Perhaps we can find a way to make input type=location work well instead. Linus On Wed, Aug 26, 2009 at 5:14 PM, Brady Eidson beid...@apple.com wrote: I started writing a detailed rebuttal to Linus's reply, but by the time I was finished, many others had already delivered more targetted replies. So I'll cut the rebuttal format and make a few specific points. - Many apps act as a shoebox for managing specific types of data, and users are used to using these apps to manage that data directly. See iTunes, Windows Media Player, iPhoto, and desktop mail clients as examples. This trend is growing, not waning. Browsers are already a shoebox for history, bookmarks, and other types of data. Claiming that this data is hidden from users who are used to handling obscure file management scenarios and therefore we shouldn't fully respect it is trying to fit in with the past, not trying to make the future better. - No one is suggesting that UAs not have whatever freedom they want in deciding *what* or *how much* to store. We're only suggesting that once the UA has committed to storing it, it *not* be allowed to arbitrarily purge it. - One use of LocalStorage is as a cache for data that is transient and non-critical in nature, or that will live on a server. But another, just-as-valid use of LocalStorage is for persistent, predictable storage in the client UA that will never rely on anything in the cloud. - And on that note, if developers don't have faith that data in LocalStorage is actually persistent and safe, they won't use it. I've given talks on this point 4 times in the last year, and I am stating this as a fact, based on real-world feedback from actual, real-world web developers: If LocalStorage is defined in the standard to be a purgable cache, developers will continue to use what they're already comfortable with, which is Flash's LocalStorage. When a developer is willing to instantiate a plug-in just to reliably store simple nuggets of data - like user preferences and settings - because they don't trust the browser, then I think we've failed in moving the web forward. I truly hope we can sway the LocalStorage is a cache crowd. But if we can't, then I would have to suggest something crazy - that we add a third Storage object. (Note that Jens - from Google - has already loosely suggested this) So we'd have something like: -SessionStorage - That fills the per browsing context role and whose optionally transient nature is already well spec'ed -CachedStorage - That fills Google's interpretation of the LocalStorage role in that it's global, and will probably be around on the disk in the future, maybe -FileStorage - That fills Apple's interpretation of the LocalStorage role in that it's global, and is as sacred as a file on the disk (or a song in your media library, or a photo in your photo library, or a bookmark, or...) The names are just suggestions at this point. ~Brady
Re: [whatwg] Web Storage: apparent contradiction in spec
This is one of those times when I *really* wish that the application developer community was more active on this list. I absolutely understand Linus' point of view, but I also feel like we are really hamstringing applications when we make choices like this and I wish that those developers were more vocally represented in these types of discussions. Going down this path would basically kill the ability to have offline web applications, because there would be no guarantees that the data would persist until the user comes back online. But since that point's already been made several times, I guess it's not a compelling argument. -atw On Wed, Aug 26, 2009 at 8:23 PM, Linus Upson li...@google.com wrote: I simply want clicking on links to be safe. In a previous thread I wrote safe and stateless but I'm coming to the opinion that stateless is a corollary of safe. Clicking on links shouldn't, either by filling my disk or hitting my global quota, someday lead to a dialog which reads, Please choose what to delete so that web sites will continue to work. The candidate delete list will be thousands long and hidden in that haystack will be a few precious needles. I also want to avoid any [Yes] [No] dialogs. Can I do something scary [Yes] [No]? Can I do something innocuous [Yes] [No]? Users shouldn't be forced to make those kinds of safety judgements. I'm guilty of instigating at least one of those dialogs. As shamed politicians do I'll retreat to the passive voice: Mistakes were made. I'm not opposed to web apps manipulating files on the user's computer, but the user should be in explicit control. I'd support input type=open and input type=save that worked similarly to input type=file. User agents are already registering for file types so that double clicking a file with a certain extension can be automatically sent to an URL, perhaps residing in an AppCache. In addition, I'd like to see the pop-up dialogs for the location API removed. I find the Can I know where you are? dialogs on the iPhone very annoying. Mistakes were made. Perhaps we can find a way to make input type=location work well instead. Linus On Wed, Aug 26, 2009 at 5:14 PM, Brady Eidson beid...@apple.com wrote: I started writing a detailed rebuttal to Linus's reply, but by the time I was finished, many others had already delivered more targetted replies. So I'll cut the rebuttal format and make a few specific points. - Many apps act as a shoebox for managing specific types of data, and users are used to using these apps to manage that data directly. See iTunes, Windows Media Player, iPhoto, and desktop mail clients as examples. This trend is growing, not waning. Browsers are already a shoebox for history, bookmarks, and other types of data. Claiming that this data is hidden from users who are used to handling obscure file management scenarios and therefore we shouldn't fully respect it is trying to fit in with the past, not trying to make the future better. - No one is suggesting that UAs not have whatever freedom they want in deciding *what* or *how much* to store. We're only suggesting that once the UA has committed to storing it, it *not* be allowed to arbitrarily purge it. - One use of LocalStorage is as a cache for data that is transient and non-critical in nature, or that will live on a server. But another, just-as-valid use of LocalStorage is for persistent, predictable storage in the client UA that will never rely on anything in the cloud. - And on that note, if developers don't have faith that data in LocalStorage is actually persistent and safe, they won't use it. I've given talks on this point 4 times in the last year, and I am stating this as a fact, based on real-world feedback from actual, real-world web developers: If LocalStorage is defined in the standard to be a purgable cache, developers will continue to use what they're already comfortable with, which is Flash's LocalStorage. When a developer is willing to instantiate a plug-in just to reliably store simple nuggets of data - like user preferences and settings - because they don't trust the browser, then I think we've failed in moving the web forward. I truly hope we can sway the LocalStorage is a cache crowd. But if we can't, then I would have to suggest something crazy - that we add a third Storage object. (Note that Jens - from Google - has already loosely suggested this) So we'd have something like: -SessionStorage - That fills the per browsing context role and whose optionally transient nature is already well spec'ed -CachedStorage - That fills Google's interpretation of the LocalStorage role in that it's global, and will probably be around on the disk in the future, maybe -FileStorage - That fills Apple's interpretation of the LocalStorage role in that it's global, and is as sacred as a file on the disk (or a song in your media library, or a photo in your photo library, or a bookmark,
Re: [whatwg] Web Storage: apparent contradiction in spec
I understand you want the web to be safe. I do too, and I think we all do. It never has been, it currently isn't, but it's better. The situation has been improving for many years. I'm not convinced that the idea of guaranteed persistent storage inherently takes us backwards. I think the current language in the spec (and whatever language we eventually settle on) allows user agents the flexibility to implement their own extra restrictions to accommodate its idea of safe while still guaranteeing the persistence of data it has claimed to store locally. I am convinced that if we change the language to solidify that LocalStorage is just a cache, we're discounting its use in offline web applications and the other examples of applications that desire local, persistent state and do not want a server/cloud component. We'd be driving these applications back to plug-ins, and I personally find that unacceptable. If that's what we're doing, then we haven't actually gained much with LocalStorage and might as well drop it. ~Brady On Aug 26, 2009, at 8:23 PM, Linus Upson wrote: I simply want clicking on links to be safe. In a previous thread I wrote safe and stateless but I'm coming to the opinion that stateless is a corollary of safe. Clicking on links shouldn't, either by filling my disk or hitting my global quota, someday lead to a dialog which reads, Please choose what to delete so that web sites will continue to work. The candidate delete list will be thousands long and hidden in that haystack will be a few precious needles. I also want to avoid any [Yes] [No] dialogs. Can I do something scary [Yes] [No]? Can I do something innocuous [Yes] [No]? Users shouldn't be forced to make those kinds of safety judgements. I'm guilty of instigating at least one of those dialogs. As shamed politicians do I'll retreat to the passive voice: Mistakes were made. I'm not opposed to web apps manipulating files on the user's computer, but the user should be in explicit control. I'd support input type=open and input type=save that worked similarly to input type=file. User agents are already registering for file types so that double clicking a file with a certain extension can be automatically sent to an URL, perhaps residing in an AppCache. In addition, I'd like to see the pop-up dialogs for the location API removed. I find the Can I know where you are? dialogs on the iPhone very annoying. Mistakes were made. Perhaps we can find a way to make input type=location work well instead. Linus On Wed, Aug 26, 2009 at 5:14 PM, Brady Eidson beid...@apple.com wrote: I started writing a detailed rebuttal to Linus's reply, but by the time I was finished, many others had already delivered more targetted replies. So I'll cut the rebuttal format and make a few specific points. - Many apps act as a shoebox for managing specific types of data, and users are used to using these apps to manage that data directly. See iTunes, Windows Media Player, iPhoto, and desktop mail clients as examples. This trend is growing, not waning. Browsers are already a shoebox for history, bookmarks, and other types of data. Claiming that this data is hidden from users who are used to handling obscure file management scenarios and therefore we shouldn't fully respect it is trying to fit in with the past, not trying to make the future better. - No one is suggesting that UAs not have whatever freedom they want in deciding *what* or *how much* to store. We're only suggesting that once the UA has committed to storing it, it *not* be allowed to arbitrarily purge it. - One use of LocalStorage is as a cache for data that is transient and non-critical in nature, or that will live on a server. But another, just-as-valid use of LocalStorage is for persistent, predictable storage in the client UA that will never rely on anything in the cloud. - And on that note, if developers don't have faith that data in LocalStorage is actually persistent and safe, they won't use it. I've given talks on this point 4 times in the last year, and I am stating this as a fact, based on real-world feedback from actual, real-world web developers: If LocalStorage is defined in the standard to be a purgable cache, developers will continue to use what they're already comfortable with, which is Flash's LocalStorage. When a developer is willing to instantiate a plug-in just to reliably store simple nuggets of data - like user preferences and settings - because they don't trust the browser, then I think we've failed in moving the web forward. I truly hope we can sway the LocalStorage is a cache crowd. But if we can't, then I would have to suggest something crazy - that we add a third Storage object. (Note that Jens - from Google - has already loosely suggested this) So we'd have something like: -SessionStorage - That fills the per browsing
[whatwg] Web Storage: apparent contradiction in spec
I've just noticed an apparent self-contradiction in the Web Storage spec (24 August draft). Section 4.3 states: Data stored in local storage areas should be considered potentially user-critical. It is expected that Web applications will use the local storage areas for storing user-written documents. Section 6.1 states: User agents should present the persistent storage feature to the user in a way that does not distinguish them from HTTP session cookies. These statements are contradictory, because cookies don't store user- critical data such as documents. The user model of cookies is that they're conveniences (at best) for keeping you logged into a site or remembering preferences like font-size, so deleting them is no more than an inconvenience. If local storage is presented to the user as being cookies, then a user may delete it without understanding the consequences. Potential result: I was having trouble logging into FooDocs.com, so my friend suggested I delete the cookies for that site. After that I could log in, but now the document I was working on this morning has lost all the changes I made! How do I get them back? I suggest that the sub-section Treating persistent storage as cookies of section 6.1 be removed. —Jens
Re: [whatwg] Web Storage: apparent contradiction in spec
On Aug 25, 2009, at 1:38 PM, Linus Upson wrote: It is important that all local state be treated as a cache. User agents need to be free to garbage collect any local state. If they can't then attackers (or the merely lazy) will be able to fill up the user's disk. We can't expect web sites or users to do the chore of taking out the garbage. Better user agents will have better garbage collection algorithms. It would be better to remove section 4.3. I disagree. One key advantage of LocalStorage and Databases over cookies is that they *do* have a predictable, persistent lifetime, and the browser is *not* allowed to prune them at will. User agents are perfectly allowed to not allow new items to go into LocalStorage or Database Storage once some quota is met, or if the user has disabled it for that domain, or disabled it altogether, or if the disk is filling up, or any other number of circumstances. But once the data is stored, it should be considered user data - as sacred as a user's file on the file system. Linus On Tue, Aug 25, 2009 at 1:18 PM, Jens Alfke s...@google.com wrote: I've just noticed an apparent self-contradiction in the Web Storage spec (24 August draft). Section 4.3 states: Data stored in local storage areas should be considered potentially user-critical. It is expected that Web applications will use the local storage areas for storing user-written documents. Section 6.1 states: User agents should present the persistent storage feature to the user in a way that does not distinguish them from HTTP session cookies. These statements are contradictory, because cookies don't store user- critical data such as documents. The user model of cookies is that they're conveniences (at best) for keeping you logged into a site or remembering preferences like font-size, so deleting them is no more than an inconvenience. If local storage is presented to the user as being cookies, then a user may delete it without understanding the consequences. Potential result: I was having trouble logging into FooDocs.com, so my friend suggested I delete the cookies for that site. After that I could log in, but now the document I was working on this morning has lost all the changes I made! How do I get them back? I suggest that the sub-section Treating persistent storage as cookies of section 6.1 be removed. I agree that the wording of the section needs great improvement and it is factually wrong in its current form. That said, I don't think it should be removed. I *think* Ian's intention was that in the same vein that user agents are obligated to present all of the cookies stored on the disk at any given time and allow manual management of them, they should be obligated to do the same for LocalStorage (and databases, too). If I'm wrong, hopefully he can correct me. ~Brady
Re: [whatwg] Web Storage: apparent contradiction in spec
On Tue, Aug 25, 2009 at 2:09 PM, Brady Eidson beid...@apple.com wrote: On Aug 25, 2009, at 1:38 PM, Linus Upson wrote: It is important that all local state be treated as a cache. User agents need to be free to garbage collect any local state. If they can't then attackers (or the merely lazy) will be able to fill up the user's disk. We can't expect web sites or users to do the chore of taking out the garbage. Better user agents will have better garbage collection algorithms. It would be better to remove section 4.3. I disagree. One key advantage of LocalStorage and Databases over cookies is that they *do* have a predictable, persistent lifetime, and the browser is *not* allowed to prune them at will. User agents are perfectly allowed to not allow new items to go into LocalStorage or Database Storage once some quota is met, or if the user has disabled it for that domain, or disabled it altogether, or if the disk is filling up, or any other number of circumstances. But once the data is stored, it should be considered user data - as sacred as a user's file on the file system. What happens when your computer blows up? When you switch browsers? What about when you re-install your OS? etc What about mobile devices where 5mb is actually a lot of space? What happens when a malicious site fills up all of your localStorage space? You're saying the UAs should not be free to have heuristics about what to delete? What do they do then? Note this exact point has been discussed on this list before, and IIRC the outcome was that localStorage should be treated like cookies: we'll try to keep them around, but the app should be resilient to them going away. J
Re: [whatwg] Web Storage: apparent contradiction in spec
It is important that all local state be treated as a cache. User agents need to be free to garbage collect any local state. If they can't then attackers (or the merely lazy) will be able to fill up the user's disk. We can't expect web sites or users to do the chore of taking out the garbage. Better user agents will have better garbage collection algorithms. It would be better to remove section 4.3. Linus On Tue, Aug 25, 2009 at 1:18 PM, Jens Alfke s...@google.com wrote: I've just noticed an apparent self-contradiction in the Web Storage spechttp://dev.w3.org/html5/webstorage (24 August draft). Section 4.3 states: Data stored in local storage areas should be considered potentially user-critical. It is expected that Web applications will use the local storage areas for storing user-written documents. Section 6.1 states: User agents should present the persistent storage feature to the user in a way that does not distinguish them from HTTP session cookies. These statements are contradictory, because cookies don't store user-critical data such as documents. The user model of cookies is that they're conveniences (at best) for keeping you logged into a site or remembering preferences like font-size, so deleting them is no more than an inconvenience. If local storage is presented to the user as being cookies, then a user may delete it without understanding the consequences. Potential result: I was having trouble logging into FooDocs.com, so my friend suggested I delete the cookies for that site. After that I could log in, but now the document I was working on this morning has lost all the changes I made! How do I get them back? I suggest that the sub-section Treating persistent storage as cookies of section 6.1 be removed. —Jens
Re: [whatwg] Web Storage: apparent contradiction in spec
On Aug 25, 2009, at 2:16 PM, Jeremy Orlow wrote: On Tue, Aug 25, 2009 at 2:09 PM, Brady Eidson beid...@apple.com wrote: On Aug 25, 2009, at 1:38 PM, Linus Upson wrote: It is important that all local state be treated as a cache. User agents need to be free to garbage collect any local state. If they can't then attackers (or the merely lazy) will be able to fill up the user's disk. We can't expect web sites or users to do the chore of taking out the garbage. Better user agents will have better garbage collection algorithms. It would be better to remove section 4.3. I disagree. One key advantage of LocalStorage and Databases over cookies is that they *do* have a predictable, persistent lifetime, and the browser is *not* allowed to prune them at will. User agents are perfectly allowed to not allow new items to go into LocalStorage or Database Storage once some quota is met, or if the user has disabled it for that domain, or disabled it altogether, or if the disk is filling up, or any other number of circumstances. But once the data is stored, it should be considered user data - as sacred as a user's file on the file system. What happens when your computer blows up? You lose the data the same way you lose your local file data. When you switch browsers? Unfortunately the same thing that happens with your bookmarks, preferences, history, etc - unless the new browser knows how to import the old data. No one would ever claim a browser should be able to arbitrarily prune a user's bookmarks just because you might lose them when switching browsers. If someone would claim that, I would raise this same objection. What about when you re-install your OS? Same thing as with local files - if you didn't backup your hard disk, you lose them. If you do backup your hard disk and restore files after you re-install your OS, you get your localstorage, databases, and hell - even your Flash cookies back, just like your files. What about mobile devices where 5mb is actually a lot of space? These mobile devices are perfectly allowed to restrict the amount of data they agree to store with respect to their limited capacity. What happens when a malicious site fills up all of your localStorage space? This is why per-security-origin quotas exist. For the counter argument of what about a site that switches subdomains to subvert the per-origin quota?, fortunately HTML5 doesn't disallow browsers from limiting per top-level domain or via some other extra limitation. You're saying the UAs should not be free to have heuristics about what to delete? Yes. What do they do then? They should be free to have whatever heuristics they'd like when choosing what to store. But once it's stored, it should be persistent. When a user's hard drive on a desktop machine fills up, should the operating system be able to decide Oh crap, I'm running out of space, and I have no other caches or temporary data to delete. So I'll just go ahead and start deleting the user's files without asking? LocalStorage is quite clearly modeled after Flash's LocalStorage - what does Flash do? It has all sorts of controls in place to limit what data is stored. But once the data *is* stored, does it ever arbitrarily decide to delete it? Note this exact point has been discussed on this list before, and IIRC the outcome was that localStorage should be treated like cookies: we'll try to keep them around, but the app should be resilient to them going away. This exact point has been discussed on this list more than once, and I've only ever seen it die out with no consensus. If the discussion took place and it *was* decided that User Agents should arbitrarily be able to decide to delete LocalStorage or database data without the user pre-clearing that action, then I'm afraid I missed it and I am raising my objection now. ~Brady J
Re: [whatwg] Web Storage: apparent contradiction in spec
On Tue, Aug 25, 2009 at 2:16 PM, Jeremy Orlow jor...@chromium.org wrote: On Tue, Aug 25, 2009 at 2:09 PM, Brady Eidson beid...@apple.com wrote: On Aug 25, 2009, at 1:38 PM, Linus Upson wrote: It is important that all local state be treated as a cache. User agents need to be free to garbage collect any local state. If they can't then attackers (or the merely lazy) will be able to fill up the user's disk. We can't expect web sites or users to do the chore of taking out the garbage. Better user agents will have better garbage collection algorithms. It would be better to remove section 4.3. I disagree. One key advantage of LocalStorage and Databases over cookies is that they *do* have a predictable, persistent lifetime, and the browser is *not* allowed to prune them at will. User agents are perfectly allowed to not allow new items to go into LocalStorage or Database Storage once some quota is met, or if the user has disabled it for that domain, or disabled it altogether, or if the disk is filling up, or any other number of circumstances. But once the data is stored, it should be considered user data - as sacred as a user's file on the file system. What happens when your computer blows up? When you switch browsers? What about when you re-install your OS? etc What about mobile devices where 5mb is actually a lot of space? What happens when a malicious site fills up all of your localStorage space? You're saying the UAs should not be free to have heuristics about what to delete? What do they do then? I just re-read your message and noticed that you suggested local storage should stop working when it's filled up. First of all, this seems completely unacceptable to me, especially since we don't have a good answer yet (besides quotas which are probably enough for your average desktop, but probably not your average mobile or netbook) to the problem of malicious sites filling up local storage with multiple sub domains. Also, consider this scenario: you fill up your hard drive only 50% of the way with localStorage data. (Possibly by a malicious site, and thus the user is unaware.) Now you try to install a bunch of apps (maybe even over time) that consume the rest of the hard drive. So now the user is either SOL unless they can realize this space has gone to their browser and can figure out how to delete it? Note this exact point has been discussed on this list before, and IIRC the outcome was that localStorage should be treated like cookies: we'll try to keep them around, but the app should be resilient to them going away. One previous reference: http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-April/019276.html I thought there was more discussion, but I couldn't find anything other than this thread after a quick search. J
Re: [whatwg] Web Storage: apparent contradiction in spec
On Tue, Aug 25, 2009 at 2:40 PM, Brady Eidson beid...@apple.com wrote: On Aug 25, 2009, at 2:16 PM, Jeremy Orlow wrote: On Tue, Aug 25, 2009 at 2:09 PM, Brady Eidson beid...@apple.com wrote: On Aug 25, 2009, at 1:38 PM, Linus Upson wrote: It is important that all local state be treated as a cache. User agents need to be free to garbage collect any local state. If they can't then attackers (or the merely lazy) will be able to fill up the user's disk. We can't expect web sites or users to do the chore of taking out the garbage. Better user agents will have better garbage collection algorithms. It would be better to remove section 4.3. I disagree. One key advantage of LocalStorage and Databases over cookies is that they *do* have a predictable, persistent lifetime, and the browser is *not* allowed to prune them at will. User agents are perfectly allowed to not allow new items to go into LocalStorage or Database Storage once some quota is met, or if the user has disabled it for that domain, or disabled it altogether, or if the disk is filling up, or any other number of circumstances. But once the data is stored, it should be considered user data - as sacred as a user's file on the file system. What happens when your computer blows up? You lose the data the same way you lose your local file data. When you switch browsers? Unfortunately the same thing that happens with your bookmarks, preferences, history, etc - unless the new browser knows how to import the old data. No one would ever claim a browser should be able to arbitrarily prune a user's bookmarks just because you might lose them when switching browsers. If someone would claim that, I would raise this same objection. What about when you re-install your OS? Same thing as with local files - if you didn't backup your hard disk, you lose them. If you do backup your hard disk and restore files after you re-install your OS, you get your localstorage, databases, and hell - even your Flash cookies back, just like your files. What about mobile devices where 5mb is actually a lot of space? These mobile devices are perfectly allowed to restrict the amount of data they agree to store with respect to their limited capacity. What happens when a malicious site fills up all of your localStorage space? This is why per-security-origin quotas exist. For the counter argument of what about a site that switches subdomains to subvert the per-origin quota?, fortunately HTML5 doesn't disallow browsers from limiting per top-level domain or via some other extra limitation. You're saying the UAs should not be free to have heuristics about what to delete? Yes. What do they do then? They should be free to have whatever heuristics they'd like when choosing what to store. But once it's stored, it should be persistent. When a user's hard drive on a desktop machine fills up, should the operating system be able to decide Oh crap, I'm running out of space, and I have no other caches or temporary data to delete. So I'll just go ahead and start deleting the user's files without asking? LocalStorage is quite clearly modeled after Flash's LocalStorage - what does Flash do? It has all sorts of controls in place to limit what data is stored. But once the data *is* stored, does it ever arbitrarily decide to delete it? Note this exact point has been discussed on this list before, and IIRC the outcome was that localStorage should be treated like cookies: we'll try to keep them around, but the app should be resilient to them going away. This exact point has been discussed on this list more than once, and I've only ever seen it die out with no consensus. If the discussion took place and it *was* decided that User Agents should arbitrarily be able to decide to delete LocalStorage or database data without the user pre-clearing that action, then I'm afraid I missed it and I am raising my objection now. Ok, well I guess we should go ahead and have this discussion now. :-) Does anyone outside of Apple and Google have an opinion on the matter (since I think it's pretty clear where we both stand).
Re: [whatwg] Web Storage: apparent contradiction in spec
Interesting comments. Linus and Jeremy appear to be coming at this from a pure cloud perspective, where any important or persistent data is kept on a remote server and the browser, so local storage can be treated as merely a cache. That's definitely a valid position, but from my perspective, much of the impetus for having local storage is to be able to support other application models, where important data is stored locally. If browsers are free to dispose HTML5 local storage without the user's informed consent, such applications become dangerously unreliable. For example, Linus wrote: User agents need to be free to garbage collect any local state. If they can't then attackers (or the merely lazy) will be able to fill up the user's disk. We can't expect web sites or users to do the chore of taking out the garbage. Replace user agent - operating system and local state - user files, and you have an argument that, when the hard disk in my MacBook gets too full, the OS should be free to start randomly deleting my local files to make room. This would be a really bad idea. Similar analogies — • If the SD card in my Wii fills up, should the system automatically start deleting saved games? • If my iPhone's Flash disk gets full, should it start deleting photos? What if I haven't synced those photos to my iTunes yet? In each of those cases, what the device actually does is warns you about the lack of free space, and lets you choose what to get rid of. Local storage is different from cloud storage. The HTML5 storage API can be used for both, so it shouldn't be limited to what's convenient for just one of them. —Jens
Re: [whatwg] Web Storage: apparent contradiction in spec
On Aug 25, 2009, at 3:09 PM, Jens Alfke wrote: Interesting comments. Linus and Jeremy appear to be coming at this from a pure cloud perspective, where any important or persistent data is kept on a remote server and the browser, so local storage can be treated as merely a cache. That's definitely a valid position, but from my perspective, much of the impetus for having local storage is to be able to support other application models, where important data is stored locally. If browsers are free to dispose HTML5 local storage without the user's informed consent, such applications become dangerously unreliable. For example, Linus wrote: User agents need to be free to garbage collect any local state. If they can't then attackers (or the merely lazy) will be able to fill up the user's disk. We can't expect web sites or users to do the chore of taking out the garbage. Replace user agent - operating system and local state - user files, and you have an argument that, when the hard disk in my MacBook gets too full, the OS should be free to start randomly deleting my local files to make room. This would be a really bad idea. Similar analogies — • If the SD card in my Wii fills up, should the system automatically start deleting saved games? • If my iPhone's Flash disk gets full, should it start deleting photos? What if I haven't synced those photos to my iTunes yet? In each of those cases, what the device actually does is warns you about the lack of free space, and lets you choose what to get rid of. Local storage is different from cloud storage. The HTML5 storage API can be used for both, so it shouldn't be limited to what's convenient for just one of them. Thank you Jens. This is a well thought out synopsis of the point I've been wanting to make whenever this comes up, but have failed to do. ~Brady —Jens
Re: [whatwg] Web Storage: apparent contradiction in spec
The statement in section 4.3 doesn't appear to specify any behavior... its just an informational statement. The statement in section 6.1 suggests to prohibit the development of a UI that mentions local storage as a distinct repository seperate from cookies. This doesn't belong in the spec imho. I think both of these statements should be dropped from the spec. Ultimately I think UAs will have to prop up out-of-band permissioning schemes to make stronger guarantees about how long lived 'local data' that accumulates really is. On Tue, Aug 25, 2009 at 3:19 PM, Aaron Boodman a...@google.com wrote: On Tue, Aug 25, 2009 at 2:44 PM, Jeremy Orlowjor...@chromium.org wrote: Ok, well I guess we should go ahead and have this discussion now. :-) Does anyone outside of Apple and Google have an opinion on the matter (since I think it's pretty clear where we both stand). FWIW, I tend to agree more with the Apple argument :). I agree that the multiple malicious subdomains thing is unfortunate. Maybe the quotas should be per eTLD instead of -- or in addition to -- per-origin? Malicious developers could then use multiple eTLDs, but at that point there is a real cost. Extensions are an example of an application that is less cloud-based. It would be unfortunate and weird for extension developers to have to worry about their storage getting tossed because the UA is running out of disk space. It seems more like if that happens the UA should direct the user to UI to free up some storage. If quotas were enforced at the eTLD level, wouldn't this be really rare? - a
Re: [whatwg] Web Storage: apparent contradiction in spec
On Tue, Aug 25, 2009 at 3:19 PM, Aaron Boodman a...@google.com wrote: On Tue, Aug 25, 2009 at 2:44 PM, Jeremy Orlowjor...@chromium.org wrote: Ok, well I guess we should go ahead and have this discussion now. :-) Does anyone outside of Apple and Google have an opinion on the matter (since I think it's pretty clear where we both stand). FWIW, I tend to agree more with the Apple argument :). I agree that the multiple malicious subdomains thing is unfortunate. Maybe the quotas should be per eTLD instead of -- or in addition to -- per-origin? Malicious developers could then use multiple eTLDs, but at that point there is a real cost. This could be helpful. I suppose UAs could do this today, even. Extensions are an example of an application that is less cloud-based. It would be unfortunate and weird for extension developers to have to worry about their storage getting tossed because the UA is running out of disk space. Extensions are pretty far out of scope of the spec (at least for now), right? (Within Chrome, we can of course special case this.) It seems more like if that happens the UA should direct the user to UI to free up some storage. If quotas were enforced at the eTLD level, wouldn't this be really rare? It would be on the desktop, but it probably won't be rare on mobile phones (and maybe even netbooks). On Tue, Aug 25, 2009 at 3:09 PM, Jens Alfke s...@google.com wrote: Interesting comments. Linus and Jeremy appear to be coming at this from a pure cloud perspective, where any important or persistent data is kept on a remote server and the browser, so local storage can be treated as merely a cache. That's definitely a valid position, but from my perspective, much of the impetus for having local storage is to be able to support *other* application models, where important data is stored locally. If browsers are free to dispose HTML5 local storage without the user's informed consent, such applications become dangerously unreliable. For example, Linus wrote: User agents need to be free to garbage collect any local state. If they can't then attackers (or the merely lazy) will be able to fill up the user's disk. We can't expect web sites or users to do the chore of taking out the garbage. Replace user agent - operating system and local state - user files, and you have an argument that, when the hard disk in my MacBook gets too full, the OS should be free to start randomly deleting my local files to make room. This would be a really bad idea. Well, it's certainly different from what we're used to. I'm not convinced it's wrong though. The web has gotten by pretty well with such a model so far. Similar analogies — • If the SD card in my Wii fills up, should the system automatically start deleting saved games? • If my iPhone's Flash disk gets full, should it start deleting photos? What if I haven't synced those photos to my iTunes yet? In each of those cases, what the device actually does is warns you about the lack of free space, and lets you choose what to get rid of. It's worth noting that today, OSs do a pretty poor job of helping you with this task. (I don't see any reason why the spec will prohibit UAs from creating a good UI for this, though.) Local storage is different from cloud storage. The HTML5 storage API can be used for both, so it shouldn't be limited to what's convenient for just one of them. I still don't understand what use local storage has outside of 'cloud storage'. Even in the extensions use case (which I think is out of scope for this spec), there's no reason you can't sync user preferences and such to the cloud. The only tricky thing I see here is enabling offline use of websites. And I don't think keep local storage forever is a very good solution, though I agree it is _a_ solution for this use case. By the way, in case it's not clear, my position is not that UAs should take deleting user information lightly, my position is 1) this behavior should be left up to the UA and 2) when possible, developers should keep information in the cloud not local storage. J
Re: [whatwg] Web Storage: apparent contradiction in spec
On Tue, Aug 25, 2009 at 3:51 PM, Jeremy Orlowjor...@chromium.org wrote: I still don't understand what use local storage has outside of 'cloud storage'. Even in the extensions use case (which I think is out of scope for this spec), there's no reason you can't sync user preferences and such to the cloud. The use case, though, is local storage, not cloud storage. Requiring cloud storage here kind of defeats the purpose and makes the API a lot harder to use. - a
