Re: [whatwg] localStorage feedback
Summary: I haven't removed the storage mutex, but I did adjust when it is released slightly to handle some cases that had been missed before. On Thu, 29 Oct 2009, Darin Fisher wrote: On Mon, Oct 12, 2009 at 7:07 PM, Ian Hickson i...@hixie.ch wrote: the problem here is that localStorage is a pile of global variables. we are trying to give people global variables without giving them tools to synchronize access to them. the claim i've heard is that developers are not savy enough to use those tools properly. i agree that developers tend to use tools without fully understanding them. ok, but then why are we giving them global variables? The global variables have implicit locks such that you can build the tools for explicit locking on top of them: // run this first, in one script block var id = localStorage['last-id'] + 1; localStorage['last-id'] = id; localStorage['email-ready-' + id] = 0; // begin // these can run each in separate script blocks as desired localStorage['email-subject-' + id] = subject; localStorage['email-from-' + id] = from; localStorage['email-to-' + id] = to; localStorage['email-body-' + id] = body; // run this last localStorage['email-ready-' + id] = 1; // commit Dividing up work like this into separate SCRIPT elements to scope the locking seems really awkward to me. Oh I agree, I'm just saying that your original point -- that we are trying to give people global variables without giving them tools to synchronize access to them -- is mistaken insofar as we _are_ trying to give them tools to synchronise access, that's what this whole thread is basically about. It just happens that those tools have unfortunate side-effects (they can require synchronous blocking of other processes running code for the same domain). The current API exposes race conditions to the web. The implicit dropping of the storage lock is that. In Chrome, we'll have to drop an existing lock whenever a new lock is acquired. That can happen due to a variety of really odd cases (usually related to nested loops or nested JS execution), which will be difficult for developers to predict, especially if they are relying on third-party JS libraries. This issue seems to be discounted for reasons I do not understand. You can only lose the lock in very specific conditions. Those conditions are rarely going to interact with code that actually does storage work in a way that relies on the lock: - changing document.domain - history.back(), .forward(), .go(n) - invoking a plugin - alert(), confirm(), prompt(), print() - showModalDialog() - yieldForStorageUpdates() I discussed this in more detail here: http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-September/023059.html You are right that the conditions are specific, but I don't know that that is the exhaustive list. If it's not, then we should fix it. Rather than defining unlock points, I would implement implicit unlocking by having any nested attempt to acquire a lock cause the existing lock to be dropped. Nesting can happen in the cases you mention, but depending on the UA, it may happen for other reasons too. I don't think this is equivalent at all. Most of the release points are points where the thread is about to hang (sync XHR, alert(), etc) and we want to unlock other processes. The plugin case is about third-party code causing a deadlock through undetectable thread synchronisation (e.g. where a plugin causes code in another process to try to grab the lock and waits for that to complete before handing control back to the UA). The nesting defence only helps in the case of preventing deadlocks with two processes trying to grab different domains' locks in reverse order from each other, as far as I can tell, which is just a subset of the cases where it is released per the spec. This combined with the fact that most people use JS libraries means that the coder is not going to have an easy time knowing when these specific conditions are met. I don't think defining a set of allowed unlock points is sufficient to make this API not be a minefield for users. I don't think anyone is arguing that this is ideal. I do think it's orders of magnitude better than having the API be always racy even without any libraries being involved. For example, a JS library might evolve to use flash for something small (like storage or sound) that it previously didn't use when I first developed my code. Voila, now my storage lock is released out from under me. At least this is documentable. On Sat, 31 Oct 2009, Darin Fisher wrote: Unlock if yieldForStorageUpdates is called. Unlock when returning from script execution. Unlock if another attempt to lock occurs (any form of nesting). In the third case, I'd probably log something to the JS console to alert developers.
Re: [whatwg] localStorage feedback
On Wed, Dec 2, 2009 at 2:06 AM, Ian Hickson i...@hixie.ch wrote: On Tue, 3 Nov 2009, Jeremy Orlow wrote: If we do this, we need to re-visit ways that scripts can tell whether the lock has been dropped. I can't remember which idea was most in favor last time we talked about it, but a counter that increments every time LocalStorage is unlocked sticks out in my mind. (Scripts can check the counter, do something that could cause unlocking, and then verify the counter is still the same after.) The counter could be a boolean -- does the event loop have the lock [for the origin of the first script]. I would recommend implementing this as a property window.navigator.webkitStorageUpdatesLocked which is true when the lock is obtained and false when it is released. If people find it useful, we can add it to the language. If it's a boolean, it's possible that it was implicitly unlocked and then re-locked. If you have a counter, you can simply check that the count is the same as when you started. Another option that just came to mind is to have some flag that says throw an exception whenever there's been a serialization violation. Could you elaborate on this? You'd have some boolean property that defaults to false (say navigator.throwOnStorageMutexDeadlocks) that when set to true would cause an exception to be thrown either immediately or the next time you try to do a localStorage operation before either calling navigator.yieldForStorageUpdates or the task ending. On Wed, 25 Nov 2009, Jeremy Orlow wrote: I know that we've discussed approximations of run to completion before, but maybe it's worth one more shot: What if on the first use of document.cookie or local storage we took a snapshot of both and used that during the task's execution. All writes would be queued up until the task finishes, at which point they'd be written to the central version of the cookie and/or local storage. This would provide a consistent view of data for the duration of the task and would solve almost all the atomicity problems except |document.cookie = document.cookie + foo;|. For that, I'd suggest adding a method that allows scripts to do atomic modifications to storage within a callback. I can understand everyone's desire to have completely serializable semantics for local storage access and cookies (if you don't count the servers' interaction with them), but maybe we need to go back to use cases. In a world with WebDatabase/WebSimpleDB, I really don't see anyone turning to LocalStorage except for more basic uses. Most of which I'm guessing need consistent reads much more than serialization of everything. And let's be realistic. IE has had this problem with document.cookie for a long time. And IE8 now has this problem with localStorage. Given that in the best case (MS and all others implement the storage mutex) web developers will not be able to assume localStorage and document.cookie access is atomic for many years at a minimum, I think we're being pretty unrealistic about how much the storage mutex is going to improve anyone's life. Let's come up with an approximation, give developers a callback for atomic access, and be done with it. As far as I can tell, this wouldn't remove race problems. It would in fact make it impossible to avoid them, since as far as I can tell you can't build a sane locking mechanism based on the above. I don't understand what you mean by building a sane locking mechanism. The idea is that we'd have near-run-to-completion semantics when using local storage normally (via repeatable reads semantics on localStorage and document.cookie) and actual run-to-completion semantics (serializable) if you use the callback based mechanism for localStorage access. Given that network based cookie access is essentially repeatable read semantics and both cookie and localStorage in IE and Chrome aren't even that currently, I'd say that such semantics seem good enough. The fact that Chrome has never gotten any complaints about the racy behavior of its current document.cookie implementation is some evidence to support that. J
Re: [whatwg] localStorage feedback
On Wed, 2 Dec 2009, Jeremy Orlow wrote: On Wed, Dec 2, 2009 at 2:06 AM, Ian Hickson i...@hixie.ch wrote: On Tue, 3 Nov 2009, Jeremy Orlow wrote: If we do this, we need to re-visit ways that scripts can tell whether the lock has been dropped. I can't remember which idea was most in favor last time we talked about it, but a counter that increments every time LocalStorage is unlocked sticks out in my mind. (Scripts can check the counter, do something that could cause unlocking, and then verify the counter is still the same after.) The counter could be a boolean -- does the event loop have the lock [for the origin of the first script]. I would recommend implementing this as a property window.navigator.webkitStorageUpdatesLocked which is true when the lock is obtained and false when it is released. If people find it useful, we can add it to the language. If it's a boolean, it's possible that it was implicitly unlocked and then re-locked. If you have a counter, you can simply check that the count is the same as when you started. Oh, I see. I misunderstood how the counter would work (I thought you meant it would go down again when unlocking). Yeah, that would work too. webkitStorageUpdatesLockedCount or something. Another option that just came to mind is to have some flag that says throw an exception whenever there's been a serialization violation. Could you elaborate on this? You'd have some boolean property that defaults to false (say navigator.throwOnStorageMutexDeadlocks) that when set to true would cause an exception to be thrown either immediately or the next time you try to do a localStorage operation before either calling navigator.yieldForStorageUpdates or the task ending. This would basically just be a development aid, then? I would be a bit worried about people accidentally enabling that kind of thing in production code and ending up with even more confusing user-facing bugs. It's generally been the kind of thing we've avoided standardising in the platform. It might make sense as some kind of UA-specific development aid, though. On Wed, 25 Nov 2009, Jeremy Orlow wrote: I know that we've discussed approximations of run to completion before, but maybe it's worth one more shot: What if on the first use of document.cookie or local storage we took a snapshot of both and used that during the task's execution. All writes would be queued up until the task finishes, at which point they'd be written to the central version of the cookie and/or local storage. This would provide a consistent view of data for the duration of the task and would solve almost all the atomicity problems except |document.cookie = document.cookie + foo;|. For that, I'd suggest adding a method that allows scripts to do atomic modifications to storage within a callback. I can understand everyone's desire to have completely serializable semantics for local storage access and cookies (if you don't count the servers' interaction with them), but maybe we need to go back to use cases. In a world with WebDatabase/WebSimpleDB, I really don't see anyone turning to LocalStorage except for more basic uses. Most of which I'm guessing need consistent reads much more than serialization of everything. And let's be realistic. IE has had this problem with document.cookie for a long time. And IE8 now has this problem with localStorage. Given that in the best case (MS and all others implement the storage mutex) web developers will not be able to assume localStorage and document.cookie access is atomic for many years at a minimum, I think we're being pretty unrealistic about how much the storage mutex is going to improve anyone's life. Let's come up with an approximation, give developers a callback for atomic access, and be done with it. As far as I can tell, this wouldn't remove race problems. It would in fact make it impossible to avoid them, since as far as I can tell you can't build a sane locking mechanism based on the above. I don't understand what you mean by building a sane locking mechanism. The idea is that we'd have near-run-to-completion semantics when using local storage normally (via repeatable reads semantics on localStorage and document.cookie) and actual run-to-completion semantics (serializable) if you use the callback based mechanism for localStorage access. If we decide to introduce a whole new API, then there are lots of things we could do that would make this much better, yes. I am not planning on introducing a fourth new storage API this year, though. I figure the Web needs time to get a handle on the ones we're introducing already. Without the additional API, I think it makes the storage mechanism much less reliable than no storage mutex at all. It would make it much more likely to have a
Re: [whatwg] localStorage feedback
On Mon, Nov 2, 2009 at 3:46 PM, Robert O'Callahan rob...@ocallahan.orgwrote: On Tue, Nov 3, 2009 at 6:36 AM, Darin Fisher da...@chromium.org wrote: 1a) Given a page (domain A) containing an iframe (domain B), have the outer page navigate the inner frame to about:blank. This navigation completes synchronously, and the unload handler for the iframe runs before the navigation request completes. This is true of all browsers. 1b) Suppose the inner page has a pending XMLHttpRequest when the outer frame navigates the inner frame. The XHR's onabort handler would run before the navigation to about:blank completes. These are really the same problem: synchronous cross-domain about:blank navigation. If navigation to about:blank has to be synchronous, then I guess it needs to drop the lock, at least in the cross-domain case. That's correct. My point is simple: Here is another case where nesting can happen that hadn't been foreseen. Trying to foresee all such issues is difficult. Will we just keep amending the spec each time we find such a possible case? I think it is far saner to say that any nesting leads to unlocking the storage mutex. The spec can then list cases where this nesting might occur. 2) Set a break point in the Mozilla JS debugger. This runs a nested event loop each time you single step so that it can drive the rest of the browser UI. 3) Install a Firefox extension that runs a nested event loop in response to an event generated by content. I debugged many Firefox crashes resulting from extensions that do this kind of thing for various reasons. These are internal Mozilla issues and should not be allowed to influence the design of the Web platform. They'll probably change for multi-process anyway. OK, but my point is that the spec should afford implementors with the ability to unlock the storage mutex at other times for reasons not mentioned in the spec. I'm not convinced. Look at Google Maps and street view. Gmail uses more Flash now than it used to. For new features, sure. But are they reimplementing existing browser-based functionality to use plugins instead? I think it is sufficient to just talk in the context of new features. A JS library or component grows a new feature that suddenly starts using a plugin. Now, API calls that were not supposed to touch plugins start touching plugins, and the storage mutex gets dropped. What will you do for Gecko when it supports content processes? Implement the spec, I hope! It seems odd to me that this behavior was put into the spec without any implementation experience to guide it. The only multi-process implementations that I know of do not have a storage mutex. Lots of things are in the spec without implementation experience. I think we have time to collect more experience on this issue with multi-process browsers and revise the spec in light of it. OK. Please note my objection to the storage mutex. -Darin
Re: [whatwg] localStorage feedback
On Wed, Nov 4, 2009 at 6:51 AM, Darin Fisher da...@chromium.org wrote: That's correct. My point is simple: Here is another case where nesting can happen that hadn't been foreseen. Trying to foresee all such issues is difficult. Yes. Will we just keep amending the spec each time we find such a possible case? I would. OK, but my point is that the spec should afford implementors with the ability to unlock the storage mutex at other times for reasons not mentioned in the spec. I disagree, because this gives implementors freedom to drop the mutex in situations that might really just be fixable implementation bugs. I think our positions are clear now so we'll just have to agree to disagree. I'm not convinced. Look at Google Maps and street view. Gmail uses more Flash now than it used to. For new features, sure. But are they reimplementing existing browser-based functionality to use plugins instead? I think it is sufficient to just talk in the context of new features. A JS library or component grows a new feature that suddenly starts using a plugin. Now, API calls that were not supposed to touch plugins start touching plugins, and the storage mutex gets dropped. That only matters if they start using the new feature in the middle of a localStorage transaction. That seems possible, but unlikely, to me. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] localStorage feedback
On Tue, Nov 3, 2009 at 9:51 AM, Darin Fisher da...@chromium.org wrote: On Mon, Nov 2, 2009 at 3:46 PM, Robert O'Callahan rob...@ocallahan.orgwrote: On Tue, Nov 3, 2009 at 6:36 AM, Darin Fisher da...@chromium.org wrote: 1a) Given a page (domain A) containing an iframe (domain B), have the outer page navigate the inner frame to about:blank. This navigation completes synchronously, and the unload handler for the iframe runs before the navigation request completes. This is true of all browsers. 1b) Suppose the inner page has a pending XMLHttpRequest when the outer frame navigates the inner frame. The XHR's onabort handler would run before the navigation to about:blank completes. These are really the same problem: synchronous cross-domain about:blank navigation. If navigation to about:blank has to be synchronous, then I guess it needs to drop the lock, at least in the cross-domain case. That's correct. My point is simple: Here is another case where nesting can happen that hadn't been foreseen. Trying to foresee all such issues is difficult. Will we just keep amending the spec each time we find such a possible case? I think it is far saner to say that any nesting leads to unlocking the storage mutex. The spec can then list cases where this nesting might occur. 2) Set a break point in the Mozilla JS debugger. This runs a nested event loop each time you single step so that it can drive the rest of the browser UI. 3) Install a Firefox extension that runs a nested event loop in response to an event generated by content. I debugged many Firefox crashes resulting from extensions that do this kind of thing for various reasons. These are internal Mozilla issues and should not be allowed to influence the design of the Web platform. They'll probably change for multi-process anyway. OK, but my point is that the spec should afford implementors with the ability to unlock the storage mutex at other times for reasons not mentioned in the spec. If we do this, we need to re-visit ways that scripts can tell whether the lock has been dropped. I can't remember which idea was most in favor last time we talked about it, but a counter that increments every time LocalStorage is unlocked sticks out in my mind. (Scripts can check the counter, do something that could cause unlocking, and then verify the counter is still the same after.) Another option that just came to mind is to have some flag that says throw an exception whenever there's been a serialization violation. Just to be clear for those following along at home: If we (Chrome) implement the storage mutex for the next version of Chrome, it'll have to be the way Darin's describing (mainly for schedule based reasons). It would be nice to eventually fix all the cases that cause unnecessary unlocking, but that's going to take a while. I'm not convinced. Look at Google Maps and street view. Gmail uses more Flash now than it used to. For new features, sure. But are they reimplementing existing browser-based functionality to use plugins instead? I think it is sufficient to just talk in the context of new features. A JS library or component grows a new feature that suddenly starts using a plugin. Now, API calls that were not supposed to touch plugins start touching plugins, and the storage mutex gets dropped. What will you do for Gecko when it supports content processes? Implement the spec, I hope! It seems odd to me that this behavior was put into the spec without any implementation experience to guide it. The only multi-process implementations that I know of do not have a storage mutex. Lots of things are in the spec without implementation experience. I think we have time to collect more experience on this issue with multi-process browsers and revise the spec in light of it. OK. Please note my objection to the storage mutex. And mine. :-)
Re: [whatwg] localStorage feedback
On Sun, Nov 1, 2009 at 3:53 AM, Darin Fisher da...@chromium.org wrote: On Fri, Oct 30, 2009 at 1:36 AM, Robert O'Callahan rob...@ocallahan.orgwrote: On Fri, Oct 30, 2009 at 7:27 PM, Darin Fisher da...@chromium.org wrote: You are right that the conditions are specific, but I don't know that that is the exhaustive list. Rather than defining unlock points, I would implement implicit unlocking by having any nested attempt to acquire a lock cause the existing lock to be dropped. Nesting can happen in the cases you mention, but depending on the UA, it may happen for other reasons too. What reasons? If these reasons are situations where it's fundamentally difficult, impossible, or non-performant to follow the spec, we should change the spec. Otherwise this would just be a bug in the UA. My point is that it is difficult to ensure that all situations where nesting can occur are understood apriori and that the list doesn't change over time. Because we are talking about multi-threading synchronization in a very complex system, I would much prefer a more isolated and less fragile solution. Unlock if yieldForStorageUpdates is called. Unlock when returning from script execution. Unlock if another attempt to lock occurs (any form of nesting). In the third case, I'd probably log something to the JS console to alert developers. I believe this simple implementation covers most of the cases enumerated in the spec, and it has the property of being easier to reason about and easier to support (more future proof). I think this would make the spec too dependent on implementation details. If your implementation needlessly or accidentally nests script execution --- e.g. by firing an event synchronously that should be, or could be, asynchronous --- then you'd still conform to your spec while the behaviour you present to authors gets quietly worse. If your description is (or can be, after suitable modifications) equivalent to what the spec currently says, but the equivalence is subtle (which it would be!), then I think they should *both* be in the spec, and the spec should say they're equivalent. Then if we find they're not equivalent, we clearly have a bug in the spec which must be fixed --- not carte blanche to proceed in an undesirable direction. It would be a sort of spec-level assertion. For example, a JS library might evolve to use flash for something small (like storage or sound) that it previously didn't use when I first developed my code. Voila, now my storage lock is released out from under me. This example still sounds overly contrived to me. Nevertheless, it seems strange to say that because there might be a few unexpected race conditions, you have decided to allow a much larger set of unexpected race conditions. Why is it contrived? Because libraries tend to initially use plugins and move towards using core browser functionality, not the other way around. But even if these library issues aren't contrived, I don't see how they justify making things a lot more unpredictable for everyone. What will you do for Gecko when it supports content processes? Implement the spec, I hope! Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] localStorage feedback
On Mon, Nov 2, 2009 at 1:28 AM, Robert O'Callahan rob...@ocallahan.orgwrote: On Sun, Nov 1, 2009 at 3:53 AM, Darin Fisher da...@chromium.org wrote: On Fri, Oct 30, 2009 at 1:36 AM, Robert O'Callahan rob...@ocallahan.orgwrote: On Fri, Oct 30, 2009 at 7:27 PM, Darin Fisher da...@chromium.orgwrote: You are right that the conditions are specific, but I don't know that that is the exhaustive list. Rather than defining unlock points, I would implement implicit unlocking by having any nested attempt to acquire a lock cause the existing lock to be dropped. Nesting can happen in the cases you mention, but depending on the UA, it may happen for other reasons too. What reasons? If these reasons are situations where it's fundamentally difficult, impossible, or non-performant to follow the spec, we should change the spec. Otherwise this would just be a bug in the UA. My point is that it is difficult to ensure that all situations where nesting can occur are understood apriori and that the list doesn't change over time. Because we are talking about multi-threading synchronization in a very complex system, I would much prefer a more isolated and less fragile solution. Unlock if yieldForStorageUpdates is called. Unlock when returning from script execution. Unlock if another attempt to lock occurs (any form of nesting). In the third case, I'd probably log something to the JS console to alert developers. I believe this simple implementation covers most of the cases enumerated in the spec, and it has the property of being easier to reason about and easier to support (more future proof). I think this would make the spec too dependent on implementation details. If your implementation needlessly or accidentally nests script execution --- e.g. by firing an event synchronously that should be, or could be, asynchronous --- then you'd still conform to your spec while the behaviour you present to authors gets quietly worse. If your description is (or can be, after suitable modifications) equivalent to what the spec currently says, but the equivalence is subtle (which it would be!), then I think they should *both* be in the spec, and the spec should say they're equivalent. Then if we find they're not equivalent, we clearly have a bug in the spec which must be fixed --- not carte blanche to proceed in an undesirable direction. It would be a sort of spec-level assertion. I think the spec currently calls attention to only some situations that could lead to nesting of implicitly acquired storage locks. I previously described some other situations, which you and others indicated should be treated as WebKit and IE bugs. I didn't look very far to dig those up. After some more thought, I came up with these additional cases that the spec doesn't cover: 1a) Given a page (domain A) containing an iframe (domain B), have the outer page navigate the inner frame to about:blank. This navigation completes synchronously, and the unload handler for the iframe runs before the navigation request completes. This is true of all browsers. 1b) Suppose the inner page has a pending XMLHttpRequest when the outer frame navigates the inner frame. The XHR's onabort handler would run before the navigation to about:blank completes. 2) Set a break point in the Mozilla JS debugger. This runs a nested event loop each time you single step so that it can drive the rest of the browser UI. 3) Install a Firefox extension that runs a nested event loop in response to an event generated by content. I debugged many Firefox crashes resulting from extensions that do this kind of thing for various reasons. For example, a JS library might evolve to use flash for something small (like storage or sound) that it previously didn't use when I first developed my code. Voila, now my storage lock is released out from under me. This example still sounds overly contrived to me. Nevertheless, it seems strange to say that because there might be a few unexpected race conditions, you have decided to allow a much larger set of unexpected race conditions. Why is it contrived? Because libraries tend to initially use plugins and move towards using core browser functionality, not the other way around. But even if these library issues aren't contrived, I don't see how they justify making things a lot more unpredictable for everyone. I'm not convinced. Look at Google Maps and street view. Gmail uses more Flash now than it used to. Wave uses Gears for a variety of little things. There's a cool video gadget that swaps between HTML5 video or Flash depending on the browser and the target media. What will you do for Gecko when it supports content processes? Implement the spec, I hope! It seems odd to me that this behavior was put into the spec without any implementation experience to guide it. The only multi-process implementations that I know of do not have a storage mutex. -Darin
Re: [whatwg] localStorage feedback
On Tue, Nov 3, 2009 at 6:36 AM, Darin Fisher da...@chromium.org wrote: 1a) Given a page (domain A) containing an iframe (domain B), have the outer page navigate the inner frame to about:blank. This navigation completes synchronously, and the unload handler for the iframe runs before the navigation request completes. This is true of all browsers. 1b) Suppose the inner page has a pending XMLHttpRequest when the outer frame navigates the inner frame. The XHR's onabort handler would run before the navigation to about:blank completes. These are really the same problem: synchronous cross-domain about:blank navigation. If navigation to about:blank has to be synchronous, then I guess it needs to drop the lock, at least in the cross-domain case. 2) Set a break point in the Mozilla JS debugger. This runs a nested event loop each time you single step so that it can drive the rest of the browser UI. 3) Install a Firefox extension that runs a nested event loop in response to an event generated by content. I debugged many Firefox crashes resulting from extensions that do this kind of thing for various reasons. These are internal Mozilla issues and should not be allowed to influence the design of the Web platform. They'll probably change for multi-process anyway. I'm not convinced. Look at Google Maps and street view. Gmail uses more Flash now than it used to. For new features, sure. But are they reimplementing existing browser-based functionality to use plugins instead? What will you do for Gecko when it supports content processes? Implement the spec, I hope! It seems odd to me that this behavior was put into the spec without any implementation experience to guide it. The only multi-process implementations that I know of do not have a storage mutex. Lots of things are in the spec without implementation experience. I think we have time to collect more experience on this issue with multi-process browsers and revise the spec in light of it. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] localStorage feedback
On Fri, Oct 30, 2009 at 1:36 AM, Robert O'Callahan rob...@ocallahan.orgwrote: On Fri, Oct 30, 2009 at 7:27 PM, Darin Fisher da...@chromium.org wrote: You are right that the conditions are specific, but I don't know that that is the exhaustive list. Rather than defining unlock points, I would implement implicit unlocking by having any nested attempt to acquire a lock cause the existing lock to be dropped. Nesting can happen in the cases you mention, but depending on the UA, it may happen for other reasons too. What reasons? If these reasons are situations where it's fundamentally difficult, impossible, or non-performant to follow the spec, we should change the spec. Otherwise this would just be a bug in the UA. My point is that it is difficult to ensure that all situations where nesting can occur are understood apriori and that the list doesn't change over time. Because we are talking about multi-threading synchronization in a very complex system, I would much prefer a more isolated and less fragile solution. Unlock if yieldForStorageUpdates is called. Unlock when returning from script execution. Unlock if another attempt to lock occurs (any form of nesting). In the third case, I'd probably log something to the JS console to alert developers. I believe this simple implementation covers most of the cases enumerated in the spec, and it has the property of being easier to reason about and easier to support (more future proof). For example, a JS library might evolve to use flash for something small (like storage or sound) that it previously didn't use when I first developed my code. Voila, now my storage lock is released out from under me. This example still sounds overly contrived to me. Nevertheless, it seems strange to say that because there might be a few unexpected race conditions, you have decided to allow a much larger set of unexpected race conditions. Why is it contrived? Many developers use high level toolkits to get their work done (e.g., jquery, prototype, dojo, google maps api, etc.). People are often one step removed from working directly with the web platform APIs. They have no idea what all is going on under the covers of those libraries, and that's a fine thing. The idea of unlocking whenever there is nesting occurred to me when Jeremy and I were discussing how to implement unlocking for all of the cases enumerated in the spec. It equates to a good number of places in the code that are quite separated from one another. It seems very fragile to ensure that all of those cases continue to be hooked properly. I think it is very hard to test that we get it right now and in the future. But, if we step back, we realize that the implicit unlocking is all about dealing with nesting of locks. So, I think it is _way_ better to just unlock the existing lock if an attempt is made to acquire a nested lock. At this point, I'm not favoring implementing the storage mutex in Chrome. I don't think we will have it in our initial implementation of LocalStorage. I think web developers that care will have to find another way to manage locking, like using a Web Database transaction or coordinating with a Shared Worker. Have you considered just not implementing LocalStorage? If it's so difficult for authors to use correctly and to implement according to the spec, this seems like the best path to me. I have definitely considered it. I would of course prefer to drop LocalStorage and focus on something better. Chrome is unfortunately in a difficult spot given that everyone else has implemented LocalStorage (though not necessarily to spec). So, we are currently on track to support this feature without locking. In the future, we might add locking. I've also considered other solutions, like copy-on-write, which could obviously lead to data loss, but at least it would ensure stability/consistency within a scripts execution. I would like it if the spec were open to such implementations. What will you do for Gecko when it supports content processes? -Darin
Re: [whatwg] localStorage feedback
On Mon, Oct 12, 2009 at 7:07 PM, Ian Hickson i...@hixie.ch wrote: ... the problem here is that localStorage is a pile of global variables. we are trying to give people global variables without giving them tools to synchronize access to them. the claim i've heard is that developers are not savy enough to use those tools properly. i agree that developers tend to use tools without fully understanding them. ok, but then why are we giving them global variables? The global variables have implicit locks such that you can build the tools for explicit locking on top of them: // run this first, in one script block var id = localStorage['last-id'] + 1; localStorage['last-id'] = id; localStorage['email-ready-' + id] = 0; // begin // these can run each in separate script blocks as desired localStorage['email-subject-' + id] = subject; localStorage['email-from-' + id] = from; localStorage['email-to-' + id] = to; localStorage['email-body-' + id] = body; // run this last localStorage['email-ready-' + id] = 1; // commit Dividing up work like this into separate SCRIPT elements to scope the locking seems really awkward to me. On Thu, 24 Sep 2009, Darin Fisher wrote: The current API exposes race conditions to the web. The implicit dropping of the storage lock is that. In Chrome, we'll have to drop an existing lock whenever a new lock is acquired. That can happen due to a variety of really odd cases (usually related to nested loops or nested JS execution), which will be difficult for developers to predict, especially if they are relying on third-party JS libraries. This issue seems to be discounted for reasons I do not understand. You can only lose the lock in very specific conditions. Those conditions are rarely going to interact with code that actually does storage work in a way that relies on the lock: - changing document.domain - history.back(), .forward(), .go(n) - invoking a plugin - alert(), confirm(), prompt(), print() - showModalDialog() - yieldForStorageUpdates() I discussed this in more detail here: http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-September/023059.html You are right that the conditions are specific, but I don't know that that is the exhaustive list. Rather than defining unlock points, I would implement implicit unlocking by having any nested attempt to acquire a lock cause the existing lock to be dropped. Nesting can happen in the cases you mention, but depending on the UA, it may happen for other reasons too. This combined with the fact that most people use JS libraries means that the coder is not going to have an easy time knowing when these specific conditions are met. I don't think defining a set of allowed unlock points is sufficient to make this API not be a minefield for users. For example, a JS library might evolve to use flash for something small (like storage or sound) that it previously didn't use when I first developed my code. Voila, now my storage lock is released out from under me. At this point, I'm not favoring implementing the storage mutex in Chrome. I don't think we will have it in our initial implementation of LocalStorage. I think web developers that care will have to find another way to manage locking, like using a Web Database transaction or coordinating with a Shared Worker. Sorry to be a grump about this, but a cross-process lock that lasts until JS returns is just going to slow down the web. It is a really bad idea for that reason. -Darin On Tue, 8 Sep 2009, Chris Jones wrote: Can those in the first camp explain why mutex semantics is better than transaction semantics? And why it's desirable to have one DB spec specify transaction semantics (Web Database) and a second specify mutex semantics (localStorage)? I don't think it's desirable. It's just what we have, though an accident of history. Where we're at: localStorage can't really change. It is what it is. We have a better proposal, Web Database, but not everybody wants to implement it. To move forward, I would recommend that someone come up with a storage proposal with the following characteristics: * All major browsers vendors are willing to implement it. * Compatible with workers. * Doesn't have any race conditions. * Doesn't involve a cross-process mutex that blocks interaction. * Stores structured data. * Can be queried in arbitrary ways. * Doesn't expose authors to locking primitives. Then we can replace Web Database with it and we can move on. I suggest that the right venue for this discussion would be the W3C Web Apps group, at public-weba...@w3.org. On Wed, 9 Sep 2009, Darin Fisher wrote: What about navigating an iframe to a reference fragment, which could trigger a scroll event? The scrolling has to be done synchronously for compat with the web. You can only do that with same-domain pages, as far as I can tell. (Does that
Re: [whatwg] localStorage feedback
On Fri, Oct 30, 2009 at 7:27 PM, Darin Fisher da...@chromium.org wrote: You are right that the conditions are specific, but I don't know that that is the exhaustive list. Rather than defining unlock points, I would implement implicit unlocking by having any nested attempt to acquire a lock cause the existing lock to be dropped. Nesting can happen in the cases you mention, but depending on the UA, it may happen for other reasons too. What reasons? If these reasons are situations where it's fundamentally difficult, impossible, or non-performant to follow the spec, we should change the spec. Otherwise this would just be a bug in the UA. For example, a JS library might evolve to use flash for something small (like storage or sound) that it previously didn't use when I first developed my code. Voila, now my storage lock is released out from under me. This example still sounds overly contrived to me. Nevertheless, it seems strange to say that because there might be a few unexpected race conditions, you have decided to allow a much larger set of unexpected race conditions. At this point, I'm not favoring implementing the storage mutex in Chrome. I don't think we will have it in our initial implementation of LocalStorage. I think web developers that care will have to find another way to manage locking, like using a Web Database transaction or coordinating with a Shared Worker. Have you considered just not implementing LocalStorage? If it's so difficult for authors to use correctly and to implement according to the spec, this seems like the best path to me. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] localStorage feedback
On Fri, Oct 30, 2009 at 1:36 AM, Robert O'Callahan rob...@ocallahan.orgwrote: On Fri, Oct 30, 2009 at 7:27 PM, Darin Fisher da...@chromium.org wrote: You are right that the conditions are specific, but I don't know that that is the exhaustive list. Rather than defining unlock points, I would implement implicit unlocking by having any nested attempt to acquire a lock cause the existing lock to be dropped. Nesting can happen in the cases you mention, but depending on the UA, it may happen for other reasons too. What reasons? If these reasons are situations where it's fundamentally difficult, impossible, or non-performant to follow the spec, we should change the spec. Otherwise this would just be a bug in the UA. For example, a JS library might evolve to use flash for something small (like storage or sound) that it previously didn't use when I first developed my code. Voila, now my storage lock is released out from under me. This example still sounds overly contrived to me. Nevertheless, it seems strange to say that because there might be a few unexpected race conditions, you have decided to allow a much larger set of unexpected race conditions. I don't have a strong opinion either way on this one, but in general I'd say that making a race more subtle isn't usually much of a win.especially when it comes to debugging it. At this point, I'm not favoring implementing the storage mutex in Chrome. I don't think we will have it in our initial implementation of LocalStorage. I think web developers that care will have to find another way to manage locking, like using a Web Database transaction or coordinating with a Shared Worker. Have you considered just not implementing LocalStorage? If it's so difficult for authors to use correctly and to implement according to the spec, this seems like the best path to me. We'd love to, but it's difficult given that most of the other vendors have already implemented it. I also believe that Microsoft's browser exhibits the same races that Darin's talking about. So I'm not really sure how you could suggest that us not implementing it is better than implementing the status quo.
Re: [whatwg] localStorage feedback
On Fri, Oct 30, 2009 at 10:03 PM, Jeremy Orlow jor...@chromium.org wrote: I don't have a strong opinion either way on this one, but in general I'd say that making a race more subtle isn't usually much of a win.especially when it comes to debugging it. It's not just a matter of degree. If the UA follows the spec, authors can (fairly easily) write code that's race free. Darin's proposing to make it so they can't --- at least, not without external help like shared workers or Web Storage, in which case I wonder why one would use LocalStorage at all. (OK, I suppose one could rely on key/value atomicity and implement Lamport's bakery algorithm on LocalStorage alone. I hope to never see that...) Have you considered just not implementing LocalStorage? If it's so difficult for authors to use correctly and to implement according to the spec, this seems like the best path to me. We'd love to, but it's difficult given that most of the other vendors have already implemented it. I also believe that Microsoft's browser exhibits the same races that Darin's talking about. So I'm not really sure how you could suggest that us not implementing it is better than implementing the status quo. That is a good point. You do have the option of doing better than IE, though :-). Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] localStorage feedback
On Tue, Oct 13, 2009 at 4:07 AM, Ian Hickson i...@hixie.ch wrote: I've edited the latter text to indicate that the expiration should only be done at user option. On a device with limited storage, is the user option of having a device that still boots and operates a sufficient option? I'm looking at a device which tends to run out of space regularly but which wants to support web standards. If I visit 50 sites and collect junk for all of them, and i have a screen which can barely show 5 site names + 1 line descriptions for sites, then i'm not quite sure my user option of being able to conveniently manage those sites will really exist. User Agents always have the option of acting on the behalf of their user. Web sites should be encouraged to design their content in a way which warns the user about unsynchronized data and explains that they probably should rely on syncing or backing up their data. If I have a draft message in GMail, when I try to navigate away from it, it helpfully warns me that my email could be lost. Web applications shouldn't become less intelligent than that.
Re: [whatwg] localStorage feedback
On Tue, 27 Oct 2009, timeless wrote: On Tue, Oct 13, 2009 at 4:07 AM, Ian Hickson i...@hixie.ch wrote: I've edited the latter text to indicate that the expiration should only be done at user option. On a device with limited storage, is the user option of having a device that still boots and operates a sufficient option? I'm looking at a device which tends to run out of space regularly but which wants to support web standards. I don't know that that's possible. If I visit 50 sites and collect junk for all of them, and i have a screen which can barely show 5 site names + 1 line descriptions for sites, then i'm not quite sure my user option of being able to conveniently manage those sites will really exist. On such a device you probably shouldn't be letting all sites store data without the user opting into it. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] localStorage feedback
On Tue, 13 Oct 2009 04:20:56 +0200, Robert O'Callahan rob...@ocallahan.org wrote: Perhaps it should be defined in CSSOM, alongside scrollTop/scrollLeft? In Gecko, it's just something that fires asynchronously after a node's scrollTop/scrollLeft have changed. Scroll events for the viewport fire at the document and bubble to the window, scroll events on elements with 'overflow' not 'visible' fire at that element and don't bubble. I forwarded this to www-style by the way. It's on the pile for cssom-view now. -- Anne van Kesteren http://annevankesteren.nl/
Re: [whatwg] localStorage feedback
On Mon, Oct 12, 2009 at 19:07, Ian Hickson i...@hixie.ch wrote:
[SNIP]
On Wed, 23 Sep 2009, Brett Cannon wrote:
Before the move to structured clones one could tell if a key was set by
calling getItem() and seeing if it returned null (had to use === as
someone could have called setItem() w/ null, but that would be coerced
to a string for storage). But with the latest draft's switch to
structured clones that test no longer clearly differentiates between
whether the value returned by getItem() signifies that the key was not
set, or the key was set with the value null.
I believe you can test if a key is in the storage area using:
if (key in storage) { ... }
For example:
if ('document' in window.localStorage) { ... }
I didn't find that in the spec anywhere. Is it somehow implicit and I just
missed it? Or will it be specified sometime in the future?
And since I just subscribed to the mailing list, I was wondering if the
whole workers/localStorage discussion ended or not, as I can provide a
(potentially minor) real-world use-case for sharing access between the
page and worker if people want to hear it (in a new email of course).
I think everyone agrees that we need a storage mechanism in workers; the
question is what it should be. That's basically the same as the question
of what should happen with the Web Database spec -- I don't think we would
want to end up with multiple storage systems in workers. The answer to
this question depends on the result of this debate in the Web Apps WG.
Since I am not a w3c member and thus cannot subscribe to the Web Apps WG
mailing list, just want to say good luck to whomever pushes this. I truly
hope you guys can agree on a single storage solution that the browsers all
implement. It would definitely help keep my PhD thesis relevant after it's
finished. =)
-Brett
Re: [whatwg] localStorage feedback
On Tue, 13 Oct 2009, Jeremy Orlow wrote:
On Tue, Oct 13, 2009 at 3:40 PM, Ian Hickson i...@hixie.ch wrote:
On Tue, 13 Oct 2009, Brett Cannon wrote:
On Mon, Oct 12, 2009 at 19:07, Ian Hickson i...@hixie.ch wrote:
I believe you can test if a key is in the storage area using:
if (key in storage) { ... }
For example:
if ('document' in window.localStorage) { ... }
I didn't find that in the spec anywhere. Is it somehow implicit and I
just missed it? Or will it be specified sometime in the future?
It's the net effect of this line in the spec:
# The names of the supported named properties on a Storage object are the
# keys of each key/value pair currently present in the list associated
# with the object.
...combined with the definitions in the WebIDL spec.
It seems odd that there wouldn't be a corresponding method on
localStorage given that the deleter, getter, and setter all have them.
Personally I'd rather remove those methods, but I think we're stuck with
them at this point.
--
Ian Hickson U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] localStorage feedback
Found how to subscribe to the webapps list. Turns out I was trying to join
the working group and not the mailing list.
On Tue, Oct 13, 2009 at 16:28, Ian Hickson i...@hixie.ch wrote:
On Tue, 13 Oct 2009, Jeremy Orlow wrote:
On Tue, Oct 13, 2009 at 3:40 PM, Ian Hickson i...@hixie.ch wrote:
On Tue, 13 Oct 2009, Brett Cannon wrote:
On Mon, Oct 12, 2009 at 19:07, Ian Hickson i...@hixie.ch wrote:
I believe you can test if a key is in the storage area using:
if (key in storage) { ... }
For example:
if ('document' in window.localStorage) { ... }
I didn't find that in the spec anywhere. Is it somehow implicit and I
just missed it? Or will it be specified sometime in the future?
It's the net effect of this line in the spec:
# The names of the supported named properties on a Storage object are
the
# keys of each key/value pair currently present in the list associated
# with the object.
...combined with the definitions in the WebIDL spec.
It seems odd that there wouldn't be a corresponding method on
localStorage given that the deleter, getter, and setter all have them.
Personally I'd rather remove those methods, but I think we're stuck with
them at this point.
While usability does go up when you use things like setting keys/values
directly off of localStorage instead of using setItem(), etc. it does make
it much more difficult to provide a drop-in library to provide enhanced
features and fix compatibility issues thanks to JavaScript not supporting
thorough operator overloading. For instance, IE 8 doesn't provide a way to
do an equivalent 'length' attribute (until ECMAScript 5 is supported). And I
have no clue how to potentially fix any browser incompatibility that the
'in' operator might introduce.
And taking away the functions would force people like me to come up with
their own API instead of using the defined functions. Providing functions
mirroring things you can do directly with an object might be a pain, but it
does help overcome a shortcoming of JavaScript.
-Brett
[whatwg] localStorage feedback
On Thu, 17 Sep 2009, Jeremy Orlow wrote:
On Thu, Sep 17, 2009 at 1:32 AM, Ian Hickson i...@hixie.ch wrote:
I think we should be very careful before introducing a fourth storage
mechanism to make sure that whatever we introduce really is something
that's going to be very useful and really solve problems. I'd really
rather not rush into adding yet another mechanism at this point.
Sure. But what about the other idea Robert and Drew had (in the workers +
local storage thread) about just having a WorkerLocalStorage mechanism?
That's a fourth storage mechanism, so my comments above apply.
On Wed, 23 Sep 2009, Brett Cannon wrote:
Before the move to structured clones one could tell if a key was set by
calling getItem() and seeing if it returned null (had to use === as
someone could have called setItem() w/ null, but that would be coerced
to a string for storage). But with the latest draft's switch to
structured clones that test no longer clearly differentiates between
whether the value returned by getItem() signifies that the key was not
set, or the key was set with the value null.
I believe you can test if a key is in the storage area using:
if (key in storage) { ... }
For example:
if ('document' in window.localStorage) { ... }
And since I just subscribed to the mailing list, I was wondering if the
whole workers/localStorage discussion ended or not, as I can provide a
(potentially minor) real-world use-case for sharing access between the
page and worker if people want to hear it (in a new email of course).
I think everyone agrees that we need a storage mechanism in workers; the
question is what it should be. That's basically the same as the question
of what should happen with the Web Database spec -- I don't think we would
want to end up with multiple storage systems in workers. The answer to
this question depends on the result of this debate in the Web Apps WG.
On Wed, 23 Sep 2009, Jeremy Orlow wrote:
What are the use cases for wanting to store data beyond strings (and
what can be serialized into strings) in LocalStorage? I can't think of
any that outweigh the negatives:
1) From previous threads, I think it's fair to say that we can all
agreed that LocalStorage is a regrettable API (mainly due to its
synchronous nature). If so, it seems that making it more powerful and
thus more attractive to developers is just asking for trouble. After
all, the more people use it, the more lock contention there'll be, and
the more browser UI jank users will be sure to experience. This will
also be worse because it'll be easier for developers to store large
objects in LoaclStorage.
2) As far as I can tell, there's no where else in the spec where you
have to serialize structured clone(able) data to disk. Given that
LocalStorage is supposed to throw an exception if any ImageData is
contained and since File and FileData objects are legal, it seems as
though making LocalStorage handle structured clone data has a fairly
high cost to implementors. Not to mention that disallowing ImageData in
only this one case is not intuitive.
I think allowing structured clone(able) data in LocalStorage is a big
mistake. Enough so that, if SessionStorage and LocalStorage can't
diverge on this issue, it'd be worth taking the power away from
SessionStorage.
The main use case is storing File objects when offline for later upload. I
think that far outweighs the negatives you list above. We need this, and
there's no other storage mechanism that everyone agrees is good enough.
the problem here is that localStorage is a pile of global variables.
we are trying to give people global variables without giving them tools
to synchronize access to them. the claim i've heard is that developers
are not savy enough to use those tools properly. i agree that
developers tend to use tools without fully understanding them. ok, but
then why are we giving them global variables?
The global variables have implicit locks such that you can build the tools
for explicit locking on top of them:
// run this first, in one script block
var id = localStorage['last-id'] + 1;
localStorage['last-id'] = id;
localStorage['email-ready-' + id] = 0; // begin
// these can run each in separate script blocks as desired
localStorage['email-subject-' + id] = subject;
localStorage['email-from-' + id] = from;
localStorage['email-to-' + id] = to;
localStorage['email-body-' + id] = body;
// run this last
localStorage['email-ready-' + id] = 1; // commit
On Thu, 24 Sep 2009, Darin Fisher wrote:
The current API exposes race conditions to the web. The implicit
dropping of the storage lock is that. In Chrome, we'll have to drop an
existing lock whenever a new lock is acquired. That can happen due to a
variety of really odd cases (usually related to nested loops or nested
JS execution), which will be difficult for
Re: [whatwg] localStorage feedback
On Tue, Oct 13, 2009 at 3:07 PM, Ian Hickson i...@hixie.ch wrote: On Wed, 9 Sep 2009, Darin Fisher wrote: What about navigating an iframe to a reference fragment, which could trigger a scroll event? The scrolling has to be done synchronously for compat with the web. You can only do that with same-domain pages, as far as I can tell. (Does that really have to be synchronous? Later in the thread I mentioned that in Gecko, the 'scroll' event is asynchronous, which suggests that Web compatibility does not require it to be synchronous. Right now we don't define the 'scroll' event anywhere. What are the semantics it needs?) Perhaps it should be defined in CSSOM, alongside scrollTop/scrollLeft? In Gecko, it's just something that fires asynchronously after a node's scrollTop/scrollLeft have changed. Scroll events for the viewport fire at the document and bubble to the window, scroll events on elements with 'overflow' not 'visible' fire at that element and don't bubble. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
