Re: [whatwg] localStorage, the storage mutex, document.domain, and workers
These two statements are true... * We can't change the API * It is seriously flawed ... and therein lies the problem. I'm sad to have to say it... but I hope this withers and dies an early death. Putting this in the web platform for perpetuity is a mistake. I don't support the adoption of this into the platform. Time to go read the SimpleDatabase proposal. On Wed, 9 Sep 2009, Darin Fisher wrote: By the way, you can already pretty much create my acquireLock / releaseLock API on top of SharedWorkers today, but in a slightly crappier way. How? Since the API is completely async, you can't make a spinlock. You must not have read Darin's proposal. It wasn't a 'lock' at all. It's a completely async, well-factored primitive. void acqureFlag('name', callback); // returns immediately in all cases void releaseFlag('name'); // returns immediately in all cases The callback is called upon 'flag' acquisition. Its all yours until you call release. Completely async. I think its self-evident that this can be composed with a SharedWorker. Darin's was an example of a good proposal... simple on all dimensions, yet powerful and broadly applicable... what is not to like? Marry the 'flag' with a unlocked storage repository and viola, you have something... the whole is greater than the sum of the parts. Another lesson to be learned from the LocalStorage debacle is to decompose things, hashmaps+(implicit)locks+events... it slices and dices (and there's more)... it was a bad idea to jumble all that together... individually, minus the implicit, those would make for nice features. Also, regarding we can't change the API... well it did get changed... the application of additional implicit locking semantics to IE's API... that is a material change. On Thu, Sep 17, 2009 at 5:13 PM, Robert O'Callahan rob...@ocallahan.orgwrote: On Thu, Sep 17, 2009 at 8:32 PM, Ian Hickson i...@hixie.ch wrote: LESSONS LEARNT If we ever define a new API that needs a lock of some kind, the way to do it is to use a callback, so that the UA can wait for the lock asynchronously, and then run the callback once it has it. (This is what the Web Database spec does, for instance.) When we add more of these features, I think we will need a way to acquire multiple locks simultaneously before running a callback. (So if we had localStorage.runTransaction(function(storage) { ... }) and otherLockingThing.runTransaction(function(thing) { ... }), we could also have, for example, window.runTransaction(localStorage, otherLockingThing, function(storage, thing) { ... }).) So it may be worth thinking about what that API should be and what we will need to add to each feature spec to support it. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] localStorage, the storage mutex, document.domain, and workers
When we add more of these features, I think we will need a way to acquire multiple locks simultaneously before running a callback. (So if we had localStorage.runTransaction(function(storage) { ... }) and otherLockingThing.runTransaction(function(thing) { ... }), we could also have, for example, window.runTransaction(localStorage, otherLockingThing, function(storage, thing) { ... }).) So it may be worth thinking about what that API should be and what we will need to add to each feature spec to support it. void acquireFlags(['foo', 'bar', baz'], callback);
[whatwg] localStorage, the storage mutex, document.domain, and workers
SUMMARY: I haven't removed the storage mutex. I don't see any other workable solution to the problem. We can't change the API, as much as we'd like to, because it's already shipped in IE, and it would simply be dumb for us to screw Microsoft over here. It's hard enough to get them to implement specs without giving their management more reasons for not being on the cutting edge. We can't simply remove the storage mutex, because then we'll have race conditions up the wazoo, especially for localStorage. We could probably get away with letting document.cookie be unprotected, since IE has done that for years and cookies aren't a high-traffic API, but who knows how many bugs that's really causing. If implementations don't implement the mutex around document.cookie access, I can't blame them, but it's a risk that each implementor needs to evaluate on their own, the spec isn't going to condone it. I have made sure that as far as I am aware, there are no ways to synchronously invoke script from another origin in HTML5 without dropping the mutex (specifically, I've made history.back() and changing of document.domain drop the mutex), and I've made it so that you can only access localStorage objects of your effective script origin (i.e. changing document.domain means your localStorage is no longer accessible). This means that it should be possible to implement the storage mutex on a per-domain basis. This should minimise the potential damage. I haven't added localStorage to workers, because it'd be too easy to grab the mutex and try to see if Pi is finite using a brute-force approach, leaving all the other event loops that try to grab the mutex SOL. I haven't added an async callback to localStorage yet, because I don't imagine that most authors will use it given that they don't need to and see no problems when testing (you'd only see issues if you tested having two apps from the same domain both doing storage updates in long scripts). LESSONS LEARNT If we ever define a new API that needs a lock of some kind, the way to do it is to use a callback, so that the UA can wait for the lock asynchronously, and then run the callback once it has it. (This is what the Web Database spec does, for instance.) On Thu, 3 Sep 2009, timeless wrote: On Sun, Aug 30, 2009 at 4:06 AM, Ian Hicksoni...@hixie.ch wrote: Upon further consideration I've renamed getStorageUpdates() to yieldForStorageUpdates(). If getStorageUpdates() actually returned how *many* updates there were, it could be a vaguely useful name. If the answer is 0, then my application knows it doesn't need to try to figure out how the world changed, right? The answer 0 would only be useful if you were to grab the storage mutex immediately upon receiving it, which defeats the point of the call. On Fri, 4 Sep 2009, Jeremy Orlow wrote: I like this idea except for one problem: It doesn't tell whether something got changed without your knowledge. If you call alert, access a plugin, etc it's possible to drop the lock. I think some sort of global counter, variable, etc would be more valuable since it solves both problems. What's the use case for this? I don't understand what a script making use of this would look like. (I mean, I can see how to write a demo that shows using it, but how would a real site use it?) On Fri, 4 Sep 2009, Chris Jones wrote: I'd like to propose that HTML5 specify different schemes than a conceptual global storage mutex to provide consistency guarantees for localStorage and cookies. Cookies would be protected according to Benjamin Smedberg's post in the [whatwg] Storage mutex and cookies can lead to browser deadlock thread. Roughly, this proposal would give scripts a consistent view of document.cookie until they completed. AIUI this is stronger consistency than Google Chrome provides today, and anecdotal evidence suggests even their weaker consistency hasn't broken the web. I think if we're willing to run the risk of pages stomping on each other, we should just go ahead and not have locking at all, rather than trying to show a consistent view that we know to be a lie. That is, if this is going to fail when run in parallel: var count = getCookie('counter'); counter += 1; setCookie('data' + counter, data); setCookie('counter', counter); ...then there's not much point worrying about whether the view is consistent. Either way, data loss will occur. localStorage would be changed in a non-backwards-compatible way. I think this is a non-starter, as described above. [transactions] The problem with a system where transactions might fail based on timing is that there's not much for the author to do but just try again, and that's really not a good pattern to encourage. It's also likely that many authors won't notice the failure case, leading to just another kind of race condition very similar to what we're trying to avoid in the first place.
Re: [whatwg] localStorage, the storage mutex, document.domain, and workers
On Thu, Sep 17, 2009 at 1:32 AM, Ian Hickson i...@hixie.ch wrote: On Tue, 15 Sep 2009, Jonas Sicking wrote: First off, I agree that not having localStorage in workers is a big problem that we need to address. If I were designing the localStorage interface today I would use the above interface that you suggest. Grabbing localStorage can only be done asynchronously, and while you're using it, no one else can get a reference to it. This way there are no race conditions, but also no way for anyone to have to lock. So one solution is to do that in parallel to the current localStorage interface. Let's say we introduce a 'clientStorage' object. You can only get a reference to it using a 'getClientStorage' function. This function is available both to workers and windows. The storage is separate from localStorage so no need to worry about the 'storage mutex'. I think we should be very careful before introducing a fourth storage mechanism to make sure that whatever we introduce really is something that's going to be very useful and really solve problems. I'd really rather not rush into adding yet another mechanism at this point. Sure. But what about the other idea Robert and Drew had (in the workers + local storage thread) about just having a WorkerLocalStorage mechanism? Only workers would be able to access it and the interface would be exactly the same as LocalStorage. The only difference is that the data is in a separate area (and pages/workers cannot share data). (Though we could, in the future, add such an async interface if we wanted to allow pages to access it.) I know the burden for implementation in WebKit is low, and I imagine (though don't know for sure) that the burden would be low in other browsers as well.
Re: [whatwg] localStorage, the storage mutex, document.domain, and workers
On Thu, Sep 17, 2009 at 10:53 AM, Jeremy Orlow jor...@chromium.org wrote: On Thu, Sep 17, 2009 at 1:32 AM, Ian Hickson i...@hixie.ch wrote: On Tue, 15 Sep 2009, Jonas Sicking wrote: First off, I agree that not having localStorage in workers is a big problem that we need to address. If I were designing the localStorage interface today I would use the above interface that you suggest. Grabbing localStorage can only be done asynchronously, and while you're using it, no one else can get a reference to it. This way there are no race conditions, but also no way for anyone to have to lock. So one solution is to do that in parallel to the current localStorage interface. Let's say we introduce a 'clientStorage' object. You can only get a reference to it using a 'getClientStorage' function. This function is available both to workers and windows. The storage is separate from localStorage so no need to worry about the 'storage mutex'. I think we should be very careful before introducing a fourth storage mechanism to make sure that whatever we introduce really is something that's going to be very useful and really solve problems. I'd really rather not rush into adding yet another mechanism at this point. Sure. But what about the other idea Robert and Drew had (in the workers + local storage thread) about just having a WorkerLocalStorage mechanism? Only workers would be able to access it and the interface would be exactly the same as LocalStorage. The only difference is that the data is in a separate area (and pages/workers cannot share data). (Though we could, in the future, add such an async interface if we wanted to allow pages to access it.) I know the burden for implementation in WebKit is low, and I imagine (though don't know for sure) that the burden would be low in other browsers as well. This is what I think we should do. I do think that an async accessor in pages would increase its utility significantly. In fact, if we did that the interface could even work as an alternative to localStorage on the main thread for applications that worry about the locking, and thus unresponsive UI, that the storage mutex creates. / Jonas
Re: [whatwg] localStorage, the storage mutex, document.domain, and workers
On Thu, Sep 17, 2009 at 8:32 PM, Ian Hickson i...@hixie.ch wrote: LESSONS LEARNT If we ever define a new API that needs a lock of some kind, the way to do it is to use a callback, so that the UA can wait for the lock asynchronously, and then run the callback once it has it. (This is what the Web Database spec does, for instance.) When we add more of these features, I think we will need a way to acquire multiple locks simultaneously before running a callback. (So if we had localStorage.runTransaction(function(storage) { ... }) and otherLockingThing.runTransaction(function(thing) { ... }), we could also have, for example, window.runTransaction(localStorage, otherLockingThing, function(storage, thing) { ... }).) So it may be worth thinking about what that API should be and what we will need to add to each feature spec to support it. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]