Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
Great analysis. I only have a few comments/questions: On Wed, Sep 9, 2009 at 1:41 PM, Chris Jones cjo...@mozilla.com wrote: Jeremy Orlow wrote: On Wed, Sep 9, 2009 at 4:39 AM, Chris Jones cjo...@mozilla.com mailto: cjo...@mozilla.com wrote: Aaron Boodman wrote: On Tue, Sep 8, 2009 at 11:23 AM, Chris Jonescjo...@mozilla.com mailto:cjo...@mozilla.com wrote: In general, I agree with Rob about this proposal. What problem with storage mutex as spec'd today does your proposal solve? The spec requires a single storage mutex for the entire UA. Therefore in a MELUA a web page can become unresponsive while waiting for some other page to give up the lock. This is not good and something we have tried to avoid everywhere else in the spec. Attempts to address this by doing per-origin locks wind up with deadlocks being possible. Aaron Boodman wrote: On Tue, Sep 8, 2009 at 1:41 AM, Robert O'Callahanrob...@ocallahan.org mailto:rob...@ocallahan.org wrote: What is the intended semantics here? Chris' explicit commitTransaction would throw an exception if the transaction was aborted due to data inconsistency, leaving it up to the script to retry --- and making it clear to script authors that non-storage side effects during the transaction are not undone. How would you handle transaction aborts? Calls to transaction() are queued and executed serially per-origin with exclusive access. There is no such thing as a transaction abort because there cannot be consistency problems because of the serialized access. No, transactions can still fail. They can fail in ways immediately hidden from the script that requested them if the UA has to interrupt the conceptually executing transaction in the ways enumerated in a separate branch of this thread. Later script executions can observe inconsistent state unless more is specified by your proposal. Transactions can also fail visibly if write-to-disk fails (probably also in other ways I haven't considered). It's not clear what should happen wrt to your proposal in this case. If so, I agree with roc's responses to them that they could probably be handled without surfacing errors to the developer. OTOH, I'm not really against adding the concept of fallibility here. In fact, I believe that the Synchronous database API describes the same transaction semantics as I proposed in the OP. That spec adds implicit begin/commitTransaction and read-only transactions, but otherwise the semantics are the same. So I'd like to amend my original proposal to be Use Synchronous Web Database API transaction semantics. Except do not offer readTransaction: a transaction is implicitly a read-only transaction if only getItem() is called on localStorage from within localStorage.transaction(). Agree. That is what I was trying to propose, too. I'm not sure where we disagree :). Is it just that my proposal has no concept of errors? I'm not against adding them, mainly I was trying to keep my proposal simple for purposes of discussion. Ay, there's the rub: I think the disagreement is between mutex vs. transaction semantics. So far, I think perhaps mutex has been used as shorthand for transaction. But they aren't the same. I think we all agree that a script may fail to modify localStorage in some situations (irrespective of global mutex vs. per-domain mutex). One camp, wanting mutex semantics, would prefer to pretend that the failures never happen and let scripts clean up the mess (partially-applied changes) if they do occur. This is semantically broken, IMHO. The second camp, wanting transaction semantics, explicitly acknowledge to web authors that localStorage is fallible, guarantee that modifications to localStorage are atomic, and notify scripts when modifications can't be made atomically. This is the same approach taken by Web Database. IMHO, this is much better semantically because (i) it gives web apps stronger guarantees; and (ii) it makes the discussion about global
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
Jeremy Orlow wrote:
Those who want a queue. I.e. those who want an asynchronous
callback based interface and the UA will only call one callback
at a time. Perhaps on a per-origin basis. Note that this can
never fail, need to be rolled back, etc.
This sounds to me like { traditional transactions, async,
unobservable transaction failures } which is the same as your first
camp above except async only. Or are you proposing that the unit of
atomicity/consistency is not all operations performed in the
callback; i.e., that modifications done in the callback can be
partially applied?
It's just an implementational difference. A queue means that the event
loop can continue processing stuff while waiting for the 'lock' (which
maybe is better described as an 'update token' or something). If you
implement it as a lock (which you would for a synchronous interface)
then the event loop is blocked.
OK, agreed. What we describe reduce to the same thing --- the trylock
implementation Rob O'Callahan and I discussed would also never block the
event loop. But I better understand now what you have in mind. (I'm
not sure which is better, but that's what implementation details are all
about!)
I believe Aaron is in the queue camp with me. I'm becoming more
and more convinced that Chromium should/will not implement the
storage mutex at all (even for LocalStorage) unless we can come
up with a way for event loops to not be blocked. And, as far as
I can tell, Async interfaces are the only way to accomplish this.
In general, agreed. I still believe that a sync API
The problem with a sync interface, especially if it's one that can be
held after the top level script context, is deadlock issues with
WebDatabase (and possibly others). What's there now doesn't have this
issue because you'd never have the lock when calling the database
transaction callback.
I didn't think of Web Database, although I had in mind a nonblocking
implementation of localStorage (i.e., no mutex) that would prevent it
from participating in deadlocks.
But in general, nesting transactions, both { localStorage { Web DataBase
} } and { localStorage { localStorage } }, is something the spec should
explicitly disallow. There's not a clearly best way to resolve the
semantic problems that arise. (Note that preventing nested transactions
also eliminates deadlock concerns for mutex implementations.)
with exposed transaction failures
You'll only have transaction failures in an optimistic transaction
model, right? So is that what you're suggesting?
(as I proposed in the OP) and the right implementation could do
quite well. But I now think that an async version of that same API
could perform even better. In addition, that API is most flexible
in terms of possible UA implementations.
IOW, I think that { traditional transactions, async, observable
failures } subsumes both { traditional transactions, sync,
observable failures } (OP's proposal) *and* { traditional
transactions, async, unobservable failures } (your and Aaron's
proposal).
IMHO there are two remaining questions: first, whether the ideal
localStorage transactional API should allow observable transaction
failures. I believe that it should, as this allows for the widest
variety of efficient implementations without changing ACID (best
effort) guarantees given to authors or significantly complicating
the localStorage API.
What failures could there be in a pesimistic/queue model?
I'm trying to think of a universal failure, something that could arise
in any implementation of any proposal. I think that universal failure
is exceeded localStorage space quota. (I believe there are others,
but one is enough for this discussion.)
So there are really two questions: first, what happens if you detect
quota exceeded in the middle of a localStorage.transaction() callback?
Are the modifications made *up to* the quota exceeded error applied to
backing store? Or are they rolled back? This is the
atomicity/consistency design decision I described: what is the unit of
atomic modification (all applied or none applied).
And second, can scripts be notified that the transaction failed (or was
only partially applied) because of quota exceeded?
AFAICT this is irrespective of sync/async/optimistic/pessimistic/queue.
Any implementation has to deal with this error somehow.
In the current localStorage spec, if a setItem() fails because space is
exhausted, it raises QUOTA_EXCEEDED_ERR and does *not* apply any part of
the change in setItem(). That is, setItem() is atomic --- it either
applies or not (not partially applied). But modifications to
localStorage *before* that failing setItem(), within the same storage
mutex acquire/release set, *are* applied to backing store.
In the
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Thu, Sep 10, 2009 at 1:59 AM, Chris Jones cjo...@mozilla.com wrote: Jeremy Orlow wrote: What failures could there be in a pesimistic/queue model? I'm trying to think of a universal failure, something that could arise in any implementation of any proposal. I think that universal failure is exceeded localStorage space quota. (I believe there are others, but one is enough for this discussion.) So there are really two questions: first, what happens if you detect quota exceeded in the middle of a localStorage.transaction() callback? Are the modifications made *up to* the quota exceeded error applied to backing store? Or are they rolled back? This is the atomicity/consistency design decision I described: what is the unit of atomic modification (all applied or none applied). And second, can scripts be notified that the transaction failed (or was only partially applied) because of quota exceeded? AFAICT this is irrespective of sync/async/optimistic/pessimistic/queue. Any implementation has to deal with this error somehow. In the current localStorage spec, if a setItem() fails because space is exhausted, it raises QUOTA_EXCEEDED_ERR and does *not* apply any part of the change in setItem(). That is, setItem() is atomic --- it either applies or not (not partially applied). But modifications to localStorage *before* that failing setItem(), within the same storage mutex acquire/release set, *are* applied to backing store. In the Web Database spec, if executing a SQL statement in a Transaction would cause space quota to be exceeded, then *none* of the statements in the transaction are executed (i.e. none committed to backing store), and the Transaction fails with QUOTA_ERR. The unit of atomicity for Web Database is that Transaction object --- either all statements in Transaction successfully execute or none do. I'm not sure whether you're proposing that modifications within a localStorage.transaction() callback are atomic (all applied or none applied) in the face of quota exceeded. (I'm proposing that they *should* be atomic.) It sounds like you're proposing that scripts *cannot* be notified of quota exceeded. If this is true, then there are some semantic issues I'd like to discuss. But I want to make sure we're on the same page first :). (I'm proposing that scripts *should* be able to be notified of this.) The way it works right now is that an exception is thrown and that's it. I think it'd be perfectly acceptable for the behavior to remain, even if it's inside a transaction. In other words, I don't think the transaction necessarily needs to stop or anything. I think most app developers will want to explicitly handle out of space errors (they're not just going to go away on their own), but probably won't want to explicitly deal with transient errors (like what you can see with optimistic transactions). I don't think quota errors should be grouped in with any atomicity guarantees, in other words. Are there any other examples of failures you can have within a pessimistic transaction?
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Fri, Sep 4, 2009 at 12:02 AM, Chris Jonescjo...@mozilla.com wrote:
I propose adding the functions
window.localStorage.beginTransaction()
window.localStorage.commitTransaction()
or
window.beginTransaction()
window.commitTransaction()
I think this is a good idea! I would modify it to follow the pattern
set by the current SQLDatabase proposal, to have a callback, like
this:
window.localStorage.transaction(function() {
// use local storage here
});
I'm against having explicit begin/commit methods for the same reason
as I am for the SQLDatabase feature:
- It is easy to forget to commit
- The most likely paths in an application to be wrong are ones that
are rarely run
- Therefore many applications will contain uncommon paths that end up
hung (responsive, but still unable to make forward progress) and with
uncommitted data
But those issues are easily fixable with the callback model above.
On Fri, Sep 4, 2009 at 12:32 AM, Jeremy Orlowjor...@chromium.org wrote:
I actually think the idea of
throwing an exception whenever there's a serialization problem could be very
compelling, and could keep the door wide open for future performance
enhancements.
What problem would this solve? It seems like it would require
cross-app coordination to avoid exceptions.
On Fri, Sep 4, 2009 at 12:32 AM, Jeremy Orlowjor...@chromium.org wrote:
Note that if we do decide to break backwards compatibility, there are some
other things we should consider...but I won't bring those up unless we do
decide to move in this direction.
Btw, I want to make it clear that I take the idea of
breaking compatibility VERY seriously. I know LocalStorage is fairly well
adopted and that changing this would be pretty major. But having a
cross-event-loop, synchronous API is really a terrible idea. And changing
it now will be easier than changing it later.
I think it is possible to make this change with a very small breaking
change. Just add window.localStorage.transaction() and keep the rest
of the API as-is. Existing implementations can add the method, but
supporting using window.localStorage without transaction(), too.
- a
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Tue, Sep 8, 2009 at 12:00 AM, Aaron Boodmana...@google.com wrote:
On Fri, Sep 4, 2009 at 12:02 AM, Chris Jonescjo...@mozilla.com wrote:
I propose adding the functions
window.localStorage.beginTransaction()
window.localStorage.commitTransaction()
or
window.beginTransaction()
window.commitTransaction()
I think this is a good idea! I would modify it to follow the pattern
set by the current SQLDatabase proposal, to have a callback, like
this:
window.localStorage.transaction(function() {
// use local storage here
});
We have discussed similar APIs in the past. Something like a:
window.getLocalStorage(function (storage) {
...use storage...
});
This is nice because it can be expanded to something like:
window.getSharedItems(window.SHARED_ITEM_LOCALSTORAGE |
window.SHARED_ITEM_COOKIES, function (...) { ... });
to let you access both cookies and localStorage safely at the same time.
However, this requires breaking compatibility with existing syntax,
something that seems impossible at this point given that Microsoft has
shipped localStorage. I know Hixie has asked them in the past about
how they plan to deal with the mutex problem, but I'm not sure if an
answer has been received as of yet.
/ Jonas
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Tue, Sep 8, 2009 at 12:54 AM, Jonas Sickingjo...@sicking.cc wrote:
On Tue, Sep 8, 2009 at 12:00 AM, Aaron Boodmana...@google.com wrote:
On Fri, Sep 4, 2009 at 12:02 AM, Chris Jonescjo...@mozilla.com wrote:
I propose adding the functions
window.localStorage.beginTransaction()
window.localStorage.commitTransaction()
or
window.beginTransaction()
window.commitTransaction()
I think this is a good idea! I would modify it to follow the pattern
set by the current SQLDatabase proposal, to have a callback, like
this:
window.localStorage.transaction(function() {
// use local storage here
});
We have discussed similar APIs in the past. Something like a:
window.getLocalStorage(function (storage) {
...use storage...
});
This is nice because it can be expanded to something like:
window.getSharedItems(window.SHARED_ITEM_LOCALSTORAGE |
window.SHARED_ITEM_COOKIES, function (...) { ... });
to let you access both cookies and localStorage safely at the same time.
I think worrying about safely accessing cookies is a bit of
over-design. As has been pointed out, cookies don't work correctly
today and the wheels haven't fallen off yet.
I think a solution for localStorage that doesn't fix cookies is fine.
However, this requires breaking compatibility with existing syntax,
something that seems impossible at this point given that Microsoft has
shipped localStorage. I know Hixie has asked them in the past about
how they plan to deal with the mutex problem, but I'm not sure if an
answer has been received as of yet.
I addressed this at the end of my last message. Specifically, I suggest:
interface LocalStorageTransactionCallback {
void handleEvent(); // note: no arguments!
};
interface LocalStorage {
...
// LocalStorage can only be accessed inside this callback. Access outside
// of it will raise an exception, except in some browsers that support such
// behavior for legacy reasons.
void transaction(LocalStorageTransactionCallback callback);
...
};
With this, there is no need to change anything about the current API.
The only change is the addition of the new transaction() method.
- a
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Tue, Sep 8, 2009 at 4:54 PM, Jonas Sicking jo...@sicking.cc wrote:
On Tue, Sep 8, 2009 at 12:00 AM, Aaron Boodmana...@google.com wrote:
On Fri, Sep 4, 2009 at 12:02 AM, Chris Jonescjo...@mozilla.com wrote:
I propose adding the functions
window.localStorage.beginTransaction()
window.localStorage.commitTransaction()
or
window.beginTransaction()
window.commitTransaction()
I think this is a good idea! I would modify it to follow the pattern
set by the current SQLDatabase proposal, to have a callback, like
this:
window.localStorage.transaction(function() {
// use local storage here
});
We have discussed similar APIs in the past. Something like a:
window.getLocalStorage(function (storage) {
...use storage...
});
This is nice because it can be expanded to something like:
window.getSharedItems(window.SHARED_ITEM_LOCALSTORAGE |
window.SHARED_ITEM_COOKIES, function (...) { ... });
to let you access both cookies and localStorage safely at the same time.
However, this requires breaking compatibility with existing syntax,
something that seems impossible at this point given that Microsoft has
shipped localStorage. I know Hixie has asked them in the past about
how they plan to deal with the mutex problem, but I'm not sure if an
answer has been received as of yet.
First of all, I'm not sure I agree that we're at the point where
breaking compatibility is impossible. It really doesn't seem like it's
terribly widely used, and what's implemented is based on an early draft of
the spec. Yes, I agree that it's really unfortunate we didn't iron these
problems out better before everyone implemented it, but if LocalStorage
changed today, it definitely wouldn't break the web. (Of course, it's
possible that we would be breaking the web by the time the next gen of the
major browsers ship.it's hard to know for sure.)
But, if we're going to say that the way things are now is sacred, then how
do we reconcile the fact that an app that works fine on Safari (and others)
may break in IE due to races? If we're so worried about compatibility,
maybe we need to cater to the lowest common denominator (IE8so no run to
completion guarantees for LocalStorage) and add in a mechanism that does
give you guarantees (like a callback).
As a side note, I'd be very interested to hear what Microsoft's response
was.
J
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Tue, Sep 8, 2009 at 1:07 AM, Aaron Boodmana...@google.com wrote:
On Tue, Sep 8, 2009 at 12:54 AM, Jonas Sickingjo...@sicking.cc wrote:
On Tue, Sep 8, 2009 at 12:00 AM, Aaron Boodmana...@google.com wrote:
On Fri, Sep 4, 2009 at 12:02 AM, Chris Jonescjo...@mozilla.com wrote:
I propose adding the functions
window.localStorage.beginTransaction()
window.localStorage.commitTransaction()
or
window.beginTransaction()
window.commitTransaction()
I think this is a good idea! I would modify it to follow the pattern
set by the current SQLDatabase proposal, to have a callback, like
this:
window.localStorage.transaction(function() {
// use local storage here
});
We have discussed similar APIs in the past. Something like a:
window.getLocalStorage(function (storage) {
...use storage...
});
This is nice because it can be expanded to something like:
window.getSharedItems(window.SHARED_ITEM_LOCALSTORAGE |
window.SHARED_ITEM_COOKIES, function (...) { ... });
to let you access both cookies and localStorage safely at the same time.
I think worrying about safely accessing cookies is a bit of
over-design. As has been pointed out, cookies don't work correctly
today and the wheels haven't fallen off yet.
I think a solution for localStorage that doesn't fix cookies is fine.
However, this requires breaking compatibility with existing syntax,
something that seems impossible at this point given that Microsoft has
shipped localStorage. I know Hixie has asked them in the past about
how they plan to deal with the mutex problem, but I'm not sure if an
answer has been received as of yet.
I addressed this at the end of my last message. Specifically, I suggest:
interface LocalStorageTransactionCallback {
void handleEvent(); // note: no arguments!
};
interface LocalStorage {
...
// LocalStorage can only be accessed inside this callback. Access outside
// of it will raise an exception, except in some browsers that support such
// behavior for legacy reasons.
void transaction(LocalStorageTransactionCallback callback);
...
};
With this, there is no need to change anything about the current API.
The only change is the addition of the new transaction() method.
While this keeps existing IDL intact, it still breaks any existing
pages, which is the real concern for any browser vendor I would think.
/ Jonas
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Tue, Sep 8, 2009 at 1:13 AM, Jonas Sickingjo...@sicking.cc wrote:
On Tue, Sep 8, 2009 at 1:07 AM, Aaron Boodmana...@google.com wrote:
On Tue, Sep 8, 2009 at 12:54 AM, Jonas Sickingjo...@sicking.cc wrote:
On Tue, Sep 8, 2009 at 12:00 AM, Aaron Boodmana...@google.com wrote:
On Fri, Sep 4, 2009 at 12:02 AM, Chris Jonescjo...@mozilla.com wrote:
I propose adding the functions
window.localStorage.beginTransaction()
window.localStorage.commitTransaction()
or
window.beginTransaction()
window.commitTransaction()
I think this is a good idea! I would modify it to follow the pattern
set by the current SQLDatabase proposal, to have a callback, like
this:
window.localStorage.transaction(function() {
// use local storage here
});
We have discussed similar APIs in the past. Something like a:
window.getLocalStorage(function (storage) {
...use storage...
});
This is nice because it can be expanded to something like:
window.getSharedItems(window.SHARED_ITEM_LOCALSTORAGE |
window.SHARED_ITEM_COOKIES, function (...) { ... });
to let you access both cookies and localStorage safely at the same time.
I think worrying about safely accessing cookies is a bit of
over-design. As has been pointed out, cookies don't work correctly
today and the wheels haven't fallen off yet.
I think a solution for localStorage that doesn't fix cookies is fine.
However, this requires breaking compatibility with existing syntax,
something that seems impossible at this point given that Microsoft has
shipped localStorage. I know Hixie has asked them in the past about
how they plan to deal with the mutex problem, but I'm not sure if an
answer has been received as of yet.
I addressed this at the end of my last message. Specifically, I suggest:
interface LocalStorageTransactionCallback {
void handleEvent(); // note: no arguments!
};
interface LocalStorage {
...
// LocalStorage can only be accessed inside this callback. Access outside
// of it will raise an exception, except in some browsers that support such
// behavior for legacy reasons.
void transaction(LocalStorageTransactionCallback callback);
...
};
With this, there is no need to change anything about the current API.
The only change is the addition of the new transaction() method.
While this keeps existing IDL intact, it still breaks any existing
pages, which is the real concern for any browser vendor I would think.
Not necessarily.
The second half of my proposal is that vendors who currently implement
local storage can choose to continue to allow access to it outside of
the transaction() callback. It seems like this would work fine for
single-event-loop browsers, right?
- a
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Tue, Sep 8, 2009 at 1:18 AM, Aaron Boodmana...@google.com wrote:
On Tue, Sep 8, 2009 at 1:13 AM, Jonas Sickingjo...@sicking.cc wrote:
On Tue, Sep 8, 2009 at 1:07 AM, Aaron Boodmana...@google.com wrote:
On Tue, Sep 8, 2009 at 12:54 AM, Jonas Sickingjo...@sicking.cc wrote:
On Tue, Sep 8, 2009 at 12:00 AM, Aaron Boodmana...@google.com wrote:
On Fri, Sep 4, 2009 at 12:02 AM, Chris Jonescjo...@mozilla.com wrote:
I propose adding the functions
window.localStorage.beginTransaction()
window.localStorage.commitTransaction()
or
window.beginTransaction()
window.commitTransaction()
I think this is a good idea! I would modify it to follow the pattern
set by the current SQLDatabase proposal, to have a callback, like
this:
window.localStorage.transaction(function() {
// use local storage here
});
We have discussed similar APIs in the past. Something like a:
window.getLocalStorage(function (storage) {
...use storage...
});
This is nice because it can be expanded to something like:
window.getSharedItems(window.SHARED_ITEM_LOCALSTORAGE |
window.SHARED_ITEM_COOKIES, function (...) { ... });
to let you access both cookies and localStorage safely at the same time.
I think worrying about safely accessing cookies is a bit of
over-design. As has been pointed out, cookies don't work correctly
today and the wheels haven't fallen off yet.
I think a solution for localStorage that doesn't fix cookies is fine.
However, this requires breaking compatibility with existing syntax,
something that seems impossible at this point given that Microsoft has
shipped localStorage. I know Hixie has asked them in the past about
how they plan to deal with the mutex problem, but I'm not sure if an
answer has been received as of yet.
I addressed this at the end of my last message. Specifically, I suggest:
interface LocalStorageTransactionCallback {
void handleEvent(); // note: no arguments!
};
interface LocalStorage {
...
// LocalStorage can only be accessed inside this callback. Access outside
// of it will raise an exception, except in some browsers that support such
// behavior for legacy reasons.
void transaction(LocalStorageTransactionCallback callback);
...
};
With this, there is no need to change anything about the current API.
The only change is the addition of the new transaction() method.
While this keeps existing IDL intact, it still breaks any existing
pages, which is the real concern for any browser vendor I would think.
Not necessarily.
The second half of my proposal is that vendors who currently implement
local storage can choose to continue to allow access to it outside of
the transaction() callback. It seems like this would work fine for
single-event-loop browsers, right?
But that results in code that works in one browser, but not another,
defeating the whole point of having a standard.
Would you be fine with having pages that work fine in Firefox and IE,
break in Chrome?
/ Jonas
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Tue, Sep 8, 2009 at 5:20 PM, Jonas Sicking jo...@sicking.cc wrote:
On Tue, Sep 8, 2009 at 1:18 AM, Aaron Boodmana...@google.com wrote:
On Tue, Sep 8, 2009 at 1:13 AM, Jonas Sickingjo...@sicking.cc wrote:
On Tue, Sep 8, 2009 at 1:07 AM, Aaron Boodmana...@google.com wrote:
On Tue, Sep 8, 2009 at 12:54 AM, Jonas Sickingjo...@sicking.cc
wrote:
On Tue, Sep 8, 2009 at 12:00 AM, Aaron Boodmana...@google.com wrote:
On Fri, Sep 4, 2009 at 12:02 AM, Chris Jonescjo...@mozilla.com
wrote:
I propose adding the functions
window.localStorage.beginTransaction()
window.localStorage.commitTransaction()
or
window.beginTransaction()
window.commitTransaction()
I think this is a good idea! I would modify it to follow the pattern
set by the current SQLDatabase proposal, to have a callback, like
this:
window.localStorage.transaction(function() {
// use local storage here
});
We have discussed similar APIs in the past. Something like a:
window.getLocalStorage(function (storage) {
...use storage...
});
This is nice because it can be expanded to something like:
window.getSharedItems(window.SHARED_ITEM_LOCALSTORAGE |
window.SHARED_ITEM_COOKIES, function (...) { ... });
to let you access both cookies and localStorage safely at the same
time.
I think worrying about safely accessing cookies is a bit of
over-design. As has been pointed out, cookies don't work correctly
today and the wheels haven't fallen off yet.
I think a solution for localStorage that doesn't fix cookies is fine.
However, this requires breaking compatibility with existing syntax,
something that seems impossible at this point given that Microsoft has
shipped localStorage. I know Hixie has asked them in the past about
how they plan to deal with the mutex problem, but I'm not sure if an
answer has been received as of yet.
I addressed this at the end of my last message. Specifically, I
suggest:
interface LocalStorageTransactionCallback {
void handleEvent(); // note: no arguments!
};
interface LocalStorage {
...
// LocalStorage can only be accessed inside this callback. Access
outside
// of it will raise an exception, except in some browsers that support
such
// behavior for legacy reasons.
void transaction(LocalStorageTransactionCallback callback);
...
};
With this, there is no need to change anything about the current API.
The only change is the addition of the new transaction() method.
While this keeps existing IDL intact, it still breaks any existing
pages, which is the real concern for any browser vendor I would think.
Not necessarily.
The second half of my proposal is that vendors who currently implement
local storage can choose to continue to allow access to it outside of
the transaction() callback. It seems like this would work fine for
single-event-loop browsers, right?
But that results in code that works in one browser, but not another,
defeating the whole point of having a standard.
Would you be fine with having pages that work fine in Firefox and IE,
break in Chrome?
Well, pages that work fine in Firefox/Safari currently can break in IE
because IE is multi-process but does not implement a storage mutex...right?
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Tue, Sep 8, 2009 at 1:20 AM, Jonas Sickingjo...@sicking.cc wrote:
On Tue, Sep 8, 2009 at 1:18 AM, Aaron Boodmana...@google.com wrote:
On Tue, Sep 8, 2009 at 1:13 AM, Jonas Sickingjo...@sicking.cc wrote:
On Tue, Sep 8, 2009 at 1:07 AM, Aaron Boodmana...@google.com wrote:
On Tue, Sep 8, 2009 at 12:54 AM, Jonas Sickingjo...@sicking.cc wrote:
On Tue, Sep 8, 2009 at 12:00 AM, Aaron Boodmana...@google.com wrote:
On Fri, Sep 4, 2009 at 12:02 AM, Chris Jonescjo...@mozilla.com wrote:
I propose adding the functions
window.localStorage.beginTransaction()
window.localStorage.commitTransaction()
or
window.beginTransaction()
window.commitTransaction()
I think this is a good idea! I would modify it to follow the pattern
set by the current SQLDatabase proposal, to have a callback, like
this:
window.localStorage.transaction(function() {
// use local storage here
});
We have discussed similar APIs in the past. Something like a:
window.getLocalStorage(function (storage) {
...use storage...
});
This is nice because it can be expanded to something like:
window.getSharedItems(window.SHARED_ITEM_LOCALSTORAGE |
window.SHARED_ITEM_COOKIES, function (...) { ... });
to let you access both cookies and localStorage safely at the same time.
I think worrying about safely accessing cookies is a bit of
over-design. As has been pointed out, cookies don't work correctly
today and the wheels haven't fallen off yet.
I think a solution for localStorage that doesn't fix cookies is fine.
However, this requires breaking compatibility with existing syntax,
something that seems impossible at this point given that Microsoft has
shipped localStorage. I know Hixie has asked them in the past about
how they plan to deal with the mutex problem, but I'm not sure if an
answer has been received as of yet.
I addressed this at the end of my last message. Specifically, I suggest:
interface LocalStorageTransactionCallback {
void handleEvent(); // note: no arguments!
};
interface LocalStorage {
...
// LocalStorage can only be accessed inside this callback. Access outside
// of it will raise an exception, except in some browsers that support
such
// behavior for legacy reasons.
void transaction(LocalStorageTransactionCallback callback);
...
};
With this, there is no need to change anything about the current API.
The only change is the addition of the new transaction() method.
While this keeps existing IDL intact, it still breaks any existing
pages, which is the real concern for any browser vendor I would think.
Not necessarily.
The second half of my proposal is that vendors who currently implement
local storage can choose to continue to allow access to it outside of
the transaction() callback. It seems like this would work fine for
single-event-loop browsers, right?
But that results in code that works in one browser, but not another,
defeating the whole point of having a standard.
Would you be fine with having pages that work fine in Firefox and IE,
break in Chrome?
I don't really see another option. People on the Chrome team are
saying it may be impractical to implement the spec as-is. Presumably
Firefox is unwillingly to break backward compatibility. If both of
these are true, we are headed for a split.
In this case, I don't think it is a big deal. My proposed API change
is so minor that it is trivial to handle in code:
function doStorageStuff() {
... use window.localStorage ...
}
if (localStorage.transaction)
localStorage.transaction(doStorageStuff);
else
doStorageStuff();
- a
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Tue, Sep 8, 2009 at 1:33 AM, Aaron Boodmana...@google.com wrote:
On Tue, Sep 8, 2009 at 1:20 AM, Jonas Sickingjo...@sicking.cc wrote:
On Tue, Sep 8, 2009 at 1:18 AM, Aaron Boodmana...@google.com wrote:
On Tue, Sep 8, 2009 at 1:13 AM, Jonas Sickingjo...@sicking.cc wrote:
On Tue, Sep 8, 2009 at 1:07 AM, Aaron Boodmana...@google.com wrote:
On Tue, Sep 8, 2009 at 12:54 AM, Jonas Sickingjo...@sicking.cc wrote:
On Tue, Sep 8, 2009 at 12:00 AM, Aaron Boodmana...@google.com wrote:
On Fri, Sep 4, 2009 at 12:02 AM, Chris Jonescjo...@mozilla.com wrote:
I propose adding the functions
window.localStorage.beginTransaction()
window.localStorage.commitTransaction()
or
window.beginTransaction()
window.commitTransaction()
I think this is a good idea! I would modify it to follow the pattern
set by the current SQLDatabase proposal, to have a callback, like
this:
window.localStorage.transaction(function() {
// use local storage here
});
We have discussed similar APIs in the past. Something like a:
window.getLocalStorage(function (storage) {
...use storage...
});
This is nice because it can be expanded to something like:
window.getSharedItems(window.SHARED_ITEM_LOCALSTORAGE |
window.SHARED_ITEM_COOKIES, function (...) { ... });
to let you access both cookies and localStorage safely at the same time.
I think worrying about safely accessing cookies is a bit of
over-design. As has been pointed out, cookies don't work correctly
today and the wheels haven't fallen off yet.
I think a solution for localStorage that doesn't fix cookies is fine.
However, this requires breaking compatibility with existing syntax,
something that seems impossible at this point given that Microsoft has
shipped localStorage. I know Hixie has asked them in the past about
how they plan to deal with the mutex problem, but I'm not sure if an
answer has been received as of yet.
I addressed this at the end of my last message. Specifically, I suggest:
interface LocalStorageTransactionCallback {
void handleEvent(); // note: no arguments!
};
interface LocalStorage {
...
// LocalStorage can only be accessed inside this callback. Access outside
// of it will raise an exception, except in some browsers that support
such
// behavior for legacy reasons.
void transaction(LocalStorageTransactionCallback callback);
...
};
With this, there is no need to change anything about the current API.
The only change is the addition of the new transaction() method.
While this keeps existing IDL intact, it still breaks any existing
pages, which is the real concern for any browser vendor I would think.
Not necessarily.
The second half of my proposal is that vendors who currently implement
local storage can choose to continue to allow access to it outside of
the transaction() callback. It seems like this would work fine for
single-event-loop browsers, right?
But that results in code that works in one browser, but not another,
defeating the whole point of having a standard.
Would you be fine with having pages that work fine in Firefox and IE,
break in Chrome?
I don't really see another option. People on the Chrome team are
saying it may be impractical to implement the spec as-is. Presumably
Firefox is unwillingly to break backward compatibility. If both of
these are true, we are headed for a split.
In this case, I don't think it is a big deal. My proposed API change
is so minor that it is trivial to handle in code:
function doStorageStuff() {
... use window.localStorage ...
}
if (localStorage.transaction)
localStorage.transaction(doStorageStuff);
else
doStorageStuff();
I think Firefox would be willing to break compat. The question is if
microsoft is. Which we need to ask them.
/ Jonas
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Tue, Sep 8, 2009 at 1:25 AM, Jeremy Orlowjor...@chromium.org wrote:
On Tue, Sep 8, 2009 at 5:20 PM, Jonas Sicking jo...@sicking.cc wrote:
On Tue, Sep 8, 2009 at 1:18 AM, Aaron Boodmana...@google.com wrote:
On Tue, Sep 8, 2009 at 1:13 AM, Jonas Sickingjo...@sicking.cc wrote:
On Tue, Sep 8, 2009 at 1:07 AM, Aaron Boodmana...@google.com wrote:
On Tue, Sep 8, 2009 at 12:54 AM, Jonas Sickingjo...@sicking.cc
wrote:
On Tue, Sep 8, 2009 at 12:00 AM, Aaron Boodmana...@google.com wrote:
On Fri, Sep 4, 2009 at 12:02 AM, Chris Jonescjo...@mozilla.com
wrote:
I propose adding the functions
window.localStorage.beginTransaction()
window.localStorage.commitTransaction()
or
window.beginTransaction()
window.commitTransaction()
I think this is a good idea! I would modify it to follow the pattern
set by the current SQLDatabase proposal, to have a callback, like
this:
window.localStorage.transaction(function() {
// use local storage here
});
We have discussed similar APIs in the past. Something like a:
window.getLocalStorage(function (storage) {
...use storage...
});
This is nice because it can be expanded to something like:
window.getSharedItems(window.SHARED_ITEM_LOCALSTORAGE |
window.SHARED_ITEM_COOKIES, function (...) { ... });
to let you access both cookies and localStorage safely at the same
time.
I think worrying about safely accessing cookies is a bit of
over-design. As has been pointed out, cookies don't work correctly
today and the wheels haven't fallen off yet.
I think a solution for localStorage that doesn't fix cookies is fine.
However, this requires breaking compatibility with existing syntax,
something that seems impossible at this point given that Microsoft
has
shipped localStorage. I know Hixie has asked them in the past about
how they plan to deal with the mutex problem, but I'm not sure if an
answer has been received as of yet.
I addressed this at the end of my last message. Specifically, I
suggest:
interface LocalStorageTransactionCallback {
void handleEvent(); // note: no arguments!
};
interface LocalStorage {
...
// LocalStorage can only be accessed inside this callback. Access
outside
// of it will raise an exception, except in some browsers that
support such
// behavior for legacy reasons.
void transaction(LocalStorageTransactionCallback callback);
...
};
With this, there is no need to change anything about the current API.
The only change is the addition of the new transaction() method.
While this keeps existing IDL intact, it still breaks any existing
pages, which is the real concern for any browser vendor I would think.
Not necessarily.
The second half of my proposal is that vendors who currently implement
local storage can choose to continue to allow access to it outside of
the transaction() callback. It seems like this would work fine for
single-event-loop browsers, right?
But that results in code that works in one browser, but not another,
defeating the whole point of having a standard.
Would you be fine with having pages that work fine in Firefox and IE,
break in Chrome?
Well, pages that work fine in Firefox/Safari currently can break in IE
because IE is multi-process but does not implement a storage mutex...right?
Yes, with a very small likelyhood. This proposal would make that
happen every time.
Again, the question needs to be posed to microsoft. I can't speak for them.
/ Jonas
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Tue, Sep 8, 2009 at 5:33 PM, Aaron Boodman a...@google.com wrote:
On Tue, Sep 8, 2009 at 1:20 AM, Jonas Sickingjo...@sicking.cc wrote:
On Tue, Sep 8, 2009 at 1:18 AM, Aaron Boodmana...@google.com wrote:
On Tue, Sep 8, 2009 at 1:13 AM, Jonas Sickingjo...@sicking.cc wrote:
On Tue, Sep 8, 2009 at 1:07 AM, Aaron Boodmana...@google.com wrote:
On Tue, Sep 8, 2009 at 12:54 AM, Jonas Sickingjo...@sicking.cc
wrote:
On Tue, Sep 8, 2009 at 12:00 AM, Aaron Boodmana...@google.com wrote:
On Fri, Sep 4, 2009 at 12:02 AM, Chris Jonescjo...@mozilla.com
wrote:
I propose adding the functions
window.localStorage.beginTransaction()
window.localStorage.commitTransaction()
or
window.beginTransaction()
window.commitTransaction()
I think this is a good idea! I would modify it to follow the pattern
set by the current SQLDatabase proposal, to have a callback, like
this:
window.localStorage.transaction(function() {
// use local storage here
});
We have discussed similar APIs in the past. Something like a:
window.getLocalStorage(function (storage) {
...use storage...
});
This is nice because it can be expanded to something like:
window.getSharedItems(window.SHARED_ITEM_LOCALSTORAGE |
window.SHARED_ITEM_COOKIES, function (...) { ... });
to let you access both cookies and localStorage safely at the same
time.
I think worrying about safely accessing cookies is a bit of
over-design. As has been pointed out, cookies don't work correctly
today and the wheels haven't fallen off yet.
I think a solution for localStorage that doesn't fix cookies is fine.
However, this requires breaking compatibility with existing syntax,
something that seems impossible at this point given that Microsoft
has
shipped localStorage. I know Hixie has asked them in the past about
how they plan to deal with the mutex problem, but I'm not sure if an
answer has been received as of yet.
I addressed this at the end of my last message. Specifically, I
suggest:
interface LocalStorageTransactionCallback {
void handleEvent(); // note: no arguments!
};
interface LocalStorage {
...
// LocalStorage can only be accessed inside this callback. Access
outside
// of it will raise an exception, except in some browsers that
support such
// behavior for legacy reasons.
void transaction(LocalStorageTransactionCallback callback);
...
};
With this, there is no need to change anything about the current API.
The only change is the addition of the new transaction() method.
While this keeps existing IDL intact, it still breaks any existing
pages, which is the real concern for any browser vendor I would think.
Not necessarily.
The second half of my proposal is that vendors who currently implement
local storage can choose to continue to allow access to it outside of
the transaction() callback. It seems like this would work fine for
single-event-loop browsers, right?
But that results in code that works in one browser, but not another,
defeating the whole point of having a standard.
Would you be fine with having pages that work fine in Firefox and IE,
break in Chrome?
I don't really see another option. People on the Chrome team are
saying it may be impractical to implement the spec as-is. Presumably
Firefox is unwillingly to break backward compatibility. If both of
these are true, we are headed for a split.
To be clear, Chrome is not going to implement the storage mutex with respect
to cookies, but we are going to implement it for LocalStorage. Because of
this, we can handle the localStorage mutex on a per-origin basis (which
I'm implementing right now). That said, we're not happy about having a
synchronous interface that connects otherwise completely independent
windows.
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Tue, Sep 8, 2009 at 7:00 PM, Aaron Boodman a...@google.com wrote:
On Fri, Sep 4, 2009 at 12:02 AM, Chris Jonescjo...@mozilla.com wrote:
I propose adding the functions
window.localStorage.beginTransaction()
window.localStorage.commitTransaction()
or
window.beginTransaction()
window.commitTransaction()
I think this is a good idea! I would modify it to follow the pattern
set by the current SQLDatabase proposal, to have a callback, like
this:
window.localStorage.transaction(function() {
// use local storage here
});
What is the intended semantics here? Chris' explicit commitTransaction would
throw an exception if the transaction was aborted due to data inconsistency,
leaving it up to the script to retry --- and making it clear to script
authors that non-storage side effects during the transaction are not undone.
How would you handle transaction aborts?
Rob
--
He was pierced for our transgressions, he was crushed for our iniquities;
the punishment that brought us peace was upon him, and by his wounds we are
healed. We all, like sheep, have gone astray, each of us has turned to his
own way; and the LORD has laid on him the iniquity of us all. [Isaiah
53:5-6]
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Tue, Sep 8, 2009 at 5:35 PM, Jonas Sicking jo...@sicking.cc wrote:
On Tue, Sep 8, 2009 at 1:25 AM, Jeremy Orlowjor...@chromium.org wrote:
On Tue, Sep 8, 2009 at 5:20 PM, Jonas Sicking jo...@sicking.cc wrote:
On Tue, Sep 8, 2009 at 1:18 AM, Aaron Boodmana...@google.com wrote:
On Tue, Sep 8, 2009 at 1:13 AM, Jonas Sickingjo...@sicking.cc
wrote:
On Tue, Sep 8, 2009 at 1:07 AM, Aaron Boodmana...@google.com wrote:
On Tue, Sep 8, 2009 at 12:54 AM, Jonas Sickingjo...@sicking.cc
wrote:
On Tue, Sep 8, 2009 at 12:00 AM, Aaron Boodmana...@google.com
wrote:
On Fri, Sep 4, 2009 at 12:02 AM, Chris Jonescjo...@mozilla.com
wrote:
I propose adding the functions
window.localStorage.beginTransaction()
window.localStorage.commitTransaction()
or
window.beginTransaction()
window.commitTransaction()
I think this is a good idea! I would modify it to follow the
pattern
set by the current SQLDatabase proposal, to have a callback, like
this:
window.localStorage.transaction(function() {
// use local storage here
});
We have discussed similar APIs in the past. Something like a:
window.getLocalStorage(function (storage) {
...use storage...
});
This is nice because it can be expanded to something like:
window.getSharedItems(window.SHARED_ITEM_LOCALSTORAGE |
window.SHARED_ITEM_COOKIES, function (...) { ... });
to let you access both cookies and localStorage safely at the same
time.
I think worrying about safely accessing cookies is a bit of
over-design. As has been pointed out, cookies don't work correctly
today and the wheels haven't fallen off yet.
I think a solution for localStorage that doesn't fix cookies is
fine.
However, this requires breaking compatibility with existing syntax,
something that seems impossible at this point given that Microsoft
has
shipped localStorage. I know Hixie has asked them in the past about
how they plan to deal with the mutex problem, but I'm not sure if
an
answer has been received as of yet.
I addressed this at the end of my last message. Specifically, I
suggest:
interface LocalStorageTransactionCallback {
void handleEvent(); // note: no arguments!
};
interface LocalStorage {
...
// LocalStorage can only be accessed inside this callback. Access
outside
// of it will raise an exception, except in some browsers that
support such
// behavior for legacy reasons.
void transaction(LocalStorageTransactionCallback callback);
...
};
With this, there is no need to change anything about the current
API.
The only change is the addition of the new transaction() method.
While this keeps existing IDL intact, it still breaks any existing
pages, which is the real concern for any browser vendor I would
think.
Not necessarily.
The second half of my proposal is that vendors who currently implement
local storage can choose to continue to allow access to it outside of
the transaction() callback. It seems like this would work fine for
single-event-loop browsers, right?
But that results in code that works in one browser, but not another,
defeating the whole point of having a standard.
Would you be fine with having pages that work fine in Firefox and IE,
break in Chrome?
Well, pages that work fine in Firefox/Safari currently can break in IE
because IE is multi-process but does not implement a storage
mutex...right?
Yes, with a very small likelyhood. This proposal would make that
happen every time.
Again, the question needs to be posed to microsoft. I can't speak for them.
Understood.
If Microsoft is not willing to break compat and they're not going to
implement the storage mutex for LocalStorage, then what do we do? Change
the spec so that there's no run to completion guarantees for LocalStorage or
leave them in the dust?
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Tue, Sep 8, 2009 at 5:54 PM, Aaron Boodman a...@google.com wrote:
You'd have to implement it via a mutex.
I think you would have a queue of all pending requests for access to
storage, along with their associated callbacks, message loops, and
execution environments. Each time the local storage frees up, if there
is any queued request, invoke the callback in the correct environment.
I guess I should have said mutex (in quotes) because something like this
is what I intended.
Not that it matters to the spec, but this is more or less how I'm
implementing locking within Chromium. The only difference is that the queue
is of renderer processes (blocked on getting the lock) rather than
individual callbacks. Doing the latter would be nice since sometimes web
pages have to share the same process (and thus one origin may block another
from executing because of Chromium implementational details).
An optimized implementation could
wait until the first operation that can't be un-done before acquiring it,
and do everything optimistically until then. This is the same situation
as
WebDatabase if I understand it correctly.
You can't do this because upgrading the lock could block the UI
thread. Same thing with the database spec.
Sure you could block the UI thread more, but there's nothing to stop you
from doing a 'while (true) { }' inside the callback, which will block the UI
thread as well. So I disagree with your assertion that waiting for a lock
upgrade is a deal breaker. I don't think any implementation is
realistically going to do anything like this in the near term, though.
If you want to get fancy, the transaction() method can take a
read/write param. This is what the database spec does, but I think it
is overkill here, personally.
Agreed.
On Tue, Sep 8, 2009 at 6:02 PM, Robert O'Callahan rob...@ocallahan.org
wrote:
On Tue, Sep 8, 2009 at 8:38 PM, Jeremy Orlow jor...@chromium.org wrote:
To be clear, Chrome is not going to implement the storage mutex with
respect to cookies, but we are going to implement it for LocalStorage.
Because of this, we can handle the localStorage mutex on a per-origin basis
(which I'm implementing right now).
Looking back over previous threads on the storage mutex, I can't seem to
remember or find the reason that implementing the storage mutex for cookies
can't easily be done with a mutex per domain. Ian pointed out this approach
breaks if you can make synchronous script calls across origins (e.g. across
IFRAME boundaries), but can you actually make such calls? Or if you can
(NPAPI?), can we just declare that those APIs release the storage mutex?
I thought it over, but I can't come up with any deadlock scenario. I would
still be a bit worried about performance issues (including many ways a
developer could shoot themself in the foot), but it is an interesting idea.
I know that setting document.domain makes this tricky because it
synchronously enables new cross-domain interactions, but can't we handle
that by declaring that setting document.domain releases the storage mutex?
I think this would be very reasonable. Especially with all the other things
that already implicitly drop the lock.
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Tue, Sep 8, 2009 at 2:02 AM, Robert O'Callahanrob...@ocallahan.org wrote: Looking back over previous threads on the storage mutex, I can't seem to remember or find the reason that implementing the storage mutex for cookies can't easily be done with a mutex per domain. Ian pointed out this approach breaks if you can make synchronous script calls across origins (e.g. across IFRAME boundaries), but can you actually make such calls? Or if you can (NPAPI?), can we just declare that those APIs release the storage mutex? I believe that synchronous cross-origin calls are possible a variety of ways. Here is one way I found with a quick test: Resize an iframe element. window.onresize is fired synchronously inside the frame. I bet there are others. I know that setting document.domain makes this tricky because it synchronously enables new cross-domain interactions, but can't we handle that by declaring that setting document.domain releases the storage mutex? All of these different ways that the storage mutex gets implicitly released lead to weird behavior in edge cases. In my opinion, it would be better to fix the API in a clean way than keep patching it like this. - a
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Tue, Sep 8, 2009 at 6:38 PM, Aaron Boodman a...@google.com wrote: On Tue, Sep 8, 2009 at 2:02 AM, Robert O'Callahanrob...@ocallahan.org wrote: Looking back over previous threads on the storage mutex, I can't seem to remember or find the reason that implementing the storage mutex for cookies can't easily be done with a mutex per domain. Ian pointed out this approach breaks if you can make synchronous script calls across origins (e.g. across IFRAME boundaries), but can you actually make such calls? Or if you can (NPAPI?), can we just declare that those APIs release the storage mutex? I believe that synchronous cross-origin calls are possible a variety of ways. Here is one way I found with a quick test: Resize an iframe element. window.onresize is fired synchronously inside the frame. I bet there are others. I know that setting document.domain makes this tricky because it synchronously enables new cross-domain interactions, but can't we handle that by declaring that setting document.domain releases the storage mutex? All of these different ways that the storage mutex gets implicitly released lead to weird behavior in edge cases. In my opinion, it would be better to fix the API in a clean way than keep patching it like this. I definitely agree. Implicitly releasing the lock is terrible. (We should at the _very least_ create some way for people to know when it's been implicitly released!)
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Tue, Sep 8, 2009 at 9:38 PM, Aaron Boodman a...@google.com wrote: On Tue, Sep 8, 2009 at 2:02 AM, Robert O'Callahanrob...@ocallahan.org wrote: Looking back over previous threads on the storage mutex, I can't seem to remember or find the reason that implementing the storage mutex for cookies can't easily be done with a mutex per domain. Ian pointed out this approach breaks if you can make synchronous script calls across origins (e.g. across IFRAME boundaries), but can you actually make such calls? Or if you can (NPAPI?), can we just declare that those APIs release the storage mutex? I believe that synchronous cross-origin calls are possible a variety of ways. Here is one way I found with a quick test: Resize an iframe element. window.onresize is fired synchronously inside the frame. I bet there are others. What spec says that onresize should fire synchronously? I'm actually pretty sure no-one should be saying that, since layout is mostly asynchronous in Webkit and Gecko and I don't want any spec to impose mostly-unobservable requirements about when layout should happen. I know that setting document.domain makes this tricky because it synchronously enables new cross-domain interactions, but can't we handle that by declaring that setting document.domain releases the storage mutex? All of these different ways that the storage mutex gets implicitly released lead to weird behavior in edge cases. In my opinion, it would be better to fix the API in a clean way than keep patching it like this. document.domain is a crazy API. So are sync XHRs which I believe also drop the mutex. I'm comfortable with these cases dropping the mutex ... much more comfortable than not having the mutex at all. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
In general, I agree with Rob about this proposal. What problem with storage mutex as spec'd today does your proposal solve? I'm eagerly willing to concede explicit beginTransaction()/commitTransaction() for a window.transaction() wrapper that implicitly does those. Aaron Boodman wrote: On Tue, Sep 8, 2009 at 1:41 AM, Robert O'Callahanrob...@ocallahan.org wrote: What is the intended semantics here? Chris' explicit commitTransaction would throw an exception if the transaction was aborted due to data inconsistency, leaving it up to the script to retry --- and making it clear to script authors that non-storage side effects during the transaction are not undone. How would you handle transaction aborts? Calls to transaction() are queued and executed serially per-origin with exclusive access. There is no such thing as a transaction abort because there cannot be consistency problems because of the serialized access. No, transactions can still fail. They can fail in ways immediately hidden from the script that requested them if the UA has to interrupt the conceptually executing transaction in the ways enumerated in a separate branch of this thread. Later script executions can observe inconsistent state unless more is specified by your proposal. Transactions can also fail visibly if write-to-disk fails (probably also in other ways I haven't considered). It's not clear what should happen wrt to your proposal in this case. (We could pretend that these failures never occur, but IMHO my original proposal handles both naturally.) You'd have to implement it via a mutex. I think you would have a queue of all pending requests for access to storage, along with their associated callbacks, message loops, and execution environments. Each time the local storage frees up, if there is any queued request, invoke the callback in the correct environment. An optimized implementation could wait until the first operation that can't be un-done before acquiring it, and do everything optimistically until then. This is the same situation as WebDatabase if I understand it correctly. You can't do this because upgrading the lock could block the UI thread. Same thing with the database spec. If you want to get fancy, the transaction() method can take a read/write param. This is what the database spec does, but I think it is overkill here, personally. I have to admit that I didn't closely read the Web Database spec before making the OP. Like Rob, my scan of the text led me to believe that it provided an API to asynchronously execute SQL free from script side effects. It does *not*. Arbitrary side-effecty script statements can execute within Web Database transactions. And these transactions can be synchronous. And like my OP concerning localStorage, Web Database transactions are allowed to fail by the spec. In fact, I believe that the Synchronous database API describes the same transaction semantics as I proposed in the OP. That spec adds implicit begin/commitTransaction and read-only transactions, but otherwise the semantics are the same. So I'd like to amend my original proposal to be Use Synchronous Web Database API transaction semantics. Except do not offer readTransaction: a transaction is implicitly a read-only transaction if only getItem() is called on localStorage from within localStorage.transaction(). SELUAs (single-event-loop UAs) can implement this semantics without mutexes. MELUAs (multi-event-loop UAs) can implement this semantics with mutexes, but more interesting and performant non-blocking implementations also seem possible (which is what I had in mind with the OP.) Cheers, Chris
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Tue, Sep 8, 2009 at 11:23 AM, Chris Jonescjo...@mozilla.com wrote: In general, I agree with Rob about this proposal. What problem with storage mutex as spec'd today does your proposal solve? The spec requires a single storage mutex for the entire UA. Therefore in a MELUA a web page can become unresponsive while waiting for some other page to give up the lock. This is not good and something we have tried to avoid everywhere else in the spec. Attempts to address this by doing per-origin locks wind up with deadlocks being possible. I'm eagerly willing to concede explicit beginTransaction()/commitTransaction() for a window.transaction() wrapper that implicitly does those. Cool, that is the only point I was trying to make. Aaron Boodman wrote: On Tue, Sep 8, 2009 at 1:41 AM, Robert O'Callahanrob...@ocallahan.org wrote: What is the intended semantics here? Chris' explicit commitTransaction would throw an exception if the transaction was aborted due to data inconsistency, leaving it up to the script to retry --- and making it clear to script authors that non-storage side effects during the transaction are not undone. How would you handle transaction aborts? Calls to transaction() are queued and executed serially per-origin with exclusive access. There is no such thing as a transaction abort because there cannot be consistency problems because of the serialized access. No, transactions can still fail. They can fail in ways immediately hidden from the script that requested them if the UA has to interrupt the conceptually executing transaction in the ways enumerated in a separate branch of this thread. Later script executions can observe inconsistent state unless more is specified by your proposal. Transactions can also fail visibly if write-to-disk fails (probably also in other ways I haven't considered). It's not clear what should happen wrt to your proposal in this case. It sounds like you're talking about these issues that you listed a bit earlier: * clear private data * close tab * quit UA * slow script timeout * store-to-disk failure * crash Right? If so, I agree with roc's responses to them that they could probably be handled without surfacing errors to the developer. OTOH, I'm not really against adding the concept of fallibility here. That wasn't really my point. Mainly, I just think that: a) We can't have synchronous access to the storage API b) We should not have an API that requires an author to explicitly begin and commit transactions I have to admit that I didn't closely read the Web Database spec before making the OP. Like Rob, my scan of the text led me to believe that it provided an API to asynchronously execute SQL free from script side effects. It does *not*. Arbitrary side-effecty script statements can execute within Web Database transactions. Yeah, I meant to correct Rob on that. And these transactions can be synchronous. And like my OP concerning localStorage, Web Database transactions are allowed to fail by the spec. Right. In fact, I believe that the Synchronous database API describes the same transaction semantics as I proposed in the OP. That spec adds implicit begin/commitTransaction and read-only transactions, but otherwise the semantics are the same. So I'd like to amend my original proposal to be Use Synchronous Web Database API transaction semantics. Except do not offer readTransaction: a transaction is implicitly a read-only transaction if only getItem() is called on localStorage from within localStorage.transaction(). Agree. That is what I was trying to propose, too. I'm not sure where we disagree :). Is it just that my proposal has no concept of errors? I'm not against adding them, mainly I was trying to keep my proposal simple for purposes of discussion. Note: you're mentioning synchronous transactions here. I think that the callback for localStorage.transaction() *must* be run asynchronously to avoid blocking on access to the storage. I presume this is what you meant. Right? - a
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Wed, Sep 9, 2009 at 6:53 AM, Aaron Boodman a...@google.com wrote: Attempts to address this by doing per-origin locks wind up with deadlocks being possible. I think we can fix this. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Tue, Sep 8, 2009 at 11:53 AM, Aaron Boodmana...@google.com wrote: On Tue, Sep 8, 2009 at 11:23 AM, Chris Jonescjo...@mozilla.com wrote: In general, I agree with Rob about this proposal. What problem with storage mutex as spec'd today does your proposal solve? The spec requires a single storage mutex for the entire UA. Therefore in a MELUA a web page can become unresponsive while waiting for some other page to give up the lock. This is not good and something we have tried to avoid everywhere else in the spec. Attempts to address this by doing per-origin locks wind up with deadlocks being possible. How could this happen. It certainly does sound scary to have a bunch of locks that can be grabbed in arbitrary order, but I can't off the top of my head think of any ways where deadlocks can happen. / Jonas
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
Benjamin Smedberg wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 9/8/09 3:00 AM, Aaron Boodman wrote:
On Fri, Sep 4, 2009 at 12:02 AM, Chris Jonescjo...@mozilla.com wrote:
I propose adding the functions
window.localStorage.beginTransaction()
window.localStorage.commitTransaction()
or
window.beginTransaction()
window.commitTransaction()
I think this is a good idea! I would modify it to follow the pattern
set by the current SQLDatabase proposal, to have a callback, like
this:
window.localStorage.transaction(function() {
// use local storage here
});
To be specific, the .transaction function would enqueue an operation to
perform at a later time when a mutex was held. The current caller would
continue to run to completion. There would never be simultaneous
transactions which could potentially conflict with eachother and require
merging or rollback.
It wasn't clear to me when this was proposed that it was asynchronous,
instead of a blocking call that *immediately* waited for the mutex and
blocked script execution.
To be clear, that's not what I had in mind at all. I envisioned a
synchronous API that speaks in terms of fallible, atomic/consistent
transactions. Mutexes are implementation details that the spec would
not mention.
It's easy to make an async API out of a sync one by using setTimeout(0,
...). It's harder to go the other way.
Would the transaction function be defined so that it never runs immediately
but is always enqueued? Also, I think the function name should make it
clearer that it's an asynchronous callback:
window.localStorage.queueTransaction or somesuch?
I'm against having explicit begin/commit methods for the same reason
as I am for the SQLDatabase feature:
- It is easy to forget to commit
- The most likely paths in an application to be wrong are ones that
are rarely run
- Therefore many applications will contain uncommon paths that end up
hung (responsive, but still unable to make forward progress) and with
uncommitted data
I agree that this is true if you never implicitly commit the transaction.
But if you were to implicitly commit the transaction when a script runs to
completion, would that negate the most serious of these issues? I'm defining
completion as all those places where the current spec says the storage
mutex is unlocked, which seems equivalent to the places script can block
on network or UI activity.
It sounds like you're proposing to make the unit of run-to-completion
implicitly delineate localStorage transactions? This is fine but it
complicates the handling of failed transactions, IMHO.
I suspect that making incompatible changes to the existing storage API is
going to be a hard sell for some authors: could we continue to support
completely transaction-free access to storage, in addition to the race-free
queued version. This would allow authors (JS libraries) to do
runtime-detection of the form:
if (window.localStorage.transaction === undefined)
window.localStorage.transaction = function(fn) {
window.setTimeout(fn, 0);
};
No one has responded directly to my original proposal of making
|window.localStorage === undefined| until |window.transaction| or
whatever has been accessed. Unlike your proposal and a similar one from
Jeremy, mine is a safe (non-racy) way for spec-compliant UAs to bend
backwards compatibility without explicitly breaking it. Web apps slow
to change should theoretically be prepared for |window.localStorage ===
undefined| anyway, as not all UAs support localStorage. And if a UA
doesn't support window.transaction, a web app that cares never needs to
worry about racy localStorage because in non-compliant UAs,
|window.transaction === undefined|.
Cheers,
Chris
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
Aaron Boodman wrote: On Tue, Sep 8, 2009 at 11:23 AM, Chris Jonescjo...@mozilla.com wrote: In general, I agree with Rob about this proposal. What problem with storage mutex as spec'd today does your proposal solve? The spec requires a single storage mutex for the entire UA. Therefore in a MELUA a web page can become unresponsive while waiting for some other page to give up the lock. This is not good and something we have tried to avoid everywhere else in the spec. Attempts to address this by doing per-origin locks wind up with deadlocks being possible. Aaron Boodman wrote: On Tue, Sep 8, 2009 at 1:41 AM, Robert O'Callahanrob...@ocallahan.org wrote: What is the intended semantics here? Chris' explicit commitTransaction would throw an exception if the transaction was aborted due to data inconsistency, leaving it up to the script to retry --- and making it clear to script authors that non-storage side effects during the transaction are not undone. How would you handle transaction aborts? Calls to transaction() are queued and executed serially per-origin with exclusive access. There is no such thing as a transaction abort because there cannot be consistency problems because of the serialized access. No, transactions can still fail. They can fail in ways immediately hidden from the script that requested them if the UA has to interrupt the conceptually executing transaction in the ways enumerated in a separate branch of this thread. Later script executions can observe inconsistent state unless more is specified by your proposal. Transactions can also fail visibly if write-to-disk fails (probably also in other ways I haven't considered). It's not clear what should happen wrt to your proposal in this case. If so, I agree with roc's responses to them that they could probably be handled without surfacing errors to the developer. OTOH, I'm not really against adding the concept of fallibility here. In fact, I believe that the Synchronous database API describes the same transaction semantics as I proposed in the OP. That spec adds implicit begin/commitTransaction and read-only transactions, but otherwise the semantics are the same. So I'd like to amend my original proposal to be Use Synchronous Web Database API transaction semantics. Except do not offer readTransaction: a transaction is implicitly a read-only transaction if only getItem() is called on localStorage from within localStorage.transaction(). Agree. That is what I was trying to propose, too. I'm not sure where we disagree :). Is it just that my proposal has no concept of errors? I'm not against adding them, mainly I was trying to keep my proposal simple for purposes of discussion. Ay, there's the rub: I think the disagreement is between mutex vs. transaction semantics. So far, I think perhaps mutex has been used as shorthand for transaction. But they aren't the same. I think we all agree that a script may fail to modify localStorage in some situations (irrespective of global mutex vs. per-domain mutex). One camp, wanting mutex semantics, would prefer to pretend that the failures never happen and let scripts clean up the mess (partially-applied changes) if they do occur. This is semantically broken, IMHO. The second camp, wanting transaction semantics, explicitly acknowledge to web authors that localStorage is fallible, guarantee that modifications to localStorage are atomic, and notify scripts when modifications can't be made atomically. This is the same approach taken by Web Database. IMHO, this is much better semantically because (i) it gives web apps stronger guarantees; and (ii) it makes the discussion about global mutex/per-domain mutex/non-blocking an implementation issue rather semantic issue, as it should be. Can those in the first camp explain why mutex semantics is better than transaction semantics? And why it's desirable to have one DB spec specify transaction semantics (Web Database) and a second specify mutex semantics (localStorage)? Note: you're mentioning synchronous transactions here. I think that the callback for localStorage.transaction() *must* be run asynchronously to avoid blocking on access to the storage. I presume this is what you meant. Right? I think this is an orthogonal issue concerning per-tab script responsiveness vs. author friendliness. One could argue for async-only, sync-only, or both. Cheers, Chris
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Tue, Sep 8, 2009 at 1:38 PM, Chris Jonescjo...@mozilla.com wrote: No one has responded directly to my original proposal of making |window.localStorage === undefined| until |window.transaction| or whatever has been accessed. Unlike your proposal and a similar one from Jeremy, mine is a safe (non-racy) way for spec-compliant UAs to bend backwards compatibility without explicitly breaking it. How is that not breaking compatibility? Any existing pages will directly access window.localStorage to read/write data. Such pages would break with your proposal, no? / Jonas
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 9/8/09 3:00 AM, Aaron Boodman wrote:
On Fri, Sep 4, 2009 at 12:02 AM, Chris Jonescjo...@mozilla.com wrote:
I propose adding the functions
window.localStorage.beginTransaction()
window.localStorage.commitTransaction()
or
window.beginTransaction()
window.commitTransaction()
I think this is a good idea! I would modify it to follow the pattern
set by the current SQLDatabase proposal, to have a callback, like
this:
window.localStorage.transaction(function() {
// use local storage here
});
To be specific, the .transaction function would enqueue an operation to
perform at a later time when a mutex was held. The current caller would
continue to run to completion. There would never be simultaneous
transactions which could potentially conflict with eachother and require
merging or rollback.
It wasn't clear to me when this was proposed that it was asynchronous,
instead of a blocking call that *immediately* waited for the mutex and
blocked script execution.
Would the transaction function be defined so that it never runs immediately
but is always enqueued? Also, I think the function name should make it
clearer that it's an asynchronous callback:
window.localStorage.queueTransaction or somesuch?
I'm against having explicit begin/commit methods for the same reason
as I am for the SQLDatabase feature:
- It is easy to forget to commit
- The most likely paths in an application to be wrong are ones that
are rarely run
- Therefore many applications will contain uncommon paths that end up
hung (responsive, but still unable to make forward progress) and with
uncommitted data
I agree that this is true if you never implicitly commit the transaction.
But if you were to implicitly commit the transaction when a script runs to
completion, would that negate the most serious of these issues? I'm defining
completion as all those places where the current spec says the storage
mutex is unlocked, which seems equivalent to the places script can block
on network or UI activity.
I suspect that making incompatible changes to the existing storage API is
going to be a hard sell for some authors: could we continue to support
completely transaction-free access to storage, in addition to the race-free
queued version. This would allow authors (JS libraries) to do
runtime-detection of the form:
if (window.localStorage.transaction === undefined)
window.localStorage.transaction = function(fn) {
window.setTimeout(fn, 0);
};
- --BDS
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iD8DBQFKprvbSSwGp5sTYNkRAg33AJwO+WnnwnUu2VB3/UyWQC/w/siYTQCfY4A7
29mcqQmITk2K6PYdodMAMzM=
=ftOi
-END PGP SIGNATURE-
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Tue, Sep 8, 2009 at 12:02 PM, Jonas Sickingjo...@sicking.cc wrote: On Tue, Sep 8, 2009 at 11:53 AM, Aaron Boodmana...@google.com wrote: On Tue, Sep 8, 2009 at 11:23 AM, Chris Jonescjo...@mozilla.com wrote: In general, I agree with Rob about this proposal. What problem with storage mutex as spec'd today does your proposal solve? The spec requires a single storage mutex for the entire UA. Therefore in a MELUA a web page can become unresponsive while waiting for some other page to give up the lock. This is not good and something we have tried to avoid everywhere else in the spec. Attempts to address this by doing per-origin locks wind up with deadlocks being possible. How could this happen. It certainly does sound scary to have a bunch of locks that can be grabbed in arbitrary order, but I can't off the top of my head think of any ways where deadlocks can happen. From a Hixie mail way back in March: http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-March/018991.html snip If it can be shown that it is not ever possible for script in one origin to synchronously invoke script in another origin, then I guess we could have per-origin locks instead of a single lock. /snip Here are two cases I know of where it is possible to have synchronous script execution across origins: * Plugins. It is possible for script to invoke a plugin function in one frame, and for the plugin to synchronously execute script in another frame. We have addressed this in the spec by saying that invoking a plugin releases the storage mutex, but that doesn't really solve the problem. We are exchanging violation of run-to-completion for deadlock. I guess it is an improvement, but it is still a bug. * In WebKit, onresize is invoked synchronously. You can cause cross-origin synchronous script execution by resizing an iframe. AFAIK, the spec does not disallow this event from being synchronous. I found the second case after only a few minutes of thinking about it. I suspect there are more since it was never a design goal to prevent synchronous script execution across origins. We could patch each case like this by releasing the mutex, but each time we do that, we poke another hole in run to completion. I also suspect such patches will be difficult to maintain. Even if we update implementations to invoke onresize asynchronously, how confident can we be in interactions between such unrelated components as event dispatch and storage continuing to work correctly? Finally, even with these hacks, we have problems. In a MELUA (I love that acronym!), with per-origin local storage mutexes, we still can end up with unresponsive UI because it is possible to have the same origin running on different event loops. Adding an asynchronous step to acquire the mutex neatly solves all these problems. We get run-to-completion, and we never block an event loop on waiting for a mutex. On Tue, Sep 8, 2009 at 12:39 PM, Chris Jonescjo...@mozilla.com wrote: I think we all agree that a script may fail to modify localStorage in some situations (irrespective of global mutex vs. per-domain mutex). One camp, wanting mutex semantics, would prefer to pretend that the failures never happen and let scripts clean up the mess (partially-applied changes) if they do occur. This is semantically broken, IMHO. The second camp, wanting transaction semantics, explicitly acknowledge to web authors that localStorage is fallible, guarantee that modifications to localStorage are atomic, and notify scripts when modifications can't be made atomically. This is the same approach taken by Web Database. IMHO, this is much better semantically because (i) it gives web apps stronger guarantees; and (ii) it makes the discussion about global mutex/per-domain mutex/non-blocking an implementation issue rather semantic issue, as it should be. Can those in the first camp explain why mutex semantics is better than transaction semantics? And why it's desirable to have one DB spec specify transaction semantics (Web Database) and a second specify mutex semantics (localStorage)? Ok, I don't have a strong preference on this. Transaction semantics seem better to me, but only marginally so. - a
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Sep 8, 2009, at 1:08 AM, Jeremy Orlow wrote: First of all, I'm not sure I agree that we're at the point where breaking compatibility is impossible. It really doesn't seem like it's terribly widely used, and what's implemented is based on an early draft of the spec. Yes, I agree that it's really unfortunate we didn't iron these problems out better before everyone implemented it, but if LocalStorage changed today, it definitely wouldn't break the web. (Of course, it's possible that we would be breaking the web by the time the next gen of the major browsers ship.it's hard to know for sure.) I know LocalStorage is used in a lot of mobile-specific Web content that targets WebKit-based browsers. I also expect there are mainstream sites using LocalStorage at least as an alternative when available, given that it's in the latest version of nearly every brrowser. If you think changing LocalStorage incompatibly definitely wouldn't break the web then I'd like to see some data. Regards, Maciej
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Sep 8, 2009, at 1:35 AM, Jonas Sicking wrote: I think Firefox would be willing to break compat. The question is if microsoft is. Which we need to ask them. We would be very hesitant to break LocalStorage API compatibility for Safari, at least without some demonstration that the level of real- world breakage is low (including for mobile-specific/iPhone-specific sites). Regards, Maciej
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Tue, Sep 8, 2009 at 3:14 PM, Aaron Boodmana...@google.com wrote: On Tue, Sep 8, 2009 at 12:02 PM, Jonas Sickingjo...@sicking.cc wrote: On Tue, Sep 8, 2009 at 11:53 AM, Aaron Boodmana...@google.com wrote: On Tue, Sep 8, 2009 at 11:23 AM, Chris Jonescjo...@mozilla.com wrote: In general, I agree with Rob about this proposal. What problem with storage mutex as spec'd today does your proposal solve? The spec requires a single storage mutex for the entire UA. Therefore in a MELUA a web page can become unresponsive while waiting for some other page to give up the lock. This is not good and something we have tried to avoid everywhere else in the spec. Attempts to address this by doing per-origin locks wind up with deadlocks being possible. How could this happen. It certainly does sound scary to have a bunch of locks that can be grabbed in arbitrary order, but I can't off the top of my head think of any ways where deadlocks can happen. From a Hixie mail way back in March: http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-March/018991.html snip If it can be shown that it is not ever possible for script in one origin to synchronously invoke script in another origin, then I guess we could have per-origin locks instead of a single lock. /snip Here are two cases I know of where it is possible to have synchronous script execution across origins: * Plugins. It is possible for script to invoke a plugin function in one frame, and for the plugin to synchronously execute script in another frame. We have addressed this in the spec by saying that invoking a plugin releases the storage mutex, but that doesn't really solve the problem. We are exchanging violation of run-to-completion for deadlock. I guess it is an improvement, but it is still a bug. * In WebKit, onresize is invoked synchronously. You can cause cross-origin synchronous script execution by resizing an iframe. AFAIK, the spec does not disallow this event from being synchronous. I found the second case after only a few minutes of thinking about it. I suspect there are more since it was never a design goal to prevent synchronous script execution across origins. When we designed postMessage it certainly was a design goal to make synchronous cross-origin script execution never happen. As I understood it for two reasons: 1. The script in the origin being called would never know how much JS stack was available, and so could never reliably call a function. This was particularly a concern expressed by chrome developers. 2. This would force the two origins to run in the same process (or at least processes running in lock-step). We could patch each case like this by releasing the mutex, but each time we do that, we poke another hole in run to completion. I also suspect such patches will be difficult to maintain. Even if we update implementations to invoke onresize asynchronously, how confident can we be in interactions between such unrelated components as event dispatch and storage continuing to work correctly? Finally, even with these hacks, we have problems. In a MELUA (I love that acronym!), with per-origin local storage mutexes, we still can end up with unresponsive UI because it is possible to have the same origin running on different event loops. Adding an asynchronous step to acquire the mutex neatly solves all these problems. We get run-to-completion, and we never block an event loop on waiting for a mutex. But we have to break compat with existing pages... / Jonas
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Tue, Sep 8, 2009 at 4:00 PM, Maciej Stachowiakm...@apple.com wrote: On Sep 8, 2009, at 1:35 AM, Jonas Sicking wrote: I think Firefox would be willing to break compat. The question is if microsoft is. Which we need to ask them. We would be very hesitant to break LocalStorage API compatibility for Safari, at least without some demonstration that the level of real-world breakage is low (including for mobile-specific/iPhone-specific sites). Even if that means that you'll for all future will need to use a per-domain mutex protecting localStorage? And even possibly a global mutex as currently specified? / Jonas
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Tue, Sep 8, 2009 at 4:27 PM, Chris Jonescjo...@mozilla.com wrote: With my proposal, a web app that tests whether localStorage is available with |window.localStorage === undefined| would conclude that it's not, because that object would not be available until |window.transaction| or whatever was accessed. In Jeremy's and Benjamin's proposals, a web app using that would check would conclude that window.localStorage *is* available. Aha, I had missed this (clever) idea in your original proposal. I also do not have a strong opinion on having vs not having this as part of the API. - a
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Wed, Sep 9, 2009 at 4:39 AM, Chris Jones cjo...@mozilla.com wrote: Aaron Boodman wrote: On Tue, Sep 8, 2009 at 11:23 AM, Chris Jonescjo...@mozilla.com wrote: In general, I agree with Rob about this proposal. What problem with storage mutex as spec'd today does your proposal solve? The spec requires a single storage mutex for the entire UA. Therefore in a MELUA a web page can become unresponsive while waiting for some other page to give up the lock. This is not good and something we have tried to avoid everywhere else in the spec. Attempts to address this by doing per-origin locks wind up with deadlocks being possible. Aaron Boodman wrote: On Tue, Sep 8, 2009 at 1:41 AM, Robert O'Callahanrob...@ocallahan.org wrote: What is the intended semantics here? Chris' explicit commitTransaction would throw an exception if the transaction was aborted due to data inconsistency, leaving it up to the script to retry --- and making it clear to script authors that non-storage side effects during the transaction are not undone. How would you handle transaction aborts? Calls to transaction() are queued and executed serially per-origin with exclusive access. There is no such thing as a transaction abort because there cannot be consistency problems because of the serialized access. No, transactions can still fail. They can fail in ways immediately hidden from the script that requested them if the UA has to interrupt the conceptually executing transaction in the ways enumerated in a separate branch of this thread. Later script executions can observe inconsistent state unless more is specified by your proposal. Transactions can also fail visibly if write-to-disk fails (probably also in other ways I haven't considered). It's not clear what should happen wrt to your proposal in this case. If so, I agree with roc's responses to them that they could probably be handled without surfacing errors to the developer. OTOH, I'm not really against adding the concept of fallibility here. In fact, I believe that the Synchronous database API describes the same transaction semantics as I proposed in the OP. That spec adds implicit begin/commitTransaction and read-only transactions, but otherwise the semantics are the same. So I'd like to amend my original proposal to be Use Synchronous Web Database API transaction semantics. Except do not offer readTransaction: a transaction is implicitly a read-only transaction if only getItem() is called on localStorage from within localStorage.transaction(). Agree. That is what I was trying to propose, too. I'm not sure where we disagree :). Is it just that my proposal has no concept of errors? I'm not against adding them, mainly I was trying to keep my proposal simple for purposes of discussion. Ay, there's the rub: I think the disagreement is between mutex vs. transaction semantics. So far, I think perhaps mutex has been used as shorthand for transaction. But they aren't the same. I think we all agree that a script may fail to modify localStorage in some situations (irrespective of global mutex vs. per-domain mutex). One camp, wanting mutex semantics, would prefer to pretend that the failures never happen and let scripts clean up the mess (partially-applied changes) if they do occur. This is semantically broken, IMHO. The second camp, wanting transaction semantics, explicitly acknowledge to web authors that localStorage is fallible, guarantee that modifications to localStorage are atomic, and notify scripts when modifications can't be made atomically. This is the same approach taken by Web Database. IMHO, this is much better semantically because (i) it gives web apps stronger guarantees; and (ii) it makes the discussion about global mutex/per-domain mutex/non-blocking an implementation issue rather semantic issue, as it should be. Can those in the first camp explain why mutex semantics is better than transaction semantics? And why it's desirable to have one DB spec specify transaction semantics (Web Database) and a second specify mutex semantics (localStorage)? The way I understand it, there's 3 camps...and I think they've been abusing both the word transaction and mutex. We should probably all start being more precise with our wording in this respect. :-) Those who want pessimistic transactions. I.e. using locking so that you never need to do a rollback (because it can never fail). This would be compatible with either a sync or an async interface. Those who optimistic transactions. I.e. rollback may happen. Either we need to restrict what can be done during a localStorage transaction or we need to have an exception that tells the script to undo itself. This was the original proposal, AFAICT. It would work with both a sync or an async interface. Those who want a queue. I.e. those who want an asynchronous callback based interface and the UA will only call one callback at a
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
Jonas Sicking wrote: On Tue, Sep 8, 2009 at 1:38 PM, Chris Jonescjo...@mozilla.com wrote: No one has responded directly to my original proposal of making |window.localStorage === undefined| until |window.transaction| or whatever has been accessed. Unlike your proposal and a similar one from Jeremy, mine is a safe (non-racy) way for spec-compliant UAs to bend backwards compatibility without explicitly breaking it. How is that not breaking compatibility? Any existing pages will directly access window.localStorage to read/write data. Such pages would break with your proposal, no? I think I haven't been explaining my proposal clearly enough. Let me try to err on the other side of brevity. Sorry if this is too pedantic. Currently, UAs either (i) support localStorage or (ii) don't. Web apps using localStorage must therefore determine whether the UA they're running in supports localStorage, and adjust their behavior accordingly. If they don't, they won't run in type (ii) UAs. So how do web apps implement this check? If I were implementing it, I would test |window.localStorage === undefined|. I have no clue how web devs actually implement it; I solicited feedback from them in earlier posts. Assuming that web apps test for |window.localStorage === undefined|, then let's explore the two possibilities for changing the localStorage API to include transactions. If the spec were to add transactional localStorage, then UAs in the field would then fall into three classes (i) support transactional localStorage [compliant]; (ii) support non-transactional localStorage [non-compliant]; (iii) don't support localStorage at all [non-compliant]. Web apps would also fall into three classes (a) use transactional localStorage [compliant]; (b) use non-transactional localStorage [non-compliant]; (c) don't use localStorage [NA]. Note that web app type (c) is irrelevant to the discussion, since changes to the localStorage API don't affect them. With my proposal, a web app that tests whether localStorage is available with |window.localStorage === undefined| would conclude that it's not, because that object would not be available until |window.transaction| or whatever was accessed. In Jeremy's and Benjamin's proposals, a web app using that would check would conclude that window.localStorage *is* available. Only type (b) web apps are affected by this difference. In my proposal, in type (i) browsers, type (b) web apps would conclude that localStorage was not supported and thus those web apps would not try to use it. In other words, they would *not* break, just not use localStorage. In Jeremy's and Benjamin's proposals, type (b) web apps would conclude that localStorage *is* supported, and would use it in a racy way (outside of transactions), to the detriment of themselves and UAs. Type (c) web apps would need to use the check |window.transaction === undefined| to determine whether *transactional* localStorage was supported, and thus aren't affected by either proposal. If this explication isn't clear, please let me know and I will construct a compatibility matrix based on (a), (b), (c), and (i), (ii), (iii). The |window.localStorage === undefined|-check assumption is a big one (!). If that's not how the compatibility check is done in the wild, then my proposal won't ameliorate the breakage of type (b) web apps. Cheers, Chris
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
Chris Jones wrote: Jonas Sicking wrote: On Tue, Sep 8, 2009 at 1:38 PM, Chris Jonescjo...@mozilla.com wrote: Type (c) web apps would need to use the check |window.transaction === undefined| to determine whether *transactional* localStorage was supported, and thus aren't affected by either proposal. Ugh, sorry, I mean type (a) web apps; those that use transactional localStorage. Cheers, Chris
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Sep 8, 2009, at 4:27 PM, Chris Jones wrote: I think I haven't been explaining my proposal clearly enough. Let me try to err on the other side of brevity. Sorry if this is too pedantic. Currently, UAs either (i) support localStorage or (ii) don't. Web apps using localStorage must therefore determine whether the UA they're running in supports localStorage, and adjust their behavior accordingly. If they don't, they won't run in type (ii) UAs. So how do web apps implement this check? If I were implementing it, I would test |window.localStorage === undefined|. I have no clue how web devs actually implement it; I solicited feedback from them in earlier posts. I think this has a few problems: 1) There are probably Web apps that actually detect based on user agent sniffing, rather than based on feature testing. No matter how much we tell people to feature-test, there are still many Web sites (and even popular JavaScript libraries) out there that look at the User-Agent string. 2) There are probably mobile-targeted Web apps aimed at devices like iPhone, Palm Pre and Android which just assume LocalStorage is there (since it is available on all their target devices). 3) For Web apps that used the older feature in good faith, it would be suddenly taken away from their existing deployed apps. I expect this will make developers unhappy even if it doesn't technically break their site but rather just makes some functionality regress in browsers that support LocalStorage, down to the level of non- supporting browsers. I would rather let them keep using a suboptimal form of the feature than take it away supposedly for their own good. Regards, Maciej
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Wed, Sep 9, 2009 at 9:54 AM, Maciej Stachowiak m...@apple.com wrote: On Sep 8, 2009, at 4:10 PM, Jonas Sicking wrote: On Tue, Sep 8, 2009 at 4:00 PM, Maciej Stachowiakm...@apple.com wrote: On Sep 8, 2009, at 1:35 AM, Jonas Sicking wrote: I think Firefox would be willing to break compat. The question is if microsoft is. Which we need to ask them. We would be very hesitant to break LocalStorage API compatibility for Safari, at least without some demonstration that the level of real-world breakage is low (including for mobile-specific/iPhone-specific sites). Even if that means that you'll for all future will need to use a per-domain mutex protecting localStorage? And even possibly a global mutex as currently specified? I don't think telling this story to users or developers will make them satisfied with the breakage, so the even if is not very relevant. I think there are ways to solve the problem without completely breaking compatibility. For example: - Keep the current LocalStorage API, but make it give no concurrency guarantees whatsoever (only single key/value accesses are guaranteed atomic). - Add a simple optional transactional model for aware authors who want better consistency guarantees. This might not meaningfully break existing content, unlike proposals for effectively mandatory new API calls. Particularly since IE doesn't have any kind of storage mutex. Yet another possibility is to keep a per-domain mutex, also offer a transactional API, and accept that careless authors may indefinitely lock up the UI for all pages in their domain (up to the slow script execution limit) if they code poorly, but in exchange won't have unexpected race conditions with themselves. I'll see if I can't get any numbers on how widely used localStorage is today. Assuming that we can't break compat (which I think is a strong possibility) I think Maciej's idea is the best one so far. That said, I think Chris's |window.localStorage == undefined| could work. Both would be confusing to web developers in different ways, but I don't think that's avoidable (unless we break compat).
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
I'm happy to see this getting sorted out. I like maciej's idea too.- Keep the current LocalStorage API, but make it give no concurrency guarantees whatsoever. (IE's impl i think) - Add a simple optional transactional model for aware authors who want better consistency guarantees. There is one use-case to keep in mind... setting key/values onunload... how can we provide transactional access at that time? Maybe we could guarantee that transact calls made in (and perhaps prior to) onbeforeunload will be satisfied prior to onunload. On Tue, Sep 8, 2009 at 6:21 PM, Jeremy Orlow jor...@chromium.org wrote: On Wed, Sep 9, 2009 at 9:54 AM, Maciej Stachowiak m...@apple.com wrote: On Sep 8, 2009, at 4:10 PM, Jonas Sicking wrote: On Tue, Sep 8, 2009 at 4:00 PM, Maciej Stachowiakm...@apple.com wrote: On Sep 8, 2009, at 1:35 AM, Jonas Sicking wrote: I think Firefox would be willing to break compat. The question is if microsoft is. Which we need to ask them. We would be very hesitant to break LocalStorage API compatibility for Safari, at least without some demonstration that the level of real-world breakage is low (including for mobile-specific/iPhone-specific sites). Even if that means that you'll for all future will need to use a per-domain mutex protecting localStorage? And even possibly a global mutex as currently specified? I don't think telling this story to users or developers will make them satisfied with the breakage, so the even if is not very relevant. I think there are ways to solve the problem without completely breaking compatibility. For example: - Keep the current LocalStorage API, but make it give no concurrency guarantees whatsoever (only single key/value accesses are guaranteed atomic). - Add a simple optional transactional model for aware authors who want better consistency guarantees. This might not meaningfully break existing content, unlike proposals for effectively mandatory new API calls. Particularly since IE doesn't have any kind of storage mutex. Yet another possibility is to keep a per-domain mutex, also offer a transactional API, and accept that careless authors may indefinitely lock up the UI for all pages in their domain (up to the slow script execution limit) if they code poorly, but in exchange won't have unexpected race conditions with themselves. I'll see if I can't get any numbers on how widely used localStorage is today. Assuming that we can't break compat (which I think is a strong possibility) I think Maciej's idea is the best one so far. That said, I think Chris's |window.localStorage == undefined| could work. Both would be confusing to web developers in different ways, but I don't think that's avoidable (unless we break compat).
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Tue, Sep 8, 2009 at 6:00 PM, Maciej Stachowiakm...@apple.com wrote: On Sep 8, 2009, at 4:27 PM, Chris Jones wrote: I think I haven't been explaining my proposal clearly enough. Let me try to err on the other side of brevity. Sorry if this is too pedantic. Currently, UAs either (i) support localStorage or (ii) don't. Web apps using localStorage must therefore determine whether the UA they're running in supports localStorage, and adjust their behavior accordingly. If they don't, they won't run in type (ii) UAs. So how do web apps implement this check? If I were implementing it, I would test |window.localStorage === undefined|. I have no clue how web devs actually implement it; I solicited feedback from them in earlier posts. I think this has a few problems: 1) There are probably Web apps that actually detect based on user agent sniffing, rather than based on feature testing. No matter how much we tell people to feature-test, there are still many Web sites (and even popular JavaScript libraries) out there that look at the User-Agent string. 2) There are probably mobile-targeted Web apps aimed at devices like iPhone, Palm Pre and Android which just assume LocalStorage is there (since it is available on all their target devices). 3) For Web apps that used the older feature in good faith, it would be suddenly taken away from their existing deployed apps. I expect this will make developers unhappy even if it doesn't technically break their site but rather just makes some functionality regress in browsers that support LocalStorage, down to the level of non-supporting browsers. I would rather let them keep using a suboptimal form of the feature than take it away supposedly for their own good. My concern with this solution is that pages are still very likely to not think of any of this and instead use the simplest API available, which undoubtedly will be the transaction-free unsafe API. Thus for most pages this isn't better than not doing anything. / Jonas
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Wed, Sep 9, 2009 at 12:54 PM, Maciej Stachowiak m...@apple.com wrote: Yet another possibility is to keep a per-domain mutex, also offer a transactional API, and accept that careless authors may indefinitely lock up the UI for all pages in their domain (up to the slow script execution limit) if they code poorly, but in exchange won't have unexpected race conditions with themselves. I prefer this. If the app has a bug that causes one window to get stuck, most likely that's the window the user cares about anyway, so the other windows (temporarily) getting stuck is not a big deal. Also, if one window is hogging the database then an asynchronously dispatched transaction is going to be delayed indefinitely*, so some or all user operations in other windows will not be able to complete, even if the other windows appear to be responsive to events. * Unless we adopt a model where scripts are required to detect aborts and restart. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
Jeremy Orlow wrote: On Wed, Sep 9, 2009 at 4:39 AM, Chris Jones cjo...@mozilla.com mailto:cjo...@mozilla.com wrote: Aaron Boodman wrote: On Tue, Sep 8, 2009 at 11:23 AM, Chris Jonescjo...@mozilla.com mailto:cjo...@mozilla.com wrote: In general, I agree with Rob about this proposal. What problem with storage mutex as spec'd today does your proposal solve? The spec requires a single storage mutex for the entire UA. Therefore in a MELUA a web page can become unresponsive while waiting for some other page to give up the lock. This is not good and something we have tried to avoid everywhere else in the spec. Attempts to address this by doing per-origin locks wind up with deadlocks being possible. Aaron Boodman wrote: On Tue, Sep 8, 2009 at 1:41 AM, Robert O'Callahanrob...@ocallahan.org mailto:rob...@ocallahan.org wrote: What is the intended semantics here? Chris' explicit commitTransaction would throw an exception if the transaction was aborted due to data inconsistency, leaving it up to the script to retry --- and making it clear to script authors that non-storage side effects during the transaction are not undone. How would you handle transaction aborts? Calls to transaction() are queued and executed serially per-origin with exclusive access. There is no such thing as a transaction abort because there cannot be consistency problems because of the serialized access. No, transactions can still fail. They can fail in ways immediately hidden from the script that requested them if the UA has to interrupt the conceptually executing transaction in the ways enumerated in a separate branch of this thread. Later script executions can observe inconsistent state unless more is specified by your proposal. Transactions can also fail visibly if write-to-disk fails (probably also in other ways I haven't considered). It's not clear what should happen wrt to your proposal in this case. If so, I agree with roc's responses to them that they could probably be handled without surfacing errors to the developer. OTOH, I'm not really against adding the concept of fallibility here. In fact, I believe that the Synchronous database API describes the same transaction semantics as I proposed in the OP. That spec adds implicit begin/commitTransaction and read-only transactions, but otherwise the semantics are the same. So I'd like to amend my original proposal to be Use Synchronous Web Database API transaction semantics. Except do not offer readTransaction: a transaction is implicitly a read-only transaction if only getItem() is called on localStorage from within localStorage.transaction(). Agree. That is what I was trying to propose, too. I'm not sure where we disagree :). Is it just that my proposal has no concept of errors? I'm not against adding them, mainly I was trying to keep my proposal simple for purposes of discussion. Ay, there's the rub: I think the disagreement is between mutex vs. transaction semantics. So far, I think perhaps mutex has been used as shorthand for transaction. But they aren't the same. I think we all agree that a script may fail to modify localStorage in some situations (irrespective of global mutex vs. per-domain mutex). One camp, wanting mutex semantics, would prefer to pretend that the failures never happen and let scripts clean up the mess (partially-applied changes) if they do occur. This is semantically broken, IMHO. The second camp, wanting transaction semantics, explicitly acknowledge to web authors that localStorage is fallible, guarantee that modifications to localStorage are atomic, and notify scripts when modifications can't be made atomically. This is the same approach taken by Web Database. IMHO, this is much better semantically because (i) it gives web apps stronger guarantees; and (ii) it makes the discussion about global mutex/per-domain mutex/non-blocking an implementation issue rather semantic issue, as it should be. Can those in the first camp
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Sun, Sep 6, 2009 at 10:00 AM, Chris Jones cjo...@mozilla.com wrote: Robert O'Callahan wrote: In HTML5 we generally take the approach that if a UA is unable to satisfy spec semantics due to resource limits or other problems in the environment, then it's OK to deviate from the spec. Applying that principle here, we would not need to provide database consistency in the presence of these failures. I don't like the idea of the spec creating an illusion that's impossible for browser vendors to maintain. Then the question is which spec features should be specified as resilient to failures in the environment. The other major unresolved questions in my mind are whether there is a class of apps worth addressing that wants to use localStorage and shouldn't be saddled with an explicitly transactional API, and whether the compatibility break is even possible. Only the wider community can answer these... Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Sat, Sep 5, 2009 at 10:22 AM, Chris Jones cjo...@mozilla.com wrote: And if the intention is to make scripts appear to run atomically, then I think there are better ways to specify that than storage mutex. That sounds good, how? My problem with storage mutex boils down to the fact that by the letter of the spec, a script can lock out the UA indefinitely by just reading a cookie. What do you mean lock out the UA? Transactional semantics seems to be a better abstraction, and an ancillary benefit is that it's much easier to implement to boot. Though even if it were harder to implement, I would still argue for it. I agree with you. However, there is still the compatibility issue. (Some people may also argue that what we currently have is simpler for authors than any explicit transactional API --- personally I'm unsure if that matters.) Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
Robert O'Callahan wrote: On Sat, Sep 5, 2009 at 10:22 AM, Chris Jones cjo...@mozilla.com mailto:cjo...@mozilla.com wrote: And if the intention is to make scripts appear to run atomically, then I think there are better ways to specify that than storage mutex. That sounds good, how? My OP is my weak attempt at getting closer to this. Weak because it's still rather imperative rather than declarative, but that's in the style of the rest of HTML5. If I could start from scratch, I would define run-to-completion semantics (which I love, BTW) in terms of global state visible to script and a happens-before relation on events that might change that state. But it's not clear to me that a Java-memory-model-type spec would be better for the web than a spec like HTML5's. My problem with storage mutex boils down to the fact that by the letter of the spec, a script can lock out the UA indefinitely by just reading a cookie. What do you mean lock out the UA? I mean prevent the UA from affecting a script's execution. The cases I've thought of so far where we will probably have to break storage-mutex semantics are * clear private data * close tab * quit UA * slow script timeout * store-to-disk failure * crash For those cases, IMHO transactional semantics is better because it gives scripts the guarantee that operations on localStorage are atomic. In the secret-storage-mutex world, if you agree that the cases above imply that the UA will have to interrupt scripts, then it's possible for scripts to make changes to localStorage that are only partially applied, going by the letter of the storage mutex spec. I think that for sites that would care (e.g. gmail), partially-applied changes are a bad thing. And as I argued in the OP, I think localStorage should be designed only with sites like gmail in mind. I'm beginning to think that this discussion is more about DB consistency than multi-event-loop UAs. I think it's telling that all of the cases I listed above could arise in single-process Gecko except clear private data (and maybe close tab, not sure). Transactional semantics seems to be a better abstraction, and an ancillary benefit is that it's much easier to implement to boot. Though even if it were harder to implement, I would still argue for it. I agree with you. However, there is still the compatibility issue. (Some people may also argue that what we currently have is simpler for authors than any explicit transactional API --- personally I'm unsure if that matters.) API breakage is a problem, but does my proposal to only sort of break compatibility through pretending not to offer window.localStorage until window.beginTransaction is accessed not satisfy? Cheers, Chris
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Sun, Sep 6, 2009 at 4:55 AM, Chris Jones cjo...@mozilla.com wrote: I mean prevent the UA from affecting a script's execution. The cases I've thought of so far where we will probably have to break storage-mutex semantics are * clear private data * close tab * quit UA I think these could appear to complete immediately while doing their work asynchronously in the background, given the assumption that content script execution time is bounded (if it takes too long we enforce the assumption using the slow-script timeout). * slow script timeout * store-to-disk failure * crash In HTML5 we generally take the approach that if a UA is unable to satisfy spec semantics due to resource limits or other problems in the environment, then it's OK to deviate from the spec. Applying that principle here, we would not need to provide database consistency in the presence of these failures. And as I argued in the OP, I think localStorage should be designed only with sites like gmail in mind. Yes ... I'm not so sure about that. I'm beginning to think that this discussion is more about DB consistency than multi-event-loop UAs. I think it's telling that all of the cases I listed above could arise in single-process Gecko except clear private data (and maybe close tab, not sure). Currently in Gecko clear private data, close tab and quit would all wait for running scripts to complete. But indeed, the consistency you want isn't provided by any browser today if the app encounters resource limits or hardware failures. Storing a single key-value pair should be atomic in the case of failure, so apps have to roll-your-own consistency and recovery using that. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
Robert O'Callahan wrote: On Sun, Sep 6, 2009 at 4:55 AM, Chris Jones cjo...@mozilla.com mailto:cjo...@mozilla.com wrote: I mean prevent the UA from affecting a script's execution. The cases I've thought of so far where we will probably have to break storage-mutex semantics are * clear private data * close tab * quit UA I think these could appear to complete immediately while doing their work asynchronously in the background, given the assumption that content script execution time is bounded (if it takes too long we enforce the assumption using the slow-script timeout). * slow script timeout * store-to-disk failure * crash In HTML5 we generally take the approach that if a UA is unable to satisfy spec semantics due to resource limits or other problems in the environment, then it's OK to deviate from the spec. Applying that principle here, we would not need to provide database consistency in the presence of these failures. I don't like the idea of the spec creating an illusion that's impossible for browser vendors to maintain. I'm beginning to think that this discussion is more about DB consistency than multi-event-loop UAs. I think it's telling that all of the cases I listed above could arise in single-process Gecko except clear private data (and maybe close tab, not sure). Currently in Gecko clear private data, close tab and quit would all wait for running scripts to complete. But indeed, the consistency you want isn't provided by any browser today if the app encounters resource limits or hardware failures. Storing a single key-value pair should be atomic in the case of failure, so apps have to roll-your-own consistency and recovery using that. Force quit will interrupt scripts. Desktop apps have to worry about DB consistency, and DBs for those provide transactional consistency guarantees. It seems undesirable to deny them to web apps using localStorage, especially since localStorage seems more author-friendly than Web Database. (Undesirable excepting compatibility with existing localStorage impls.) This is a rhetorical point not necessarily directed at you personally, since I think we're pretty much on the same page. Cheers, Chris
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Fri, Sep 4, 2009 at 4:02 PM, Chris Jones cjo...@mozilla.com wrote: I'd like to propose that HTML5 specify different schemes than a conceptual global storage mutex to provide consistency guarantees for localStorage and cookies. Cookies would be protected according to Benjamin Smedberg's post in the [whatwg] Storage mutex and cookies can lead to browser deadlock thread. Roughly, this proposal would give scripts a consistent view of document.cookie until they completed. AIUI this is stronger consistency than Google Chrome provides today, and anecdotal evidence suggests even their weaker consistency hasn't broken the web. To be fair, IE is in the same boat...which makes this argument even stronger, I think. localStorage would be changed in a non-backwards-compatible way. I believe that web apps can be partitioned into two classes: those that have planned for running concurrently (single-event-loop or not) in multiple browsing contexts, and those that haven't. I further posit that the second class would break when run concurrently in multiple contexts regardless of multiple event loops, and thus regardless of the storage mutex. Even in the single-event-loop world, sites not prepared to be loaded in multiple tabs can stomp each other's data even though script execution is atomic. (I wouldn't dare use my bank's website in two tabs at the same time in a single-event-loop browser.) In other words, storage mutex can't help the second class of sites. (I also believe that there's a very large, third class of pages that work accidentally when run concurrently in multiple contexts, even though they don't plan for that. This is likely because they don't keep quasi-persistent data on the client side.) Based on that, I believe localStorage should be designed with the first class of web apps (those that have considered data consistency across multiple concurrent contexts) in mind, rather than the second class. Is a conceptual global storage mutex the best way for, say, gmail to guarantee consistency of its e-mail/contacts database? I don't believe so: I think that a transactional localStorage is preferable. Transactional localStorage is easier for browser vendors to implement and should result in better performance for web apps in multi-process UAs. It's more of a burden on web app authors than the hidden storage mutex, but I think the benefits outweigh the cost. I propose adding the functions window.localStorage.beginTransaction() window.localStorage.commitTransaction() or window.beginTransaction() window.commitTransaction() (The latter might be preferable if we later decide to add more resources with transactional semantics.) localStorage.getItem(),. setItem(), .removeItem(), and .clear() would remain specified as they are today. beginTransaction() would do just that, open a transaction. Calling localStorage.*() outside of an open transaction would cause a script exception to be thrown; this would unfortunately break all current clients of localStorage. There might be cleverer ways to mitigate this breakage by a UA pretending not to support localStorage until a script called beginTransaction(). yieldForStorageUpdates() would no longer be meaningful and should be removed. A transaction would successfully commit, atomically applying its modifications to localStorage, if localStorage was not modified between beginTransaction() and commitTransaction(). Note that a transaction consisting entirely of getItem() could fail just as those actually modifying localStorage. If a transaction failed, the UA would throw a TransactionFailed exception to script. The UA would be allowed to throw this exception at any time between beginTransaction() and commitTransaction(). There are numerous ways to implement transactional semantics. Single-event-loop UAs could implement beginTransaction() and commitTransaction() as no-ops. Multi-event-loop UAs could reuse the global storage mutex if they had already implemented that (beginTransaction() == lock, commitTransaction() == unlock). Some edge cases: * calling commitTransaction() without beginTransaction() would throw an exception * transactions would not be allowed to be nested, even on different localStorage DBs. E.g. if site A's script begins a transaction on A.localStorage, and calls into site B's script embedded in an iframe which begins a transaction on B.localStorage, an exception would be thrown. * transactions *could* be spread across script executions, alert() dialogs, sync XHR, or anywhere else the current HTML5 spec requires the storage mutex be released. Note that UAs wishing to forbid that behavior could simply throw a TransactionFailed exception where the storage mutex would have been released in the current spec. Or this could be made illegal by the spec. * it's not clear to me how to handle async XHRs and Worker messages sent from within a failed
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Fri, 04 Sep 2009 09:02:45 +0200, Chris Jones cjo...@mozilla.com wrote: Feedback very much desired. I'm not really sure what to say other than that I'm not at all a fan of a change that breaks existing deployments. I thought that was a pretty clear outcome from last time we went about this. I also thought it was pretty clear we wanted the burden to be on user agents. (I also recall, but am not a 100% sure, that developers from Mozilla agreed to this, even though it would be hard to make it all work in Gecko.) -- Anne van Kesteren http://annevankesteren.nl/
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
Jeremy Orlow wrote:
I mostly agree with your assertions about the type of developer who's
using localStorage. It sure would be nice if we could give developers
powerful APIs and keep them simple and make it possible to implement
them in a performant manner. Unfortunately, I think the current design
cannot be changed to meet possible to implement in a performant manner
without breaking backwards compatibility.
I mostly agree with that. I like the idea of storage mutex
conceptually, but I think it could end up hurting the Web as much as it
helps. For multi-event-loop UAs, |x = window.localStorage.getItem('x')|
is an innocent-looking operation with global (browser-wide)
consequences, and web devs will need to be mindful of those consequences.
Part of me thinks that this API should match the WebDatabase API more.
For example, you'd call a function with a callback. That callback
would be given the localStorage object which it'd use to
do manipulations. Etc. But part of me like what you're suggesting
here. I actually think the idea of throwing an exception whenever
there's a serialization problem could be very compelling, and could keep
the door wide open for future performance enhancements. It's even
possible that javascript engines could embed elements of software
transactional memory in the future to eliminate the need to make such
calls. That seems really exciting.
It might also be possible to combine the 2 ideas: you call a function
with your callback and the callback is given a localStorage object which
is only valid within the callback, but an exception can be thrown when
there's a problem with the transaction. Of course, the benefit to
explicitly starting and ending a transaction is that it can span
setTimeouts, event handlers, etc. On the other hand, I wonder if the
cases where an app would do this and still be able to recover from a
transaction failure would be limited.
I would be satisfied with a callback API, even an async one. If we were
to go forward with transactional localStorage, I think the main API
design decision would be whether or not to hide transactions from
authors. Obviously hiding them would be desirable, but unlike
WebDatabase, localStorage can't easily import SQL semantics. If we were
to hide them, I think we should offer a StorageTransaction object
describing the operations to be performed on the DB, and offer the
function localStorage.executeTransaction(txnObject). In this way we
could avoid the side-effecty problems of generic scripts accessing
localStorage.
But personally, I prefer exposed transactions. A transaction might
contain data dependencies --- for example, x = getItem('x'), x += 1,
setItem('x', x). It seems to me very hard to capture this in an API
that hides transactions from the user while at the same time not turning
into a scripting language unto itself (or providing a strongly
restricted scripting environment in a transaction callback).
Another thing we might want to consider is making transactions optional.
This would satisfy group 1 and 2, but would put the group 3 you
mentioned at more risk. In other words, not calling beginTransaction
would not be fatal. It would just mean localStorage works as currently
spec'ed. But, doing it within a transaction (be it a callback or within
___Transaction calls) would give you additional guarantees.
I would really prefer to not do that. Can any web devs using
localStorage comment on how they detect whether browsers support that
feature? If the test is something like
|window.hasProperty('localStorage')|, then I would prefer to make this
return |false| until |window.beginTransaction| was accessed.
Cheers,
Chris
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
On Fri, Sep 4, 2009 at 9:44 PM, Jeremy Orlow jor...@chromium.org wrote: I think it's pretty clear that the spec, as is, is not possible to implement without making it trivial for a single website to lock up all of your event loops I don't think that's clear at all, yet. It's clearly *hard* to implement, and Chris' proposal for transactional localStorage is a lot easier to implement, so if we can get away with the compatibility break, we should. This is especially true if the storage mutex extends to cookies since one tab running a poorly written site can lock everything up. Only if you actually implement the semantics using a single global lock. I think we could do better. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] RFC: Alternatives to storage mutex for cookies and localStorage
Robert O'Callahan wrote: On Fri, Sep 4, 2009 at 9:44 PM, Jeremy Orlow jor...@chromium.org mailto:jor...@chromium.org wrote: I think it's pretty clear that the spec, as is, is not possible to implement without making it trivial for a single website to lock up all of your event loops I don't think that's clear at all, yet. It's clearly *hard* to implement, and Chris' proposal for transactional localStorage is a lot easier to implement, so if we can get away with the compatibility break, we should. I don't think it's hard to implement, even implementing it without literally using a global mutex. I just don't think it solves the problem of guaranteeing cross-tab data integrity for poorly written sites. And if the intention is to make scripts appear to run atomically, then I think there are better ways to specify that than storage mutex. My problem with storage mutex boils down to the fact that by the letter of the spec, a script can lock out the UA indefinitely by just reading a cookie. Obviously we're going to have to break that in some situations. So if that's the case, why present that abstraction to scripts? Transactional semantics seems to be a better abstraction, and an ancillary benefit is that it's much easier to implement to boot. Though even if it were harder to implement, I would still argue for it. Cheers, Chris This is especially true if the storage mutex extends to cookies since one tab running a poorly written site can lock everything up. Only if you actually implement the semantics using a single global lock. I think we could do better. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
