Re: [whatwg] Storage mutex feedback

[timeless]

On Sun, Aug 30, 2009 at 4:06 AM, Ian Hicksoni...@hixie.ch wrote:
 Upon further consideration I've renamed getStorageUpdates() to
 yieldForStorageUpdates().

If getStorageUpdates() actually returned how *many* updates there
were, it could be a vaguely useful name.

If the answer is 0, then my application knows it doesn't need to try
to figure out how the world changed, right?

Re: [whatwg] Storage mutex and cookies can lead to browser deadlock

[Drew Wilson]

I think the canonical racy case is the page wants to keep a counter for the
number of times event X occurs in a cookie or local storage.
It doesn't seem to be possible to achieve this without the mutex - the
proposal below would break down if two pages tried to increment the cookie
value simultaneously (if both pages changed cookieValue=3 to
cookieValue=4 at the same time, the result of your merge step would likely
be cookieValue=4, not cookieValue=5 as one might intend).

-atw

On Thu, Sep 3, 2009 at 1:08 PM, Benjamin Smedberg bsmedb...@mozilla.comwrote:

 On 9/1/09 7:31 PM, Jeremy Orlow wrote:

  Does the silence mean that no one has any intention of implementing
  this?  If so, maybe we should resign ourselves to a break in the single
  threaded illusion for cookies.  This doesn't seem too outlandish
  considering that web servers working with cookies will never have such a
  guarantee and given that we have no evidence of widespread breakage with
  IE 8 and Chrome.

 We (Firefox) just started looking at this seriously: the implications of
 the
 global lock are pretty unpleasant.

 The major race condition appears to be code on the web that gets
 document.cookie, parses and modifies the string it to add or remove a
 particular cookie, and sets document.cookie again. This pattern could race
 against HTTP requests which also set cookies.

 Chris Jones proposed that we behave in a script-consistent manner without
 actually doing a global mutex:

 * When a script gets document.cookie, check out a consistent view of the
 cookie data. While the script runs to completion, its view of
 document.cookie does not change.
 * When the script sets document.cookie and runs to completion, calculate
 the
 delta with the original data and commit the changes.
 * HTTP Set-Cookie headers stomp on prior data at any time, but don't
 interfere with the consistent script view above.

 It would be nice to provide an web API to perform the operation of setting
 a
 cookie atomically, just as the Set-Cookie HTTP header does. That is:
 document.setCookie('foo=bar; domain=subd.example.com').

 It's not clear whether/how this same algorithm could be applied to
 localStorage, since the amount of data required to create a consistent
 state
 is potentially much larger. Is there an inherently racy API in
 .localStorage
 which we need to protect with complicate mutex/transactional schemes?

 --BDS

Re: [whatwg] Storage mutex and cookies can lead to browser deadlock

[Benjamin Smedberg]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 9/1/09 7:31 PM, Jeremy Orlow wrote:

 Does the silence mean that no one has any intention of implementing
 this?  If so, maybe we should resign ourselves to a break in the single
 threaded illusion for cookies.  This doesn't seem too outlandish
 considering that web servers working with cookies will never have such a
 guarantee and given that we have no evidence of widespread breakage with
 IE 8 and Chrome.

We (Firefox) just started looking at this seriously: the performance
implications of the current global mutex are pretty unpleasant. Reading the
spec, I didn't realize that the mutex was global instead of per-domain; I
only figured that out reading the discussions here.

The major race condition appears to be code on the web that gets
document.cookie, parses and modifies the string it to add or remove a
particular cookie, and sets document.cookie again. This pattern could race
against HTTP requests which also set cookies.

Chris Jones proposed that we behave in a script-consistent manner without
actually implementing any sort of mutex:

* When a script gets document.cookie, check out a consistent view of the
cookie data. While the script runs to completion, its view of
document.cookie does not change.
* When the script sets document.cookie, it calculates the delta with the
original data and commit the changes.
* The consistent view is maintained until the script runs to completion (or
enters a nested event loop, all the locations which are currently marked as
releasing the storage Mutex in the current spec)
* HTTP Set-Cookie headers stomp on prior data at any time, but don't
interfere with the consistent script view above.

It would be nice to provide a web API to perform the operation of setting a
cookie atomically, just as the Set-Cookie HTTP header does. That is:
document.setCookie('foo=bar; domain=subd.example.com')

It's not clear whether/how this same algorithm could be applied to
localStorage, since the amount of data required to create a consistent state
is potentially much larger. Is there an inherently racy API in .localStorage
which we need to protect with complicated mutex/transactional schemes?

In addition, we had concerns about how the web-browser UI itself might
interact with the global mutex. I don't think we'd want browser UI such as
the cookie manager or Clear Private Data to wait for content script to run
to completion. Can we make it explicit in the spec that the UA may remove
cookies or localStorage data at any time without owning the storage mutex?

- --BDS
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iD8DBQFKoCYdSSwGp5sTYNkRAi9yAKCq3GesFDNDrn134621Hm8XJaMB8ACgmIhC
3VcXlEKmSb2l83ekH0j3Hgg=
=FldI
-END PGP SIGNATURE-

Re: [whatwg] Storage mutex and cookies can lead to browser deadlock

[Benjamin Smedberg]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 9/3/09 4:24 PM, Drew Wilson wrote:
 I think the canonical racy case is the page wants to keep a counter for
 the number of times event X occurs in a cookie or local storage.

Is it important that continue to work without racing? I don't think it's
especially important: it's similar to how HTTP responses may race. The only
thing I'm particularly concerned about is data consistency so that setting
document.cookie does not inadvertently undo changes which happened while the
script was executing.

- --BDS


-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iD8DBQFKoCqPSSwGp5sTYNkRAm7YAJ9TtPVdK8j6sOwRD6ZA+qn45sU0BACfSPlZ
vZMlhKARLnu9PU57C3xjrP4=
=0kXs
-END PGP SIGNATURE-

Re: [whatwg] Storage mutex and cookies can lead to browser deadlock

[Aryeh Gregor]

On Thu, Sep 3, 2009 at 4:25 PM, Benjamin Smedbergbenja...@smedbergs.us wrote:
 * When the script sets document.cookie, it calculates the delta with the
 original data and commit the changes.

What if there's a conflict with other changes that happened in the meantime?

Re: [whatwg] Storage mutex and cookies can lead to browser deadlock

[Drew Wilson]

On Thu, Sep 3, 2009 at 1:32 PM, Benjamin Smedberg bsmedb...@mozilla.comwrote:

 On 9/3/09 4:24 PM, Drew Wilson wrote:
  I think the canonical racy case is the page wants to keep a counter for
  the number of times event X occurs in a cookie or local storage.
 
  It doesn't seem to be possible to achieve this without the mutex - the
  proposal below would break down if two pages tried to increment the
  cookie value simultaneously (if both pages changed cookieValue=3 to
  cookieValue=4 at the same time, the result of your merge step would
  likely be cookieValue=4, not cookieValue=5 as one might intend).

 Is that case important? I think that it's more important to make sure that
 script doesn't inadvertently remove or modify cookies due to races in
 document.cookie, but that maintaining a single-threaded view of the cookie
 data is not desirable.


You're asking the wrong guy - I'm in the don't bother trying to ensure
consistency camp :) I'm just parroting the arguments that were offered
previously.

If I hadn't lost the arguments about providing workers access to cookies,
then I'd say that pages that care about cookie consistency should set
cookies via a SharedWorker. But failing that, it seems like the choices are
fairly stark: ensure consistency and take the performance hit in all
multi-threaded browsers, or explicitly disavow consistency, and console
ourselves by making our future APIs more multi-process friendly.

I do understand the point of view of the correctness over performance camp
- however, as Jeremy points out, it doesn't sound like those arguments have
gotten much traction with actual implementors yet.



 --BDS

Re: [whatwg] Storage mutex and cookies can lead to browser deadlock

[Benjamin Smedberg]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 9/3/09 5:06 PM, Aryeh Gregor wrote:
 On Thu, Sep 3, 2009 at 4:25 PM, Benjamin Smedbergbenja...@smedbergs.us 
 wrote:
 * When the script sets document.cookie, it calculates the delta with the
 original data and commit the changes.
 
 What if there's a conflict with other changes that happened in the meantime?

What kind of conflict? There is no need to merge individual cookies:
whichever one was set (or removed) last wins.

- --BDS
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iD8DBQFKoE5vSSwGp5sTYNkRAhNzAKCb8bvv51i0es73UI8zLgJ+j7d/IwCg1e2K
XYAcugpbUpEqLRgz8SONvxI=
=uf1l
-END PGP SIGNATURE-

Re: [whatwg] Storage mutex and cookies can lead to browser deadlock

[Jeremy Orlow]

On Fri, Sep 4, 2009 at 8:17 AM, Benjamin Smedberg benja...@smedbergs.uswrote:

 What kind of conflict? There is no need to merge individual cookies:
 whichever one was set (or removed) last wins.


I think this strategy would work fine for cookies since the HTTP side of
them is inherently racy.  I think such behavior would be pretty
counter-intuitive for localStorage, though.

If we did go with this strategy, I think we could give access to shared
workers, and someone could use those if they needed better atomicity.

Re: [whatwg] Storage mutex and cookies can lead to browser deadlock

[Drew Wilson]

To be clear, I'm not trying to reopen the topic of giving cookie access to
workers - I'm happy to restrict cookie access to document context (I
probably shouldn't have brought it up again).
I do agree with Jeremy that we should rethink the spec language around
cookie consistency to reflect what implementors are actually willing to
build.

-atw

On Thu, Sep 3, 2009 at 6:17 PM, Jeremy Orlow jor...@chromium.org wrote:

 On Fri, Sep 4, 2009 at 8:17 AM, Benjamin Smedberg 
 benja...@smedbergs.uswrote:

 What kind of conflict? There is no need to merge individual cookies:
 whichever one was set (or removed) last wins.


 I think this strategy would work fine for cookies since the HTTP side of
 them is inherently racy.  I think such behavior would be pretty
 counter-intuitive for localStorage, though.

 If we did go with this strategy, I think we could give access to shared
 workers, and someone could use those if they needed better atomicity.

Re: [whatwg] Storage mutex and cookies can lead to browser deadlock

[Jeremy Orlow]

On Fri, Sep 4, 2009 at 10:25 AM, Drew Wilson atwil...@google.com wrote:

 To be clear, I'm not trying to reopen the topic of giving cookie access to
 workers - I'm happy to restrict cookie access to document context (I
 probably shouldn't have brought it up again).


And to be clear: I don't have strong opinions about cookies
being accessible in Workers, but I do think localStorage needs to
be accessible.  That said, my primary goal here is to make the storage mutex
more reasonable (i.e. something vendors will actually implement).  Shared
worker access would be a plus, though.

Re: [whatwg] Storage mutex feedback

[Jeremy Orlow]

On Fri, Sep 4, 2009 at 2:24 AM, timeless timel...@gmail.com wrote:

 On Sun, Aug 30, 2009 at 4:06 AM, Ian Hicksoni...@hixie.ch wrote:
  Upon further consideration I've renamed getStorageUpdates() to
  yieldForStorageUpdates().

 If getStorageUpdates() actually returned how *many* updates there
 were, it could be a vaguely useful name.

 If the answer is 0, then my application knows it doesn't need to try
 to figure out how the world changed, right?


I like this idea except for one problem:  It doesn't tell whether something
got changed without your knowledge.  If you call alert, access a plugin, etc
it's possible to drop the lock.  I think some sort of global counter,
variable, etc would be more valuable since it solves both problems.

Re: [whatwg] Storage mutex and cookies can lead to browser deadlock

[Michael Nordman]

 Shared worker access would be a plus.
Indeed. The lack of access to LocalStorage in 'workers' forces developers to
use the more difficult database api for all storage needs, and to roll their
own change event mechanisms (based on postMessage). Thats a bunch of busy
work if a name/value pair schema is all your app really needs.

How hard can it be to find a way to allow LocalStorage access to workers :)


On Thu, Sep 3, 2009 at 6:32 PM, Jeremy Orlow jor...@chromium.org wrote:

 On Fri, Sep 4, 2009 at 10:25 AM, Drew Wilson atwil...@google.com wrote:

 To be clear, I'm not trying to reopen the topic of giving cookie access to
 workers - I'm happy to restrict cookie access to document context (I
 probably shouldn't have brought it up again).


 And to be clear: I don't have strong opinions about cookies
 being accessible in Workers, but I do think localStorage needs to
 be accessible.  That said, my primary goal here is to make the storage mutex
 more reasonable (i.e. something vendors will actually implement).  Shared
 worker access would be a plus, though.

Re: [whatwg] Storage mutex and cookies can lead to browser deadlock

[Jeremy Orlow]

On Fri, Sep 4, 2009 at 10:48 AM, Michael Nordman micha...@google.comwrote:

  Shared worker access would be a plus.
 Indeed. The lack of access to LocalStorage in 'workers' forces developers
 to use the more difficult database api for all storage needs, and to roll
 their own change event mechanisms (based on postMessage). Thats a bunch of
 busy work if a name/value pair schema is all your app really needs.


For the record, all the developers I've talked to about the current state of
AppCache+storage+workers have been VERY disheartened.  IE and Firefox have
no intentions of supporting WebDatabase any time soon.  localStorage is not
available from workers.  AppCache requires apps to be 100% client based (the
server needs to server static pages and the logic must be in JS) if you have
any personalization/authentication.  Workers are only accessible via message
passing.  Sure, we can imagine ways that nearly every application _can_ be
written in such environments, but in many cases these designs are quite
different from what web developers are used to.

I think there are good reasons for all the design decisions we're making,
but I'm worried we're not looking at the big picture enough.

Re: [whatwg] Storage mutex and cookies can lead to browser deadlock

[Jeremy Orlow]

On Fri, Sep 4, 2009 at 10:59 AM, Jeremy Orlow jor...@chromium.org wrote:

 On Fri, Sep 4, 2009 at 10:48 AM, Michael Nordman micha...@google.comwrote:

  Shared worker access would be a plus.
 Indeed. The lack of access to LocalStorage in 'workers' forces developers
 to use the more difficult database api for all storage needs, and to roll
 their own change event mechanisms (based on postMessage). Thats a bunch of
 busy work if a name/value pair schema is all your app really needs.


 For the record, all the developers I've talked to about the current state
 of AppCache+storage+workers have been VERY disheartened.  IE and Firefox
 have no intentions of supporting WebDatabase any time soon.  localStorage is
 not available from workers.  AppCache requires apps to be 100% client based
 (the server needs to server static pages and the logic must be in JS) if you
 have any personalization/authentication.  Workers are only accessible via
 message passing.  Sure, we can imagine ways that nearly every application
 _can_ be written in such environments, but in many cases these designs are
 quite different from what web developers are used to.

 I think there are good reasons for all the design decisions we're making,
 but I'm worried we're not looking at the big picture enough.


On a related note, it might even be possible that saving developers from
explicitly having to think about _any_ concurrency is actually hurting them
more than it's helping.  And I'm not just talking about the short term,
here.

Re: [whatwg] Storage mutex and cookies can lead to browser deadlock

[Darin Fisher]

On Tue, Sep 1, 2009 at 4:31 PM, Jeremy Orlow jor...@chromium.org wrote:

 On Wed, Aug 26, 2009 at 3:24 PM, Jeremy Orlow jor...@chromium.org wrote:

 On Wed, Aug 26, 2009 at 3:05 PM, Robert O'Callahan 
 rob...@ocallahan.orgwrote:

 On Wed, Aug 26, 2009 at 2:54 PM, Jeremy Orlow jor...@chromium.orgwrote:

 Is there any data (or any way to collect the data) on how much of the
 web IE and Chrome's current behavior has broken?  Given that there hasn't
 been panic in the streets, I'm assuming approximately 0%?


 We previously had a lengthy discussion about this.

 If a site has a cookie race that causes a problem in IE/Chrome one in
 every 10,000 page loads, are you comfortable with that?


 I'm much more comfortable with that than the cost of a global mutex that
 all cookies and LocalStorage share.  There are other ways to come about this
 problem (like developer tools).

 I'm pretty sure Chromium has no intention of implementing a global storage
 mutex and putting all cookie access under it.  Has anyone heard anything
 (either way) from Microsoft?  Are there any browsers moving to a
 multi-event-loop (be it multi-threaded or multi-process) based model that
 intend to implement this?  If not, then it would seem like the spec is not
 grounded in reality.


 Does the silence mean that no one has any intention of implementing this?
  If so, maybe we should resign ourselves to a break in the single threaded
 illusion for cookies.  This doesn't seem too outlandish considering that web
 servers working with cookies will never have such a guarantee and given that
 we have no evidence of widespread breakage with IE 8 and Chrome.


IE 6 -- it is also multi process.  you can poke at wininet from any
application and change the cookies for IE.

-darin



 If we were to get rid of the storage mutex for cookie manipulation (as I
 believe we should) maybe we should re-examine it for local storage.  At a
 minimum, it could be implemented as a per-origin mutex.  But I feel strongly
 we should go further.  Why not have an asynchronous mechanism for atomic
 updates?  For example, if I wanted to write an ATM application, I would have
 to do the following:

 var accountDelta = /* something */;
 window.localStorage.executeAtomic(function() {
 localStorage.accountBalance = localStorage.accountBalance +
 accountDelta;
 }

 Alternatively, we could make it so that each statement is atomic, but that
 you have to use such a mechanism for anything more complicated. For example:

 localStorage.accountBalance = localStorage.accountBalance + accountDelta;
  // It's atomic, so no worries!
 var balance = localStorage.accountBalance;  /* Oh no  This isn't safe
 since it's implemented via multiple statements... */
 localStorage.accountBalance = balance + accountDelta;  /* we should
 have used localStorage.executeAtomic! */

 Such ideas would definitely lighten lock contention and possibly eliminate
 the need for yieldForStorageUpdates (formerly getStorageUpdates).  Another
 major bonus is that it'd allow us to expose localStorage to workers again,
 which is one of the top complaints I've gotten when talking to web
 developers about localStorage.

 I know this is radical stuff, but the way things are speced currently just
 are not practical.

 J

Re: [whatwg] Storage mutex and cookies can lead to browser deadlock

[Drew Wilson]

When we had this discussion last, there was significant pushback on this -
the argument was basically we have no evidence that cookie-based race
conditions *aren't* causing sporadic breakages, which is true. It's
inherently difficult to measure.
As an aside, I'll note that the majority of pushback came from developers of
platforms that were inherently single-threaded, and so enforcing
synchronicity had no impact on the performance of their platforms. It's easy
to be a purist when there's no cost.

Now that more browsers are moving to multi-process architectures and will
soon be faced with having to reduce the performance of their platforms to
enforce cookie coherence, I wonder if people's attitudes have changed. I,
too, would be interested in hearing if the folks working on multi-process
firefox are planning to implement this piece of the spec.

-atw

On Wed, Sep 2, 2009 at 9:55 AM, Darin Fisher da...@chromium.org wrote:



 On Tue, Sep 1, 2009 at 4:31 PM, Jeremy Orlow jor...@chromium.org wrote:

 On Wed, Aug 26, 2009 at 3:24 PM, Jeremy Orlow jor...@chromium.orgwrote:

 On Wed, Aug 26, 2009 at 3:05 PM, Robert O'Callahan rob...@ocallahan.org
  wrote:

 On Wed, Aug 26, 2009 at 2:54 PM, Jeremy Orlow jor...@chromium.orgwrote:

 Is there any data (or any way to collect the data) on how much of the
 web IE and Chrome's current behavior has broken?  Given that there hasn't
 been panic in the streets, I'm assuming approximately 0%?


 We previously had a lengthy discussion about this.

 If a site has a cookie race that causes a problem in IE/Chrome one in
 every 10,000 page loads, are you comfortable with that?


 I'm much more comfortable with that than the cost of a global mutex that
 all cookies and LocalStorage share.  There are other ways to come about this
 problem (like developer tools).

 I'm pretty sure Chromium has no intention of implementing a global
 storage mutex and putting all cookie access under it.  Has anyone heard
 anything (either way) from Microsoft?  Are there any browsers moving to a
 multi-event-loop (be it multi-threaded or multi-process) based model that
 intend to implement this?  If not, then it would seem like the spec is not
 grounded in reality.


 Does the silence mean that no one has any intention of implementing this?
  If so, maybe we should resign ourselves to a break in the single threaded
 illusion for cookies.  This doesn't seem too outlandish considering that web
 servers working with cookies will never have such a guarantee and given that
 we have no evidence of widespread breakage with IE 8 and Chrome.


 IE 6 -- it is also multi process.  you can poke at wininet from any
 application and change the cookies for IE.

 -darin



 If we were to get rid of the storage mutex for cookie manipulation (as I
 believe we should) maybe we should re-examine it for local storage.  At a
 minimum, it could be implemented as a per-origin mutex.  But I feel strongly
 we should go further.  Why not have an asynchronous mechanism for atomic
 updates?  For example, if I wanted to write an ATM application, I would have
 to do the following:

 var accountDelta = /* something */;
 window.localStorage.executeAtomic(function() {
 localStorage.accountBalance = localStorage.accountBalance +
 accountDelta;
 }

 Alternatively, we could make it so that each statement is atomic, but that
 you have to use such a mechanism for anything more complicated. For example:

 localStorage.accountBalance = localStorage.accountBalance + accountDelta;
  // It's atomic, so no worries!
 var balance = localStorage.accountBalance;  /* Oh no  This isn't safe
 since it's implemented via multiple statements... */
 localStorage.accountBalance = balance + accountDelta;  /* we should
 have used localStorage.executeAtomic! */

 Such ideas would definitely lighten lock contention and possibly eliminate
 the need for yieldForStorageUpdates (formerly getStorageUpdates).  Another
 major bonus is that it'd allow us to expose localStorage to workers again,
 which is one of the top complaints I've gotten when talking to web
 developers about localStorage.

 I know this is radical stuff, but the way things are speced currently just
 are not practical.

 J

Re: [whatwg] Storage mutex and cookies can lead to browser deadlock

[Jeremy Orlow]

On Wed, Aug 26, 2009 at 3:24 PM, Jeremy Orlow jor...@chromium.org wrote:

 On Wed, Aug 26, 2009 at 3:05 PM, Robert O'Callahan 
 rob...@ocallahan.orgwrote:

 On Wed, Aug 26, 2009 at 2:54 PM, Jeremy Orlow jor...@chromium.orgwrote:

 Is there any data (or any way to collect the data) on how much of the web
 IE and Chrome's current behavior has broken?  Given that there hasn't been
 panic in the streets, I'm assuming approximately 0%?


 We previously had a lengthy discussion about this.

 If a site has a cookie race that causes a problem in IE/Chrome one in
 every 10,000 page loads, are you comfortable with that?


 I'm much more comfortable with that than the cost of a global mutex that
 all cookies and LocalStorage share.  There are other ways to come about this
 problem (like developer tools).

 I'm pretty sure Chromium has no intention of implementing a global storage
 mutex and putting all cookie access under it.  Has anyone heard anything
 (either way) from Microsoft?  Are there any browsers moving to a
 multi-event-loop (be it multi-threaded or multi-process) based model that
 intend to implement this?  If not, then it would seem like the spec is not
 grounded in reality.


Does the silence mean that no one has any intention of implementing this?
 If so, maybe we should resign ourselves to a break in the single threaded
illusion for cookies.  This doesn't seem too outlandish considering that web
servers working with cookies will never have such a guarantee and given that
we have no evidence of widespread breakage with IE 8 and Chrome.

If we were to get rid of the storage mutex for cookie manipulation (as I
believe we should) maybe we should re-examine it for local storage.  At a
minimum, it could be implemented as a per-origin mutex.  But I feel strongly
we should go further.  Why not have an asynchronous mechanism for atomic
updates?  For example, if I wanted to write an ATM application, I would have
to do the following:

var accountDelta = /* something */;
window.localStorage.executeAtomic(function() {
localStorage.accountBalance = localStorage.accountBalance +
accountDelta;
}

Alternatively, we could make it so that each statement is atomic, but that
you have to use such a mechanism for anything more complicated. For example:

localStorage.accountBalance = localStorage.accountBalance + accountDelta;
 // It's atomic, so no worries!
var balance = localStorage.accountBalance;  /* Oh no  This isn't safe
since it's implemented via multiple statements... */
localStorage.accountBalance = balance + accountDelta;  /* we should have
used localStorage.executeAtomic! */

Such ideas would definitely lighten lock contention and possibly eliminate
the need for yieldForStorageUpdates (formerly getStorageUpdates).  Another
major bonus is that it'd allow us to expose localStorage to workers again,
which is one of the top complaints I've gotten when talking to web
developers about localStorage.

I know this is radical stuff, but the way things are speced currently just
are not practical.

J

Re: [whatwg] Storage mutex and cookies can lead to browser deadlock

[Robert O'Callahan]

On Wed, Sep 2, 2009 at 11:31 AM, Jeremy Orlow jor...@chromium.org wrote:

 Does the silence mean that no one has any intention of implementing this?


I'm silent because I'm not currently working on this stuff so I can't say
what our plans are. But I'll be upset if I find out our plans are to break
the single-threaded model.

Alternatively, we could make it so that each statement is atomic, but that
 you have to use such a mechanism for anything more complicated. For example:

 localStorage.accountBalance = localStorage.accountBalance + accountDelta;
  // It's atomic, so no worries!
 var balance = localStorage.accountBalance;  /* Oh no  This isn't safe
 since it's implemented via multiple statements... */
 localStorage.accountBalance = balance + accountDelta;  /* we should
 have used localStorage.executeAtomic! */

 Such ideas would definitely lighten lock contention and possibly eliminate
 the need for yieldForStorageUpdates (formerly getStorageUpdates).  Another
 major bonus is that it'd allow us to expose localStorage to workers again,
 which is one of the top complaints I've gotten when talking to web
 developers about localStorage.


Making what should be (and always have been) meaning-preserving
transformations like splitting one JS statement into two no longer
meaning-preserving --- and in a non-testable way, to boot --- seems like a
pretty bad idea to me.


I know this is radical stuff, but the way things are speced currently just
 are not practical.


I think that expecting Web developers to reason about concurrency and race
conditions is not practical.

Rob
-- 
He was pierced for our transgressions, he was crushed for our iniquities;
the punishment that brought us peace was upon him, and by his wounds we are
healed. We all, like sheep, have gone astray, each of us has turned to his
own way; and the LORD has laid on him the iniquity of us all. [Isaiah
53:5-6]

Re: [whatwg] Storage mutex feedback

[Mike Wilson]

Jonas Sicking wrote:
 On Sat, Aug 29, 2009 at 10:06 PM, Ian Hicksoni...@hixie.ch wrote:
  Upon further consideration I've renamed getStorageUpdates() to
  yieldForStorageUpdates().
 
 I really liked Darin's (?) suggestion of allowStorageUpdates as that
 seems to exactly describe the intended use of the function. We no
 longer prevent other page from updating the storage.

Quite nice. One little nit is that it could be
interpreted as allowing oneself's updates, while
it is really about allowing other's updates.
allowOtherStorageUpdates? (not so nice, but... ;-)

Is there a preference for having the name indicate
that we are finishing our own transaction, ie
  myStorageUpdatesAreNowComplete
  endStorageUpdates
  finishStorageUpdates
  commitStorageUpdates
  flushStorageUpdates
  saveStorageUpdates
or should it indicate that others are now welcome
to do their stuff, such as in:
  otherPagesStorageUpdatesAreNowWelcome
  allow(Other)StorageUpdates
  enable(Other)StorageUpdates
?

Best regards
Mike

Re: [whatwg] Storage mutex

[Michael Nordman]

'Commit' implies that it hasn't been committed yet, which is not true. This
will be misleading to those developers that do understand the locking
semantics (and transactional semantics) and are looking to defeat them. With
the 'commit' naming, we'd be doing a diservice to exactly the sort of
developers this API is intended for, all to avoid exposing the fact that
locks actually are involved to the more casual developer.

Back to a comment long ago in this thread...

 Authors would be confused that there's no aquireLock() API.

Regardless of what you call it, authors are going to be confused by this
API. This is because the locking semantics are otherwise hidden, the lock
acquisition is implicit. This API provides an explicit means of unlocking.
No matter how you slice it... thats what's going on.

Without an understanding of the locking semantics, its going to be
confusing. Hiding those semantics is the source of the confusion to start
with. Not naming this this releaseStorageLock only adds to the confusion.



On Sat, Aug 29, 2009 at 6:10 PM, Mike Shaver mike.sha...@gmail.com wrote:

 On Fri, Aug 28, 2009 at 3:36 PM, Jeremy Orlowjor...@chromium.org wrote:
  Can a plugin ever call into a script while a script is running besides
 when
  the script is making a synchronous call to the plugin?  If so, that
 worries
  me since it'd be a way for the script to lose its lock at _any_ time.

 Does the Chromium out-of-process plugin model take steps to prevent
 it?  I had understood that it let such things race freely, but maybe
 that was just at the NPAPI level and there's some other interlocking
 protocol before script can be invoked.

 Mike

[whatwg] Storage mutex feedback

[Ian Hickson]

On Tue, 18 Aug 2009, Jeremy Orlow wrote:

 First of all, I was wondering why all user prompts are specified as 
 must release the storage mutex ( 
 http://dev.w3.org/html5/spec/Overview.html#user-prompts).

This is because otherwise, if the script has the storage mutex, the user 
can't open any new pages in the entire browser app from any sites that set 
cookies, until he has dismissed the alert. This would prevent him from, 
for instance, going to a URL listed in the alert to check something before 
dismissing the alert.


 It seems like there might be some UA implementation specific ways this 
 could deadlock

I'm not aware of any implementation-specific issues with the storage mutex 
that are black-box detectable.


 I don't see why the language should be must.

The idea is to make sure all the browsers do the same thing.


 It's also worth noting that Chromium is probably going to need to drop 
 the storage mutex for most if not all plugin related calls due to 
 deadlock conditions.

This is required already by the spec.


 Given that different UAs are probably going to have other scenarios 
 where they have to drop the lock (some of them may even be purely 
 implementational issues), should we add some way for us to notify 
 scripts the lock was dropped?

Could you list some examples of such cases? I'm not aware of any such 
cases at the moment.

I'd rather not expose the lock to authors at all, if we can avoid doing 
so.


 Lastly, is navigator.getStorageUpdates() the right name for the function 
 that drops the lock?  Why was it changed from navigator.releaseLock()?  

What lock? Authors won't know anything about the storage mutex, so we 
have to describe what the method will appear to do from their perspective.


On Sat, 22 Aug 2009, Jeremy Orlow wrote:
 
 [...] shouldn't it be left up to the UAs what to do?

Generally speaking we want to leave as little black-box detectable 
behaviour as possible undefined.


On Tue, 25 Aug 2009, Jeremy Orlow wrote:
 
 I guess it would depend on the use.  Let's say a library/framework 
 dependeds on the lock being held but does a callback (that might do 
 something that causes the lock to be dropped).  It could check the 
 counter and raise an exception.  It could also re-start processing if 
 that were an acceptable answer (but by having the counter, it would only 
 do so when necessary).  I think it'll be very application specific 
 _what_ you do when you catch such an error, but I do think it'll be 
 valuable to developers.

If you can come up with a viable scenario that would need such a feature, 
then it's worth considering. However, so far it's been mostly a 
theoretical concern. I'd rather not add anything to do with this feature 
if we can avoid it, since we'd like to find a way to drop the whole thing 
in the future, and we want to allow implementors to do optimisations that 
might break assumptions we might make in any API we expose.


On Tue, 25 Aug 2009, Drew Wilson wrote:
 
 The problem with leaving this up to the UA is it becomes a point of 
 incompatibility - on one browser, it's safe to put up an alert, on 
 another it isn't. So if applications have to fall back to the LCD 
 behavior, then we might as well codify it in the spec, which we have :)

Indeed.


On Tue, 25 Aug 2009, Jeremy Orlow wrote:

 What about yieldStorageMutex?  Yield is enough different from unlock 
 that I don't think it'll leave developers looking for the lock function.  
 Yield fits pretty well since this is essentially cooperative 
 multi-tasking.

yieldForStorageUpdates() maybe?


 StorageMutex is good because that's what its actually affecting.

Yeah but that's what we want to hide from the author. Only people who read 
the spec with a fine tooth comb are going to know anything about the 
storage mutex.


On Wed, 26 Aug 2009, Jens Alfke wrote:
 
 (At a higher level, though, I'm still uncomfortable with semantics that 
 allow a poorly-written worker thread to indefinitely lock up JS code 
 running in web pages.)

Workers can't get the storage mutex.


On Fri, 28 Aug 2009, Kevin Benson wrote:
 
 How about:
 
 commitStorageUpdates
 
 ... since a new transactor cannot write to storage until a commit point 
 is reached by the current transactor finishing up and releasing the 
 lock.

That could work too.


Upon further consideration I've renamed getStorageUpdates() to 
yieldForStorageUpdates().


On Fri, 28 Aug 2009, Jeremy Orlow wrote:
 
 One question about this, actually.  I'm a bit troubled by the following 
 language:
 
 Whenever a script calls into a plugin, and whenever a plugin calls into 
 a script, the user agent must release the storage mutex.
 
 Can a plugin ever call into a script while a script is running besides 
 when the script is making a synchronous call to the plugin?  If so, that 
 worries me since it'd be a way for the script to lose its lock at _any_ 
 time.

The HTML5 spec doesn't say what plugins can do, so yes, theoretically a 
plugin could call in 

Re: [whatwg] Storage mutex

[Mike Shaver]

On Fri, Aug 28, 2009 at 3:36 PM, Jeremy Orlowjor...@chromium.org wrote:
 Can a plugin ever call into a script while a script is running besides when
 the script is making a synchronous call to the plugin?  If so, that worries
 me since it'd be a way for the script to lose its lock at _any_ time.

Does the Chromium out-of-process plugin model take steps to prevent
it?  I had understood that it let such things race freely, but maybe
that was just at the NPAPI level and there's some other interlocking
protocol before script can be invoked.

Mike

Re: [whatwg] Storage mutex

[Mike Shaver]

On Tue, Aug 25, 2009 at 10:36 PM, Jeremy Orlowjor...@chromium.org wrote:
 On Tue, Aug 25, 2009 at 10:28 PM, Robert O'Callahan rob...@ocallahan.org
 wrote:

 On Tue, Aug 25, 2009 at 11:51 AM, Jeremy Orlow jor...@chromium.org
 wrote:

 To me, getStorageUpdates seems to imply that updates have already
 happened and we're working with an old version of the data.  I think many
 developers will be quite shocked that getStorageUpdates _enables_ others to
 update storage.  In other words, 'get' seems to imply that you're consuming
 state that's happening anyway, not affecting behavior.

 fetchStorageUpdates?

 fetch has the same problem.  If we want to keep the StorageUpdates suffix,
 I'd go with something like allowStorageUpdates.  But, no matter what, it
 just doesn't seem very clear that you're actively allowing another thread to
 use the storage mutex.
 What about yieldStorageMutex?  Yield is enough different from unlock that I
 don't think it'll leave developers looking for the lock function.  Yield
 fits pretty well since this is essentially cooperative multi-tasking.
  StorageMutex is good because that's what its actually affecting.

processPendingStorageUpdates? processStorageUpdates?

FWIW I would expect getStorageUpdates to return some storage updates
to the caller, like a getter.  I would expect fetchStorageUpdates to
do the same thing, except maybe involving something over the network,
and I would be a little puzzled about why it wasn't just
getStorageUpdates.

Mike

Re: [whatwg] Storage mutex

[Kevin Benson]

On Sun, Aug 23, 2009 at 1:22 AM, Jeremy Orlowjor...@chromium.org wrote:
 On Tue, Aug 18, 2009 at 4:26 PM, Jeremy Orlow jor...@chromium.org wrote:

 Lastly, is navigator.getStorageUpdates() the right name for the function
 that drops the lock?  Why was it changed from navigator.releaseLock()?  I
 assume we're trying to avoid the word lock, but the reason why you'd need
 to call a function to get updates is not clear without understanding the
 concept of a lock...so what's the point of making this so cryptic?

 Authors would be confused that there's no aquireLock() API.

 Good point.
 But getStorageUpdates is still not the right name for it.  The only way that
 there'd be any updates to get is if, when you call the function, someone
 else takes the lock and makes some updates.  Maybe it should be yieldStorage
 (or yieldStorageMutex)?  In other words, maybe the name should imply that
 you're allowing concurrent updates to happen?

How about:

commitStorageUpdates

... since a new transactor cannot write to storage until a commit
point is reached by the current transactor finishing up and releasing
the lock.

-- 
-- 
   --
   --
   ô¿ô¬
K e V i N
   /¯\

Re: [whatwg] Storage mutex

[Jeremy Orlow]

On Fri, Aug 28, 2009 at 4:05 AM, Kevin Benson kevin.m.ben...@gmail.comwrote:

 On Sun, Aug 23, 2009 at 1:22 AM, Jeremy Orlowjor...@chromium.org wrote:
  On Tue, Aug 18, 2009 at 4:26 PM, Jeremy Orlow jor...@chromium.org
  wrote:
 
  Lastly, is navigator.getStorageUpdates() the right name for the
 function
  that drops the lock?  Why was it changed from navigator.releaseLock()?
  I
  assume we're trying to avoid the word lock, but the reason why you'd
 need
  to call a function to get updates is not clear without understanding
 the
  concept of a lock...so what's the point of making this so cryptic?
 
  Authors would be confused that there's no aquireLock() API.
 
  Good point.
  But getStorageUpdates is still not the right name for it.  The only way
 that
  there'd be any updates to get is if, when you call the function, someone
  else takes the lock and makes some updates.  Maybe it should be
 yieldStorage
  (or yieldStorageMutex)?  In other words, maybe the name should imply that
  you're allowing concurrent updates to happen?

 How about:

 commitStorageUpdates

 ... since a new transactor cannot write to storage until a commit
 point is reached by the current transactor finishing up and releasing
 the lock.


Both commit and allow seem like good alternatives.  I still like
yieldStorageMutex, but I understand using the word Mutex might seem too
scary.  Now I'll just pray that Ian also finds one of these names to be
better than getStorageUpdates.  :-)


On Sun, Aug 23, 2009 at 11:33 PM, Robert O'Callahan rob...@ocallahan.org
 wrote:

 On Sat, Aug 22, 2009 at 10:22 PM, Jeremy Orlow jor...@chromium.org
  wrote:

 On Sat, Aug 22, 2009 at 5:54 AM, Robert O'Callahan rob...@ocallahan.org
  wrote:

 On Wed, Aug 19, 2009 at 11:26 AM, Jeremy Orlow jor...@chromium.org
  wrote:

 First of all, I was wondering why all user prompts are specified as
 must release the storage mutex (
 http://dev.w3.org/html5/spec/Overview.html#user-prompts).  Should this
 really say must instead of may?  IIRC (I couldn't find the original
 thread, unfortunately) this was added because of deadlock concerns.  It
 seems like there might be some UA implementation specific ways this could
 deadlock and there is the question of whether we'd want an alert() while
 holding the lock to block other execution requiring the lock, but I don't
 see why the language should be must.  For Chromium, I don't think we'll
 need to release the lock for any of these, unless there's some
 deadlock scenario I'm missing here.


 So if one page grabs the lock and then does an alert(), and another page
 in the same domain tries to get the lock, you're going to let the latter
 page hang until the user dismisses the alert in the first page?


 Yes.  And I agree this is sub-optimal, but shouldn't it be left up to the
 UAs what to do?  I feel like this is somewhat of an odd case to begin with
 since alerts lock up most (all?) browsers to a varying degrees even without
 using localStorage.


 That behaviour sounds worse than what Firefox currently does, where an
 alert disables input to all tabs in the window (which is already pretty
 bad), because it willl make applications in visually unrelated tabs and
 windows hang.


Apparently (based on the reply from Jonas on the thread I split this into)
it sounds like this behavior is actually a bug.  Either way, it breaks
run-to-completion and (as far as I can tell) goes against the spec (but I'm
fuzzy on this part).

So I guess the question is why we should make a special case for
LocalStorage regarding modal dialog boxes?


On Sat, Aug 22, 2009 at 10:22 PM, Jeremy Orlow jor...@chromium.org wrote:

 On Tue, Aug 18, 2009 at 4:26 PM, Jeremy Orlow jor...@chromium.org wrote:

 It's also worth noting that Chromium is probably going to need to drop the
 storage mutex for most if not all plugin related calls due to deadlock
 conditions.  If there were some place to mention this as a may type thing,
 it'd be good, but I realize it's probably out of scope for HTML 5.


 Oops.  The spec already does specify this behavior:
 http://dev.w3.org/html5/spec/Overview.html#storage-mutex


One question about this, actually.  I'm a bit troubled by the following
language:

Whenever a scripthttp://dev.w3.org/html5/spec/Overview.html#concept-script
calls
into a plugin http://dev.w3.org/html5/spec/Overview.html#plugin, and
whenever a plugin http://dev.w3.org/html5/spec/Overview.html#plugin calls
into a script http://dev.w3.org/html5/spec/Overview.html#concept-script,
the user agent must release the storage
mutexhttp://dev.w3.org/html5/spec/Overview.html#storage-mutex
.

Can a plugin ever call into a script while a script is running besides when
the script is making a synchronous call to the plugin?  If so, that worries
me since it'd be a way for the script to lose its lock at _any_ time.

Re: [whatwg] Storage mutex

[Darin Fisher]

On Sun, Aug 23, 2009 at 11:33 PM, Robert O'Callahan rob...@ocallahan.orgwrote:

 On Sat, Aug 22, 2009 at 10:22 PM, Jeremy Orlow jor...@chromium.orgwrote:

 On Sat, Aug 22, 2009 at 5:54 AM, Robert O'Callahan 
 rob...@ocallahan.orgwrote:

 On Wed, Aug 19, 2009 at 11:26 AM, Jeremy Orlow jor...@chromium.orgwrote:

 First of all, I was wondering why all user prompts are specified as
 must release the storage mutex (
 http://dev.w3.org/html5/spec/Overview.html#user-prompts).  Should this
 really say must instead of may?  IIRC (I couldn't find the original
 thread, unfortunately) this was added because of deadlock concerns.  It
 seems like there might be some UA implementation specific ways this could
 deadlock and there is the question of whether we'd want an alert() while
 holding the lock to block other execution requiring the lock, but I don't
 see why the language should be must.  For Chromium, I don't think we'll
 need to release the lock for any of these, unless there's some
 deadlock scenario I'm missing here.


 So if one page grabs the lock and then does an alert(), and another page
 in the same domain tries to get the lock, you're going to let the latter
 page hang until the user dismisses the alert in the first page?


 Yes.  And I agree this is sub-optimal, but shouldn't it be left up to the
 UAs what to do?  I feel like this is somewhat of an odd case to begin with
 since alerts lock up most (all?) browsers to a varying degrees even without
 using localStorage.


 That behaviour sounds worse than what Firefox currently does, where an
 alert disables input to all tabs in the window (which is already pretty
 bad), because it willl make applications in visually unrelated tabs and
 windows hang.


You can have script connections that span multiple tabs in multiple windows,
so in order to preserve the run-to-completion semantics of JavaScript, it is
important that window.{alert,confirm,prompt,showModalDialog} be modal across
all windows in the browser.  This is why those APIs suck rocks, and we
should never create APIs like them again.





  Given that different UAs are probably going to have
 other scenarios where they have to drop the lock (some of them may even be
 purely implementational issues), should we add some way for us to notify
 scripts the lock was dropped?  A normal event isn't going to be of much 
 use,
 since it'll fire after the scripts execution ends (so the lock would have
 been dropped by then anyway).  A boolean doesn't seem super useful, but 
 it's
 better than nothing and could help debugging.  Maybe fire an exception?  
 Are
 there other options?


 A generation counter might be useful.


 Ooo, I like that idea.  When would the counter increment?  It'd be nice if
 it didn't increment if the page did something synchronous but no one else
 took the lock in the mean time.


 Defining no-one else may be difficult. I haven't thought this through, to
 be honest, but I think you could update the counter every time the storage
 mutex is released and the shared state was modified since the storage mutex
 was acquired. Reading the counter would acquire the storage mutex. You'd
 basically write

 var counter = window.storageMutexGenerationCounter;
 ... do lots of stuff ...
 if (window.storageMutexGenerationCounter != counter) {
   // abort, or refresh local state, or something
 }

 I'm not sure what you'd do if you discovered an undesired lock-drop,
 though. If you can't write something sensible instead of abort, or
 something, it's not worth doing.


Implementation-wise, the easiest thing to support is a boolean that becomes
true when the lock is release and false when the lock is acquired.  Trying
to update a counter based on modifications to the local storage backend
which may be happening on another thread seems like more effort than it is
worth.

But, what would you call this boolean?  storageMayHaveBeenUpdated? :-P

I'm struggling to find a good use case for this.




  But getStorageUpdates is still not the right name for it.  The only way
 that there'd be any updates to get is if, when you call the function,
 someone else takes the lock and makes some updates.  Maybe it should be
 yieldStorage (or yieldStorageMutex)?  In other words, maybe the name should
 imply that you're allowing concurrent updates to happen?


 I thought that's what getStorageUpdates implied :-).


The getStorageUpdates name seems pretty decent to me when considering it
from the perspective of the caller.  The caller is saying that they are OK
with being able to see changes made to the localStorage by other threads.
 This cleverly avoids the need to talk about locks, which seems like a good
thing.  It is okay for there to be no updates to storage.

-Darin

Re: [whatwg] Storage mutex

[Darin Fisher]

On Sun, Aug 23, 2009 at 11:33 PM, Robert O'Callahan rob...@ocallahan.orgwrote:

 On Sat, Aug 22, 2009 at 10:22 PM, Jeremy Orlow jor...@chromium.orgwrote:

 On Sat, Aug 22, 2009 at 5:54 AM, Robert O'Callahan 
 rob...@ocallahan.orgwrote:

 On Wed, Aug 19, 2009 at 11:26 AM, Jeremy Orlow jor...@chromium.orgwrote:

 First of all, I was wondering why all user prompts are specified as
 must release the storage mutex (
 http://dev.w3.org/html5/spec/Overview.html#user-prompts).  Should this
 really say must instead of may?  IIRC (I couldn't find the original
 thread, unfortunately) this was added because of deadlock concerns.  It
 seems like there might be some UA implementation specific ways this could
 deadlock and there is the question of whether we'd want an alert() while
 holding the lock to block other execution requiring the lock, but I don't
 see why the language should be must.  For Chromium, I don't think we'll
 need to release the lock for any of these, unless there's some
 deadlock scenario I'm missing here.


 So if one page grabs the lock and then does an alert(), and another page
 in the same domain tries to get the lock, you're going to let the latter
 page hang until the user dismisses the alert in the first page?


 Yes.  And I agree this is sub-optimal, but shouldn't it be left up to the
 UAs what to do?  I feel like this is somewhat of an odd case to begin with
 since alerts lock up most (all?) browsers to a varying degrees even without
 using localStorage.


 That behaviour sounds worse than what Firefox currently does, where an
 alert disables input to all tabs in the window (which is already pretty
 bad), because it willl make applications in visually unrelated tabs and
 windows hang.


You can have script connections that span multiple tabs in multiple windows,
so in order to preserve the run-to-completion semantics of JavaScript, it is
important that window.{alert,confirm,prompt,showModalDialog} be modal across
all windows in the browser.  This is why those APIs suck rocks, and we
should never create APIs like them again.




  Given that different UAs are probably going to have
 other scenarios where they have to drop the lock (some of them may even be
 purely implementational issues), should we add some way for us to notify
 scripts the lock was dropped?  A normal event isn't going to be of much 
 use,
 since it'll fire after the scripts execution ends (so the lock would have
 been dropped by then anyway).  A boolean doesn't seem super useful, but 
 it's
 better than nothing and could help debugging.  Maybe fire an exception?  
 Are
 there other options?


 A generation counter might be useful.


 Ooo, I like that idea.  When would the counter increment?  It'd be nice if
 it didn't increment if the page did something synchronous but no one else
 took the lock in the mean time.


 Defining no-one else may be difficult. I haven't thought this through, to
 be honest, but I think you could update the counter every time the storage
 mutex is released and the shared state was modified since the storage mutex
 was acquired. Reading the counter would acquire the storage mutex. You'd
 basically write

 var counter = window.storageMutexGenerationCounter;
 ... do lots of stuff ...
 if (window.storageMutexGenerationCounter != counter) {
   // abort, or refresh local state, or something
 }

 I'm not sure what you'd do if you discovered an undesired lock-drop,
 though. If you can't write something sensible instead of abort, or
 something, it's not worth doing.


Implementation-wise, the easiest thing to support is a boolean that becomes
true when the lock is release and false when the lock is acquired.  Trying
to update a counter based on modifications to the local storage backend
which may be happening on another thread seems like more effort than it is
worth.

But, what would you call this boolean?  storageMayHaveBeenUpdated? :-P

I'm struggling to find a good use case for this.



  But getStorageUpdates is still not the right name for it.  The only way
 that there'd be any updates to get is if, when you call the function,
 someone else takes the lock and makes some updates.  Maybe it should be
 yieldStorage (or yieldStorageMutex)?  In other words, maybe the name should
 imply that you're allowing concurrent updates to happen?


 I thought that's what getStorageUpdates implied :-).


The getStorageUpdates name seems pretty decent to me when considering it
from the perspective of the caller.  The caller is saying that they are OK
with being able to see changes made to the localStorage by other threads.
 This cleverly avoids the need to talk about locks, which seems like a good
thing.  It is okay for there to be no updates to storage.

-Darin

Re: [whatwg] Storage mutex

[Jeremy Orlow]

On Wed, Aug 26, 2009 at 12:51 AM, Darin Fisher da...@chromium.org wrote:

 On Sun, Aug 23, 2009 at 11:33 PM, Robert O'Callahan 
 rob...@ocallahan.orgwrote:

 That behaviour sounds worse than what Firefox currently does, where an
 alert disables input to all tabs in the window (which is already pretty
 bad), because it willl make applications in visually unrelated tabs and
 windows hang.


 You can have script connections that span multiple tabs in multiple
 windows, so in order to preserve the run-to-completion semantics of
 JavaScript, it is important that
 window.{alert,confirm,prompt,showModalDialog} be modal across all windows in
 the browser.  This is why those APIs suck rocks, and we should never create
 APIs like them again.


I don't understand your point here.  Are you saying that the current firefox
behavior is not correct, that releasing the storage lock on these events is
not correct, or something else?

Defining no-one else may be difficult. I haven't thought this through, to
 be honest, but I think you could update the counter every time the storage
 mutex is released and the shared state was modified since the storage mutex
 was acquired. Reading the counter would acquire the storage mutex. You'd
 basically write

 var counter = window.storageMutexGenerationCounter;
 ... do lots of stuff ...
 if (window.storageMutexGenerationCounter != counter) {
   // abort, or refresh local state, or something
 }

 I'm not sure what you'd do if you discovered an undesired lock-drop,
 though. If you can't write something sensible instead of abort, or
 something, it's not worth doing.


 Implementation-wise, the easiest thing to support is a boolean that becomes
 true when the lock is release and false when the lock is acquired.  Trying
 to update a counter based on modifications to the local storage backend
 which may be happening on another thread seems like more effort than it is
 worth.


Such a boolean could be useful, but I disagree with the assertion that the
implementation is significantly more difficult.  I'm pretty sure both would
be trivial in Chromium, for example.


  But, what would you call this boolean?  storageMayHaveBeenUpdated? :-P

 I'm struggling to find a good use case for this.


None of the ones I already listed seemed interesting?  If nothing else, I
would think debugability would be a key one.  If we're going to do something
halfway magical, we should make it possible for developers to know it
happened, right??

The getStorageUpdates name seems pretty decent to me when considering it
 from the perspective of the caller.  The caller is saying that they are OK
 with being able to see changes made to the localStorage by other threads.
  This cleverly avoids the need to talk about locks, which seems like a good
 thing.  It is okay for there to be no updates to storage.


So the use case I've had in my mind that maybe isn't clear is this:

localStorage.getItem/setItem
navigator.getStorageUpdates()
localStorage.getItem/setItem

In other words, no processing or anything between calls.

If the act of calling getStorageUpdates gives the lock to everyone who's
waiting for it before letting the caller get it again, then I guess I can
buy this argument.  I still think yieldStorageMutex (or just about anything
for that matter) would be a much better name, but given that no one's
agreeing with me (on this list, anyhow) I'm just going to drop it.

Re: [whatwg] Storage mutex

[Darin Fisher]

On Wed, Aug 26, 2009 at 1:27 AM, Jeremy Orlow jor...@chromium.org wrote:

 On Wed, Aug 26, 2009 at 12:51 AM, Darin Fisher da...@chromium.org wrote:

 On Sun, Aug 23, 2009 at 11:33 PM, Robert O'Callahan rob...@ocallahan.org
  wrote:

 That behaviour sounds worse than what Firefox currently does, where an
 alert disables input to all tabs in the window (which is already pretty
 bad), because it willl make applications in visually unrelated tabs and
 windows hang.


 You can have script connections that span multiple tabs in multiple
 windows, so in order to preserve the run-to-completion semantics of
 JavaScript, it is important that
 window.{alert,confirm,prompt,showModalDialog} be modal across all windows in
 the browser.  This is why those APIs suck rocks, and we should never create
 APIs like them again.


 I don't understand your point here.  Are you saying that the current
 firefox behavior is not correct, that releasing the storage lock on these
 events is not correct, or something else?


I meant that the current Firefox behavior is technically incorrect.  No one
likes app modal dialogs, but how else can you guarantee run-to-completion
semantics? How else do you prevent other scripts from modifying your state
while you are stuck calling into window.alert().




 Defining no-one else may be difficult. I haven't thought this through, to
 be honest, but I think you could update the counter every time the storage
 mutex is released and the shared state was modified since the storage mutex
 was acquired. Reading the counter would acquire the storage mutex. You'd
 basically write

 var counter = window.storageMutexGenerationCounter;
 ... do lots of stuff ...
 if (window.storageMutexGenerationCounter != counter) {
   // abort, or refresh local state, or something
 }

 I'm not sure what you'd do if you discovered an undesired lock-drop,
 though. If you can't write something sensible instead of abort, or
 something, it's not worth doing.


 Implementation-wise, the easiest thing to support is a boolean that
 becomes true when the lock is release and false when the lock is acquired.
  Trying to update a counter based on modifications to the local storage
 backend which may be happening on another thread seems like more effort than
 it is worth.


 Such a boolean could be useful, but I disagree with the assertion that the
 implementation is significantly more difficult.  I'm pretty sure both would
 be trivial in Chromium, for example.


incrementing a counter on each modification to the database would involve
some broadcasting of notifications to each renderer, or we'd need to store
the counter in shared memory.  either seems unfortunate given the debugging
use case.

if we only record the fact that getStorageUpdates (or equivalent) was
called, then it is just a local boolean in the renderer process.  that seems
substantially simpler to implement without performance penalty.






  But, what would you call this boolean?  storageMayHaveBeenUpdated? :-P

 I'm struggling to find a good use case for this.


 None of the ones I already listed seemed interesting?  If nothing else, I
 would think debugability would be a key one.  If we're going to do something
 halfway magical, we should make it possible for developers to know it
 happened, right??

 The getStorageUpdates name seems pretty decent to me when considering it
 from the perspective of the caller.  The caller is saying that they are OK
 with being able to see changes made to the localStorage by other threads.
  This cleverly avoids the need to talk about locks, which seems like a good
 thing.  It is okay for there to be no updates to storage.


 So the use case I've had in my mind that maybe isn't clear is this:

 localStorage.getItem/setItem
 navigator.getStorageUpdates()
  localStorage.getItem/setItem

 In other words, no processing or anything between calls.

 If the act of calling getStorageUpdates gives the lock to everyone who's
 waiting for it before letting the caller get it again, then I guess I can
 buy this argument.


right, this ^^^ is what i meant.

-darin

Re: [whatwg] Storage mutex

[Darin Fisher]

On Wed, Aug 26, 2009 at 12:49 AM, Darin Fisher da...@google.com wrote:

 On Sun, Aug 23, 2009 at 11:33 PM, Robert O'Callahan 
 rob...@ocallahan.orgwrote:

 On Sat, Aug 22, 2009 at 10:22 PM, Jeremy Orlow jor...@chromium.orgwrote:


  But getStorageUpdates is still not the right name for it.  The only way
 that there'd be any updates to get is if, when you call the function,
 someone else takes the lock and makes some updates.  Maybe it should be
 yieldStorage (or yieldStorageMutex)?  In other words, maybe the name should
 imply that you're allowing concurrent updates to happen?


 I thought that's what getStorageUpdates implied :-).


 The getStorageUpdates name seems pretty decent to me when considering it
 from the perspective of the caller.  The caller is saying that they are OK
 with being able to see changes made to the localStorage by other threads.
  This cleverly avoids the need to talk about locks, which seems like a good
 thing.  It is okay for there to be no updates to storage.

 -Darin



What about allowStorageUpdates?

-Darin

[whatwg] Storage mutex and cookies can lead to browser deadlock

[Jens Alfke]

I know that one of the design issues with worker threads and local  
storage has been how to resolve concurrency issues, and that for this  
reason, in the current spec worker threads can't access local storage.


However, there's a scenario under the current spec that doesn't  
involve local storage, whereby a worker thread can deadlock the  
browser. This is because access to cookies, by workers or the browser  
itself, is also subject to that global mutex.


Consider these steps:
1. A worker thread accesses document.cookie. This causes it to  
acquire the mutex (sec. 3.1.3).
2. The thread now performs some long-lasting operation without  
exiting. In the simplest case it just goes into an infinite loop.

3. Meanwhile, the user loads a new web page in the browser.
4. The resulting HTTP response contains a Cookie: header. The spec  
requires that the browser's loader temporarily acquire the mutex while  
updating the cookie (sec. 2.6, item 4).
5. The page-load blocks indefinitely because the worker thread still  
has the mutex and never lets go of it.


The result is that the browser becomes incapable of loading any web  
pages that use cookies. Assuming the worker thread never exits, the  
only way to recover from this is to quit the browser. A worker thread  
like this could very easily be created by a malicious website,  
resulting in a DoS attack on the browser. And of course, a merely  
poorly-written script could create the same effect without intending to.


I honestly can't think of any safe way of tweaking the semantics of  
the existing 'document.cookie' API to make it transactional. :(


Has anyone implemented this portion of the spec yet?

—Jens

Re: [whatwg] Storage mutex and cookies can lead to browser deadlock

[Drew Wilson]

My recollection is that we prohibit worker access to cookies for exactly
this reason (WorkerGlobalScope does not expose a cookies attribute).
-atw

On Wed, Aug 26, 2009 at 2:05 PM, Jens Alfke s...@google.com wrote:

 I know that one of the design issues with worker threads and local storage
 has been how to resolve concurrency issues, and that for this reason, in the
 current spec worker threads can't access local storage.

 However, there's a scenario under the current spec that *doesn't* involve
 local storage, whereby a worker thread can deadlock the browser. This is
 because access to cookies, by workers or the browser itself, is also subject
 to that global mutex.

 Consider these steps:
 1. A worker thread accesses document.cookie. This causes it to acquire
 the mutex (sec. 3.1.3).
 2. The thread now performs some long-lasting operation without exiting. In
 the simplest case it just goes into an infinite loop.
 3. Meanwhile, the user loads a new web page in the browser.
 4. The resulting HTTP response contains a Cookie: header. The spec requires
 that the browser's loader temporarily acquire the mutex while updating the
 cookie (sec. 2.6, item 4).
 5. The page-load blocks indefinitely because the worker thread still has
 the mutex and never lets go of it.

 The result is that the browser becomes incapable of loading any web pages
 that use cookies. Assuming the worker thread never exits, the only way to
 recover from this is to quit the browser. A worker thread like this could
 very easily be created by a malicious website, resulting in a DoS attack on
 the browser. And of course, a merely poorly-written script could create the
 same effect without intending to.

 I honestly can't think of any safe way of tweaking the semantics of the
 existing 'document.cookie' API to make it transactional. :(

 Has anyone implemented this portion of the spec yet?

 —Jens

Re: [whatwg] Storage mutex and cookies can lead to browser deadlock

[Jens Alfke]

On Aug 26, 2009, at 2:11 PM, Drew Wilson wrote:

My recollection is that we prohibit worker access to cookies for  
exactly this reason (WorkerGlobalScope does not expose a cookies  
attribute).


Looks like you're right; section 5 of the Web Workers spec says:
The DOM APIs (Node objects, Document objects, etc) are not available  
to workers in this version of this specification.


and there's no defined way to access cookies except through Document.  
Crisis averted.


(If the spec does get modified to allow local-storage access from  
worker threads, though, this same problem will arise, since they use  
the same lock.)


—Jens

Re: [whatwg] Storage mutex and cookies can lead to browser deadlock

[Drew Wilson]

We discussed this in more detail here:
http://www.mail-archive.com/whatwg@lists.whatwg.org/msg13799.html

At the time, I suggested not protecting cookies with a mutex (allow
asynchronous access - the current behavior on IE and Chrome), which made the
monocles pop out of everyone's eyes :)

-atw

On Wed, Aug 26, 2009 at 2:21 PM, Jens Alfke s...@google.com wrote:


 On Aug 26, 2009, at 2:11 PM, Drew Wilson wrote:

  My recollection is that we prohibit worker access to cookies for exactly
 this reason (WorkerGlobalScope does not expose a cookies attribute).


 Looks like you're right; section 5 of the Web Workers spec says:

 The DOM APIs (Node objects, Document objects, etc) are not available to
 workers in this version of this specification.

  and there's no defined way to access cookies except through Document.
 Crisis averted.

 (If the spec does get modified to allow local-storage access from worker
 threads, though, this same problem will arise, since they use the same
 lock.)

 —Jens

Re: [whatwg] Storage mutex and cookies can lead to browser deadlock

[Jeremy Orlow]

Is there any data (or any way to collect the data) on how much of the web IE
and Chrome's current behavior has broken?  Given that there hasn't been
panic in the streets, I'm assuming approximately 0%?

Given that web developers have never had any guarantees for cookie
access/setting in terms of network communications and given that (it sounds
like) it won't break the web, maybe we should ditch the idea of locking
during cookie access/setting?

On Wed, Aug 26, 2009 at 2:42 PM, Drew Wilson atwil...@google.com wrote:

 We discussed this in more detail here:
 http://www.mail-archive.com/whatwg@lists.whatwg.org/msg13799.html

 At the time, I suggested not protecting cookies with a mutex (allow
 asynchronous access - the current behavior on IE and Chrome), which made the
 monocles pop out of everyone's eyes :)

 -atw


 On Wed, Aug 26, 2009 at 2:21 PM, Jens Alfke s...@google.com wrote:


 On Aug 26, 2009, at 2:11 PM, Drew Wilson wrote:

  My recollection is that we prohibit worker access to cookies for exactly
 this reason (WorkerGlobalScope does not expose a cookies attribute).


 Looks like you're right; section 5 of the Web Workers spec says:

 The DOM APIs (Node objects, Document objects, etc) are not available to
 workers in this version of this specification.

  and there's no defined way to access cookies except through Document.
 Crisis averted.

 (If the spec does get modified to allow local-storage access from worker
 threads, though, this same problem will arise, since they use the same
 lock.)

 —Jens

Re: [whatwg] Storage mutex and cookies can lead to browser deadlock

[Jeremy Orlow]

On Wed, Aug 26, 2009 at 3:05 PM, Robert O'Callahan rob...@ocallahan.orgwrote:

 On Wed, Aug 26, 2009 at 2:54 PM, Jeremy Orlow jor...@chromium.org wrote:

 Is there any data (or any way to collect the data) on how much of the web
 IE and Chrome's current behavior has broken?  Given that there hasn't been
 panic in the streets, I'm assuming approximately 0%?


 We previously had a lengthy discussion about this.

 If a site has a cookie race that causes a problem in IE/Chrome one in every
 10,000 page loads, are you comfortable with that?


I'm much more comfortable with that than the cost of a global mutex that all
cookies and LocalStorage share.  There are other ways to come about this
problem (like developer tools).

I'm pretty sure Chromium has no intention of implementing a global storage
mutex and putting all cookie access under it.  Has anyone heard anything
(either way) from Microsoft?  Are there any browsers moving to a
multi-event-loop (be it multi-threaded or multi-process) based model that
intend to implement this?  If not, then it would seem like the spec is not
grounded in reality.

Re: [whatwg] Storage mutex

[Jeremy Orlow]

On Sun, Aug 23, 2009 at 11:33 PM, Robert O'Callahan rob...@ocallahan.orgwrote:

 On Sat, Aug 22, 2009 at 10:22 PM, Jeremy Orlow jor...@chromium.orgwrote:

 On Sat, Aug 22, 2009 at 5:54 AM, Robert O'Callahan 
 rob...@ocallahan.orgwrote:

 On Wed, Aug 19, 2009 at 11:26 AM, Jeremy Orlow jor...@chromium.orgwrote:

 First of all, I was wondering why all user prompts are specified as
 must release the storage mutex (
 http://dev.w3.org/html5/spec/Overview.html#user-prompts).  Should this
 really say must instead of may?  IIRC (I couldn't find the original
 thread, unfortunately) this was added because of deadlock concerns.  It
 seems like there might be some UA implementation specific ways this could
 deadlock and there is the question of whether we'd want an alert() while
 holding the lock to block other execution requiring the lock, but I don't
 see why the language should be must.  For Chromium, I don't think we'll
 need to release the lock for any of these, unless there's some
 deadlock scenario I'm missing here.


 So if one page grabs the lock and then does an alert(), and another page
 in the same domain tries to get the lock, you're going to let the latter
 page hang until the user dismisses the alert in the first page?


 Yes.  And I agree this is sub-optimal, but shouldn't it be left up to the
 UAs what to do?  I feel like this is somewhat of an odd case to begin with
 since alerts lock up most (all?) browsers to a varying degrees even without
 using localStorage.


 That behaviour sounds worse than what Firefox currently does, where an
 alert disables input to all tabs in the window (which is already pretty
 bad), because it willl make applications in visually unrelated tabs and
 windows hang.


OK...I guess it makes sense to leave this as is.

One thing I just realized that kind of sucks though:  This makes alert based
debugging much more difficult.  I guess that's acceptable, though.


  Given that different UAs are probably going to have other scenarios where
 they have to drop the lock (some of them may even be purely 
 implementational
 issues), should we add some way for us to notify scripts the lock was
 dropped?  A normal event isn't going to be of much use, since it'll fire
 after the scripts execution ends (so the lock would have been dropped by
 then anyway).  A boolean doesn't seem super useful, but it's better than
 nothing and could help debugging.  Maybe fire an exception?  Are there 
 other
 options?


 A generation counter might be useful.


 Ooo, I like that idea.  When would the counter increment?  It'd be nice if
 it didn't increment if the page did something synchronous but no one else
 took the lock in the mean time.


 Defining no-one else may be difficult. I haven't thought this through, to
 be honest, but I think you could update the counter every time the storage
 mutex is released and the shared state was modified since the storage mutex
 was acquired. Reading the counter would acquire the storage mutex. You'd
 basically write

 var counter = window.storageMutexGenerationCounter;
 ... do lots of stuff ...
 if (window.storageMutexGenerationCounter != counter) {
   // abort, or refresh local state, or something
 }

 I'm not sure what you'd do if you discovered an undesired lock-drop,
 though. If you can't write something sensible instead of abort, or
 something, it's not worth doing.


I guess it would depend on the use.  Let's say a library/framework dependeds
on the lock being held but does a callback (that might do something that
causes the lock to be dropped).  It could check the counter and raise an
exception.  It could also re-start processing if that were an acceptable
answer (but by having the counter, it would only do so when necessary).  I
think it'll be very application specific _what_ you do when you catch such
an error, but I do think it'll be valuable to developers.


  But getStorageUpdates is still not the right name for it.  The only way
 that there'd be any updates to get is if, when you call the function,
 someone else takes the lock and makes some updates.  Maybe it should be
 yieldStorage (or yieldStorageMutex)?  In other words, maybe the name should
 imply that you're allowing concurrent updates to happen?


 I thought that's what getStorageUpdates implied :-).


To me, getStorageUpdates seems to imply that updates have already happened
and we're working with an old version of the data.  I think many developers
will be quite shocked that getStorageUpdates _enables_ others to update
storage.  In other words, 'get' seems to imply that you're consuming state
that's happening anyway, not affecting behavior.

For what it's worth, I sanity checked my point with a web developer here at
Google working with LocalStorage and he too thought the name was
misleading/not clear.  Are we the only ones??

Re: [whatwg] Storage mutex

[Drew Wilson]

On Tue, Aug 25, 2009 at 11:51 AM, Jeremy Orlow jor...@chromium.org wrote:

 On Sun, Aug 23, 2009 at 11:33 PM, Robert O'Callahan 
 rob...@ocallahan.orgwrote:

 On Sat, Aug 22, 2009 at 10:22 PM, Jeremy Orlow jor...@chromium.orgwrote:

 On Sat, Aug 22, 2009 at 5:54 AM, Robert O'Callahan rob...@ocallahan.org
  wrote:

 On Wed, Aug 19, 2009 at 11:26 AM, Jeremy Orlow jor...@chromium.orgwrote:

 First of all, I was wondering why all user prompts are specified as
 must release the storage mutex (
 http://dev.w3.org/html5/spec/Overview.html#user-prompts).  Should this
 really say must instead of may?  IIRC (I couldn't find the original
 thread, unfortunately) this was added because of deadlock concerns.  It
 seems like there might be some UA implementation specific ways this could
 deadlock and there is the question of whether we'd want an alert() while
 holding the lock to block other execution requiring the lock, but I don't
 see why the language should be must.  For Chromium, I don't think we'll
 need to release the lock for any of these, unless there's some
 deadlock scenario I'm missing here.


 So if one page grabs the lock and then does an alert(), and another page
 in the same domain tries to get the lock, you're going to let the latter
 page hang until the user dismisses the alert in the first page?


 Yes.  And I agree this is sub-optimal, but shouldn't it be left up to the
 UAs what to do?  I feel like this is somewhat of an odd case to begin with
 since alerts lock up most (all?) browsers to a varying degrees even without
 using localStorage.


 That behaviour sounds worse than what Firefox currently does, where an
 alert disables input to all tabs in the window (which is already pretty
 bad), because it willl make applications in visually unrelated tabs and
 windows hang.


 OK...I guess it makes sense to leave this as is.

 One thing I just realized that kind of sucks though:  This makes alert
 based debugging much more difficult.  I guess that's acceptable, though.


I'm not sure why, unless you are saying that alert based debugging while
another document is updating the same database simultaneously, then yeah.
But that seems like an obscure case for alert debugging.

The problem with leaving this up to the UA is it becomes a point of
incompatibility - on one browser, it's safe to put up an alert, on another
it isn't. So if applications have to fall back to the LCD behavior, then we
might as well codify it in the spec, which we have :)

Re: [whatwg] Storage mutex

[Robert O'Callahan]

On Tue, Aug 25, 2009 at 11:51 AM, Jeremy Orlow jor...@chromium.org wrote:

 To me, getStorageUpdates seems to imply that updates have already happened
 and we're working with an old version of the data.  I think many developers
 will be quite shocked that getStorageUpdates _enables_ others to update
 storage.  In other words, 'get' seems to imply that you're consuming state
 that's happening anyway, not affecting behavior.


fetchStorageUpdates?

Rob
-- 
He was pierced for our transgressions, he was crushed for our iniquities;
the punishment that brought us peace was upon him, and by his wounds we are
healed. We all, like sheep, have gone astray, each of us has turned to his
own way; and the LORD has laid on him the iniquity of us all. [Isaiah
53:5-6]

Re: [whatwg] Storage mutex

[Jeremy Orlow]

On Tue, Aug 25, 2009 at 10:28 PM, Robert O'Callahan rob...@ocallahan.orgwrote:

 On Tue, Aug 25, 2009 at 11:51 AM, Jeremy Orlow jor...@chromium.orgwrote:

 To me, getStorageUpdates seems to imply that updates have already happened
 and we're working with an old version of the data.  I think many developers
 will be quite shocked that getStorageUpdates _enables_ others to update
 storage.  In other words, 'get' seems to imply that you're consuming state
 that's happening anyway, not affecting behavior.


 fetchStorageUpdates?


fetch has the same problem.  If we want to keep the StorageUpdates suffix,
I'd go with something like allowStorageUpdates.  But, no matter what, it
just doesn't seem very clear that you're actively allowing another thread to
use the storage mutex.

What about yieldStorageMutex?  Yield is enough different from unlock that I
don't think it'll leave developers looking for the lock function.  Yield
fits pretty well since this is essentially cooperative multi-tasking.
 StorageMutex is good because that's what its actually affecting.

Re: [whatwg] Storage mutex

[Robert O'Callahan]

On Sat, Aug 22, 2009 at 10:22 PM, Jeremy Orlow jor...@chromium.org wrote:

 On Sat, Aug 22, 2009 at 5:54 AM, Robert O'Callahan 
 rob...@ocallahan.orgwrote:

 On Wed, Aug 19, 2009 at 11:26 AM, Jeremy Orlow jor...@chromium.orgwrote:

 First of all, I was wondering why all user prompts are specified as must
 release the storage mutex (
 http://dev.w3.org/html5/spec/Overview.html#user-prompts).  Should this
 really say must instead of may?  IIRC (I couldn't find the original
 thread, unfortunately) this was added because of deadlock concerns.  It
 seems like there might be some UA implementation specific ways this could
 deadlock and there is the question of whether we'd want an alert() while
 holding the lock to block other execution requiring the lock, but I don't
 see why the language should be must.  For Chromium, I don't think we'll
 need to release the lock for any of these, unless there's some
 deadlock scenario I'm missing here.


 So if one page grabs the lock and then does an alert(), and another page
 in the same domain tries to get the lock, you're going to let the latter
 page hang until the user dismisses the alert in the first page?


 Yes.  And I agree this is sub-optimal, but shouldn't it be left up to the
 UAs what to do?  I feel like this is somewhat of an odd case to begin with
 since alerts lock up most (all?) browsers to a varying degrees even without
 using localStorage.


That behaviour sounds worse than what Firefox currently does, where an alert
disables input to all tabs in the window (which is already pretty bad),
because it willl make applications in visually unrelated tabs and windows
hang.


  Given that different UAs are probably going to have other scenarios where
 they have to drop the lock (some of them may even be purely implementational
 issues), should we add some way for us to notify scripts the lock was
 dropped?  A normal event isn't going to be of much use, since it'll fire
 after the scripts execution ends (so the lock would have been dropped by
 then anyway).  A boolean doesn't seem super useful, but it's better than
 nothing and could help debugging.  Maybe fire an exception?  Are there other
 options?


 A generation counter might be useful.


 Ooo, I like that idea.  When would the counter increment?  It'd be nice if
 it didn't increment if the page did something synchronous but no one else
 took the lock in the mean time.


Defining no-one else may be difficult. I haven't thought this through, to
be honest, but I think you could update the counter every time the storage
mutex is released and the shared state was modified since the storage mutex
was acquired. Reading the counter would acquire the storage mutex. You'd
basically write

var counter = window.storageMutexGenerationCounter;
... do lots of stuff ...
if (window.storageMutexGenerationCounter != counter) {
  // abort, or refresh local state, or something
}

I'm not sure what you'd do if you discovered an undesired lock-drop, though.
If you can't write something sensible instead of abort, or something, it's
not worth doing.

 But getStorageUpdates is still not the right name for it.  The only way
 that there'd be any updates to get is if, when you call the function,
 someone else takes the lock and makes some updates.  Maybe it should be
 yieldStorage (or yieldStorageMutex)?  In other words, maybe the name should
 imply that you're allowing concurrent updates to happen?


I thought that's what getStorageUpdates implied :-).

Rob
-- 
He was pierced for our transgressions, he was crushed for our iniquities;
the punishment that brought us peace was upon him, and by his wounds we are
healed. We all, like sheep, have gone astray, each of us has turned to his
own way; and the LORD has laid on him the iniquity of us all. [Isaiah
53:5-6]

Re: [whatwg] Storage mutex

[Robert O'Callahan]

On Wed, Aug 19, 2009 at 11:26 AM, Jeremy Orlow jor...@chromium.org wrote:

 First of all, I was wondering why all user prompts are specified as must
 release the storage mutex (
 http://dev.w3.org/html5/spec/Overview.html#user-prompts).  Should this
 really say must instead of may?  IIRC (I couldn't find the original
 thread, unfortunately) this was added because of deadlock concerns.  It
 seems like there might be some UA implementation specific ways this could
 deadlock and there is the question of whether we'd want an alert() while
 holding the lock to block other execution requiring the lock, but I don't
 see why the language should be must.  For Chromium, I don't think we'll
 need to release the lock for any of these, unless there's some
 deadlock scenario I'm missing here.


So if one page grabs the lock and then does an alert(), and another page in
the same domain tries to get the lock, you're going to let the latter page
hang until the user dismisses the alert in the first page?

Given that different UAs are probably going to have other scenarios where
 they have to drop the lock (some of them may even be purely implementational
 issues), should we add some way for us to notify scripts the lock was
 dropped?  A normal event isn't going to be of much use, since it'll fire
 after the scripts execution ends (so the lock would have been dropped by
 then anyway).  A boolean doesn't seem super useful, but it's better than
 nothing and could help debugging.  Maybe fire an exception?  Are there other
 options?


A generation counter might be useful.


 Lastly, is navigator.getStorageUpdates() the right name for the function
 that drops the lock?  Why was it changed from navigator.releaseLock()?  I
 assume we're trying to avoid the word lock, but the reason why you'd need
 to call a function to get updates is not clear without understanding the
 concept of a lock...so what's the point of making this so cryptic?


Authors would be confused that there's no aquireLock() API.

Rob
-- 
He was pierced for our transgressions, he was crushed for our iniquities;
the punishment that brought us peace was upon him, and by his wounds we are
healed. We all, like sheep, have gone astray, each of us has turned to his
own way; and the LORD has laid on him the iniquity of us all. [Isaiah
53:5-6]

Re: [whatwg] Storage mutex

[Jeremy Orlow]

On Tue, Aug 18, 2009 at 4:26 PM, Jeremy Orlow jor...@chromium.org wrote:

 It's also worth noting that Chromium is probably going to need to drop the
 storage mutex for most if not all plugin related calls due to deadlock
 conditions.  If there were some place to mention this as a may type thing,
 it'd be good, but I realize it's probably out of scope for HTML 5.


Oops.  The spec already does specify this behavior:
http://dev.w3.org/html5/spec/Overview.html#storage-mutex


On Sat, Aug 22, 2009 at 5:54 AM, Robert O'Callahan rob...@ocallahan.orgwrote:

 On Wed, Aug 19, 2009 at 11:26 AM, Jeremy Orlow jor...@chromium.orgwrote:

 First of all, I was wondering why all user prompts are specified as must
 release the storage mutex (
 http://dev.w3.org/html5/spec/Overview.html#user-prompts).  Should this
 really say must instead of may?  IIRC (I couldn't find the original
 thread, unfortunately) this was added because of deadlock concerns.  It
 seems like there might be some UA implementation specific ways this could
 deadlock and there is the question of whether we'd want an alert() while
 holding the lock to block other execution requiring the lock, but I don't
 see why the language should be must.  For Chromium, I don't think we'll
 need to release the lock for any of these, unless there's some
 deadlock scenario I'm missing here.


 So if one page grabs the lock and then does an alert(), and another page in
 the same domain tries to get the lock, you're going to let the latter page
 hang until the user dismisses the alert in the first page?


Yes.  And I agree this is sub-optimal, but shouldn't it be left up to the
UAs what to do?  I feel like this is somewhat of an odd case to begin with
since alerts lock up most (all?) browsers to a varying degrees even without
using localStorage.


 Given that different UAs are probably going to have other scenarios where
 they have to drop the lock (some of them may even be purely implementational
 issues), should we add some way for us to notify scripts the lock was
 dropped?  A normal event isn't going to be of much use, since it'll fire
 after the scripts execution ends (so the lock would have been dropped by
 then anyway).  A boolean doesn't seem super useful, but it's better than
 nothing and could help debugging.  Maybe fire an exception?  Are there other
 options?


 A generation counter might be useful.


Ooo, I like that idea.  When would the counter increment?  It'd be nice if
it didn't increment if the page did something synchronous but no one else
took the lock in the mean time.


  Lastly, is navigator.getStorageUpdates() the right name for the function
 that drops the lock?  Why was it changed from navigator.releaseLock()?  I
 assume we're trying to avoid the word lock, but the reason why you'd need
 to call a function to get updates is not clear without understanding the
 concept of a lock...so what's the point of making this so cryptic?


 Authors would be confused that there's no aquireLock() API.


Good point.

But getStorageUpdates is still not the right name for it.  The only way that
there'd be any updates to get is if, when you call the function, someone
else takes the lock and makes some updates.  Maybe it should be yieldStorage
(or yieldStorageMutex)?  In other words, maybe the name should imply that
you're allowing concurrent updates to happen?

[whatwg] Storage mutex

[Jeremy Orlow]

I was looking through the WebStorage and HTML 5 specs regarding the storage
mutex.  I have a few comments/questions.
First of all, I was wondering why all user prompts are specified as must
release the storage mutex (
http://dev.w3.org/html5/spec/Overview.html#user-prompts).  Should this
really say must instead of may?  IIRC (I couldn't find the original
thread, unfortunately) this was added because of deadlock concerns.  It
seems like there might be some UA implementation specific ways this could
deadlock and there is the question of whether we'd want an alert() while
holding the lock to block other execution requiring the lock, but I don't
see why the language should be must.  For Chromium, I don't think we'll
need to release the lock for any of these, unless there's some
deadlock scenario I'm missing here.

It's also worth noting that Chromium is probably going to need to drop the
storage mutex for most if not all plugin related calls due to deadlock
conditions.  If there were some place to mention this as a may type thing,
it'd be good, but I realize it's probably out of scope for HTML 5.

Given that different UAs are probably going to have other scenarios where
they have to drop the lock (some of them may even be purely implementational
issues), should we add some way for us to notify scripts the lock was
dropped?  A normal event isn't going to be of much use, since it'll fire
after the scripts execution ends (so the lock would have been dropped by
then anyway).  A boolean doesn't seem super useful, but it's better than
nothing and could help debugging.  Maybe fire an exception?  Are there other
options?

Lastly, is navigator.getStorageUpdates() the right name for the function
that drops the lock?  Why was it changed from navigator.releaseLock()?  I
assume we're trying to avoid the word lock, but the reason why you'd need
to call a function to get updates is not clear without understanding the
concept of a lock...so what's the point of making this so cryptic?

Thanks,
Jeremy