Web Storage Mutex
The use of the storage mutex to avoid race conditions is currently considered by certain implementors to be too high a performance burden, to the point where allowing data corruption is considered preferable. Alternatives that do not require a user-agent-wide per-origin script lock are eagerly sought after. It's not a question of mutex versus data corruption, but of implementation: Database storage is served by SQLite. LocalStorage would be better served by Tokyo Cabinet: http://1978th.net/tokyocabinet/. I doubt the current localStorage implementation is better than the current Tokyo Cabinet implementation. Joran Greef
[AC/CORS] Proper behavior for user agents who return 'null' Access-Control-Allow-Origin
I discovered today that Origin handling for CORS is a bit odd on Firefox with respect to requests made from webpages that are loaded locally (e.g. loaded from the file:// access scheme). In this case, CORS preflight requests and simple cross-origin requests are sent with a null (String) value for Origin. Initially, I thought this was a bug and filed it with Mozilla[1]. Jonas pointed out (rightfully) that I need to do a better job reading the spec and that a null string value is perfectly acceptable. However, I noticed that Firefox would fail to issue the follow on request after a successful pre-flight request IFF the server returned the null string for Access-Control-Allow-Origin, even though that's what the user agent originally sent. I added this finding onto the same bug (see also). Jonas responded that it appears that the CORS spec had changed since that was implemented in Firefox, and that he believes the spec may be incorrect. I was able to verify that Firefox behaves properly only if the server sends * for Access-Control-Allow-Origin. I dug a bit through the archives but I couldn't find the rationale for the change to the CORS spec. I did notice that the change occurred *after* the spec dated 14 Feb 2008[2], or at least the notion that null matches nothing disappeared after that time, and that the current spec[3] explicitly states in section 6.2 that the Resource Sharing Check algorithm ...also functions when the ASCII serialization of an origin is the string 'null'. --sgp cf. smerpology.org [1] https://bugzilla.mozilla.org/show_bug.cgi?id=533987 [2] http://www.w3.org/TR/2008/WD-access-control-20080214 [3] http://www.w3.org/TR/2009/WD-cors-20090317/
Re: Web Storage Mutex
On Thu, Dec 10, 2009 at 10:36 PM, Joran Greef jo...@sexbyfood.com wrote: The use of the storage mutex to avoid race conditions is currently considered by certain implementors to be too high a performance burden, to the point where allowing data corruption is considered preferable. Alternatives that do not require a user-agent-wide per-origin script lock are eagerly sought after. It's not a question of mutex versus data corruption, but of implementation: Database storage is served by SQLite. LocalStorage would be better served by Tokyo Cabinet: http://1978th.net/tokyocabinet/. I doubt the current localStorage implementation is better than the current Tokyo Cabinet implementation. The issue has nothing to do with SQLite. If you support run-to-completion (i.e. require serialization) and can't abort and retry a transaction (which the LocalStorage API doesn't support) then there's no way around locking as far as I know. If you're arguing there is, can you please explain how (rather than linking to a rather large document)? Thanks, J
Re: Widget specification - liquid height support
Hi Amit, On Dec 10, 2009, at 15:54 , Amit Kasher wrote: I couldn’t find any mentioning to the “liquidness” of the new HTML element “widget”. What happens if one does not configure any height or width to it? In terms of width, I assume it behaves like a div and takes the entire container width, but what happens with height? Does it behave like a div that changes its height according to its content, or does it behave like an iframe that doesn’t? I think you have misunderstood the purpose of the widget element. It is not meant to be used in HTML documents, or for that matter to ever be rendered. It simply is a container for the configuration of widgets, as part of the config.xml file that widget packages contain. -- Robin Berjon - http://berjon.com/
[widgets] Widget Interface Test Suite
Hi All, In preparation for CR of the Widget Interface, You can now find the test suite for the Widget interface spec in the CVS repo (Yes! I know what you are thinking, xmas _has_ come early this year!:)). The test suite includes Dom's automatically generated IDL tests as a single test widget (za.wgt); 47 grueling tests that are sure to test the might of any willing User Agent. View the tests here: http://dev.w3.org/2006/waf/widgets-api/test-suite/ This test suite is currently alpha quality... so, a) beware!, and b) I need your help verifying tests, etc. Please send me your feedback on things that are broken, etc. As with PC, please manage your own results (I've created the result xml files already for you): http://dev.w3.org/2006/waf/widgets-api/imp-report/results/ You can see your progress as you go in the implementation report: http://dev.w3.org/2006/waf/widgets-api/imp-report/ Happy testing!
Re: WebSimpleDB Issues
Hi Kris, Sorry for taking so long to get back on these issues. On Dec 1, 2009, at 10:33 PM, Kris Zyp wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I had few thoughts/questions/issues with the WebSimpleDB proposal: * No O(log n) access to position/counts in index sequences - If you want find all the entities that have a price less than 10, it is quite easy (assuming there is an index on that property) with the WebSimpleDB to access the price index, iterate through the index until you hit 10 (or vice versa). However, if this is a large set of data, the vast majority of applications I have built involve providing a subset or page of data at a time to keep constant or logarithmic time access to data, *and* an indication of how many rows/items/entities are available. Limiting a query to a certain number of items is easy enough with WebSimple, you just only iterate so far, but determining how many items are less than 10 is no longer a logarithmic complexity problem, but is linear with the number of items that are less 10, because you have to iterate over all of them to count them. If a cursor were to indicate the numeric position within an index (even if it was an estimate, without transactional strictness), one could easily determine the count of these type of queries in O(log n) time. This would presumably entail B-tree nodes keeping track of their number of children. There are definitely important implementation challenges with keep track of the number of children for each node. I have added a mechanism to the API but am willing to be flexible based on implementation feedback. This mechanism is a count attribute on the CursorSync interface. It will allow you to find out the number of records corresponding to the current position of a cursor. Please take a look. * Asynchronicity is not well-aligned with the expensive operations - The asynchronous actions are starting and committing transactions. It makes sense that committing transactions would be expensive, but why do we make the start of a transaction asynchronous? Is there an expectation that the a global lock will be sought on the entire DB when the transaction is started? That certainly doesn't seem desirable. Normally a DB would create locks as data is accessed, correct? If anything a get operation would be more costly than starting a transaction. There are three transaction behaviors that are possible - a dynamically scoped transaction (like what you are suggesting), a statically scoped transaction that includes the entire database and a statically scoped transaction that includes only a subset of the database objects. Obviously in the static scope case, we would like to reserve the objects prior to beginning the transaction. That is why the call should block (in the sync case). We are providing the statically scoped transaction to support the use case for deadlock- free operation (even though it may cause timeouts). * Hanging EntityStores off of transactions creates unnecessary complexity in referencing stores - A typical pattern in applications is to provide a reference to a store to a widget that will use it. However, with the WebSimpleDB, you can't really just hand off a reference to an EntityStore, since each store object is transaction-specific. You would either need to pass the name of store to a widget, and have it generate its own transaction to get a store (which seems like very bad form from object capability perspective), or pass in a store for every action, which may not be viable in many situations. This has changed just prior to TPAC (early November). I encourage you to take a look at the current API, which has taken in to all the feedback I have been getting since then. Would it be reasonable (based on the last two points) to have getEntityStore be a method on database objects, rather than transaction objects? Actions would just take place in the current transaction for the context. With the single-threaded nature of JS contexts, having a single active transaction at a time doesn't not seem like a hardship, but rather makes things a lot simpler to work with. Also, if an impl wanted to support auto-commit, it would be very intuitive, devs just wouldn't be required to start a transaction prior performing actions on a store. Thanks, I greatly appreciate your interest and detailed feedback. Nikunj http://o-micron.blogspot.com
Re: [WebSimpleDB] Allowing schema operations anywhere
I have gone ahead and updated the spec to allow option B (only). Please take a look. Nikunj On Dec 8, 2009, at 10:14 AM, Nikunj R. Mehta wrote: Hi Pablo, Sorry for the long delay in responding to your comments. Hopefully, we can continue the discussion now. Schema changes interact with the locking model of the database. As I see it, here are several ways in which the API could be designed and the consequences of doing so: A. Allow schema changes inside a metadata transaction which can only be performed at connection time B. Allow schema changes inside a data transaction, which can be performed any time a connection is open C. Allow schema changes inside a metadata transaction, which can be performed any time a connection is open Option A's disadvantages are that metadata manipulation cannot be combined with data changes. Moreover, version numbers are no longer issued by the application but rather by a user agent. Option A's advantages are that resource acquisition is simplified and deadlocks can be avoided considering that a connection acquires and releases the metadata resource in a consistent sequence. Another upside is that version number maintenance is automated. Option B's main disadvantage is that there is no real notion of version that can be managed by the user agent. Another is that deadlocks could occur because there is no a priori declaration of intent about metadata modification. This could be remedied by including the database itself in the list of objects that are intended to be modified in the transaction. Option B's advantages are closer interleaving of and atomic metadata changes with data changes, and application controlled version numbers used for the database. Option C's disadvantage is that data and metadata changes cannot be interleaved atomically. Option C's advantages are that deadlocks can be avoided and version number management can be performed by an application. Overall, I think version management and metadata changes are exclusive in some sense. IOW, if we want Option B and Option C, then we have to remove the connection time version check. Hope that helps. Please feel free to add if I missed anything. Nikunj On Nov 22, 2009, at 3:14 PM, Pablo Castro wrote: We are finding a number of reasons for wanting to create tables on the fly, and without bumping up the database version. A few examples: - Packaged components that create side tables to maintain its own state - Query processors often need to spill to disk during query execution. For example, sorting large sets requires storing temporary sets of rows on disk to be merged later. So we're thinking it would be better to have these methods directly in the DatabaseSync/DatabaseAsync objects (with proper corresponding patterns), instead of their current location in the Upgrade interface. For the common case where several schema changes need to be done atomically, developers can simply wrap the calls in a transaction, and they would do for regular data manipulation. We would need an extra method to bump up the version explicitly, as that would no longer be in the upgrade callback. Does this seem reasonable? Regards, -pablo Nikunj http://o-micron.blogspot.com Nikunj http://o-micron.blogspot.com
Re: [WebSimpleDB] Flatting APIs to simplify primary cases
Hi Pablo,
Application feedback is very helpful for the async API. This is one
area that I have been the least convinced about (poor programmability
means dissatisfied users).
I am currently investigating the impacts of doing what is needed to
simplify the async API. At the very least, your proposal will add some
20 methods to a single interface - some applying to the database,
others to the store, and the rest to an index.
Additionally, we have to consider the case where more than one request
is concurrently made to the database.
I will get back to the WG with my findings. In the meanwhile, if you
have more findings to report, I would be glad to take them into
consideration.
Thanks,
Nikunj
On Nov 19, 2009, at 12:25 PM, Pablo Castro wrote:
We're busy creating experimental implementations of WebSimpleDB to
both understand what it takes to implement and also to see what the
developer experience looks like.
As we started to write application code against the API
(particularly the async one) the first thing that popped is the fact
that you need two levels of nested callbacks for everything. While
the current factoring of the API makes sense on the design board,
it's kind of noisy in app code. For example:
// assume you already have a database opened in dbReq
var html = ul;
var storeReq = new ObjectStoreRequest(dbReq.database);
storeReq.success = function() {
var cursorReq = new CursorRequest(storeReq.store);
cursorReq.callback = function(key, cursor, value) {
html += li + value.Name + /li;
}
cursorReq.onsuccess = function(r) {
document.getElementById(output).innerHTML = html + /ul;
}
cursorReq.open();
}
storeReq.open();
One option that we would like to explore is to flatten the API, so
most common methods are straight in the database class. This trades
off some of the factoring in favor of usability for common cases
using the async API.
The change would span a couple of aspects:
1. Move operations from object store interface and the index
interface into the Database interface.
Accessing indexes and stores through specialized objects is
problematic for the following reasons:
- It's always the case that we need to consider when objects are
invalidated because something changes from underneath them, for
example a schema change. So for example, if there is an explicit
store object, then when the store is dropped we need to consider
what is valid/invalid and what its failure points and modes are. By
not having a standalone store object, we significantly reduce the
gotchas to consider.
- From a usability perspective, it's simpler to work with a store in
a single step, rather than having to open it first and then work
with it (see patterns below with a single request and one DBRequest
object).
- With no two-step access pattern, the API has one less level of
asynchronicity, as effectively the table lookup + operation are
atomic within the store. This also consolidates all operations with
an async variant in a single interface (the Database), which is a
great simplification for discoverability.
var html = ul;
var request = asyncDb.forEachStoreObject(contacts, function(row) {
html += li + row.Name + /li;
});
request.onsuccess = function(r) {
document.getElementById(output).innerHTML = html + /ul; }
In moving the operations, it's probably best to rename them to
something more descriptive, so we can have for example
'getFromStore(storeName, key)' and 'getFromIndex(storeName,
indexName, key)'. This also helps in that 'delete' won't collide
with the Javascript keyword.
Note that the store and index interfaces are still around to provide
metadata, but at this point they behave as simple read-only snapshots.
2. Generalize the use of DBRequest, add a 'result' member to it and
have all asynchronous operations be initiated from a DatabaseAsync
interface.
As a result of the previous changes, all operations that have an
async counterpart should now exist on the DatabaseAsync interface.
Rather than having multiple types of requests depending on the
target object, it is possible to have operations on a DatabaseAsync
interface that provide a uniform invocation and handling programming
pattern.
This gives a nice pattern for understanding how a sync API maps to
an async API.
So for example:
var record = db.getFromStore(store, key); // use record...
Becomes:
var request = asyncDb.getFromStore(store, key); request.onsuccess
= function(req) {
var record = req.result;
// use record...
};
We could include more data in DBRequest or DBRequest.result as
needed if in some cases a method produces more than just a simple
result. Further specializatons of DBRequest (subtypes) are still
possible in the future if we need to introduce special cases for
specific operations.
Similarly, we would have something like asyncDb.forEachStoreObject()
that queues a task to call a
Re: [DataCache] Updated Examples
I was mistaken. commit() is available for offline transactions! Its in the CacheTransaction IDL interface, not in the OnlineTransaction IDL interface. commit() should be added back into the examples. However, this raises the same question I raised about send() and MutableHttpResponse. What happens in the case where you do not issue a commit()? Again, I feel it should implicitly commit() the transaction. In which case the commit() could be left out and potentially removed. Thanks, Joseph Pecoraro
Re: CfC: to publish LCWD of: Server-Events, Web {SQL Database, Sockets, Storage, Worker}; deadline 15 December
Hi Art, This is Oracle's official position on Web SQL Database: Oracle does not believe that Web SQL Database is ready for Last Call. Oracle believes that: 1. It is not good for the industry to start a new SQL language standard track. A better approach would be to define a profile based on the existing SQL language standard, ISO/IEC 9075, that meets the needs of Web applications. We should not ignore the many years of work by that committee and the database community, and should instead build on it. 2. The draft being proposed for Last Call, defines the SQL language by referring to a particular version of the SQLite implementation. Such a definition is not a valid standard as it does not allow for alternative implementations. To turn this into a valid specification we need to include a normative textual specification of the language accepted by SQLite. We can start with the SQLite SQL language manual (http://www.sqlite.org/lang.html ), but again, just referring to this URL is not acceptable as a normative specification. 3. Assuming that the editor decides to specify the SQL language by including its normative textual specification, the WG needs time to review this substantial amount of new material before sending this proposal out to the larger community. I want to remind the members that silence today means assent for Last Call, which means they agree that: (1) A new SQL language standards track is good, (2) Defining a bag of implementation bits as a standard is good. Even if you personally have lost interest in this specification and can't be bothered to respond, remember that we in the WG have a collective responsibility to do no harm. Nikunj P. S. Consider this paragraph from the SQLite manual (http://www.sqlite.org/lang_droptable.html ): The DROP TABLE statement does not reduce the size of the database file in the default mode. Empty space in the database is retained for later INSERT statements. To remove free space in the database, use the VACUUM statement. If auto_vacuum mode is enabled for a database then space will be freed automatically by DROP TABLE. This is a perfectly fine behavior and a perfectly fine paragraph in a manual. However this is not the kind of language you expect to see in a standard where a wider range of behaviors is desired. On Dec 7, 2009, at 4:46 PM, Arthur Barstow wrote: This is a Call for Consensus (CfC) to publish a Last Call Working Draft of the following specs: 1. Server-Sent Events http://dev.w3.org/html5/eventsource/ 2. Web SQL Database http://dev.w3.org/html5/webdatabase/ 3. Web Sockets API http://dev.w3.org/html5/websockets/ 4. Web Storage http://dev.w3.org/html5/webstorage/ 5. Web Workers http://dev.w3.org/html5/workers/ This CfC satisfies the group's requirement to record the group's decision to request advancement to LCWD. Note that as specified in the Process Document [PD], a Working Group's Last Call announcement is a signal that: * the Working Group believes that it has satisfied its relevant technical requirements (e.g., of the charter or requirements document) in the Working Draft; * the Working Group believes that it has satisfied significant dependencies with other groups; * other groups SHOULD review the document to confirm that these dependencies have been satisfied. In general, a Last Call announcement is also a signal that the Working Group is planning to advance the technical report to later maturity levels. As with all of our CfCs, positive response is preferred and encouraged and silence will be assumed to be assent. The deadline for comments is 14 December. The comment period length will be 6 months (ending ~17 June 2010 if the LCWD is published ~ 17 December 2009) unless someone commits (by 15 December) to completing the review earlier including actively editing the spec, responding to comments, etc. -Regards, Art Barstow [PD] http://www.w3.org/2005/10/Process-20051014/tr.html#last-call Nikunj http://o-micron.blogspot.com
Re: CfC: to publish LCWD of: Server-Events, Web {SQL Database, Sockets, Storage, Worker}; deadline 15 December
As always, I should point out that there is no such thing as an official mozilla position. With that out of the way, my feelings are: * I don't think that source code makes a good specification. It's bound to specify things that we'd rather not specify (i.e. bugs in the code), and it'll make it hard to write multiple independent specifications. * I personally don't consider multiple implementations that are all backed by SQLite to be independent implementations. Given that, and that it sounds like no one is planning on writing an implementation that is not backed by SQLite, it does not seem like this spec is likely to go to Rec. However, if despite these comments others want to spend time on taking the spec further, and if W3C is willing to allow that despite the above technical comments, then I'm not going to stand in the way. I'd rather spend my time on improving the spec formerly known as WebSimpleDB. / Jonas On Fri, Dec 11, 2009 at 6:26 PM, Nikunj R. Mehta nikunj.me...@oracle.com wrote: Hi Art, This is Oracle's official position on Web SQL Database: Oracle does not believe that Web SQL Database is ready for Last Call. Oracle believes that: 1. It is not good for the industry to start a new SQL language standard track. A better approach would be to define a profile based on the existing SQL language standard, ISO/IEC 9075, that meets the needs of Web applications. We should not ignore the many years of work by that committee and the database community, and should instead build on it. 2. The draft being proposed for Last Call, defines the SQL language by referring to a particular version of the SQLite implementation. Such a definition is not a valid standard as it does not allow for alternative implementations. To turn this into a valid specification we need to include a normative textual specification of the language accepted by SQLite. We can start with the SQLite SQL language manual (http://www.sqlite.org/lang.html), but again, just referring to this URL is not acceptable as a normative specification. 3. Assuming that the editor decides to specify the SQL language by including its normative textual specification, the WG needs time to review this substantial amount of new material before sending this proposal out to the larger community. I want to remind the members that silence today means assent for Last Call, which means they agree that: (1) A new SQL language standards track is good, (2) Defining a bag of implementation bits as a standard is good. Even if you personally have lost interest in this specification and can't be bothered to respond, remember that we in the WG have a collective responsibility to do no harm. Nikunj P. S. Consider this paragraph from the SQLite manual (http://www.sqlite.org/lang_droptable.html): The DROP TABLE statement does not reduce the size of the database file in the default mode. Empty space in the database is retained for later INSERT statements. To remove free space in the database, use the VACUUM statement. If auto_vacuum mode is enabled for a database then space will be freed automatically by DROP TABLE. This is a perfectly fine behavior and a perfectly fine paragraph in a manual. However this is not the kind of language you expect to see in a standard where a wider range of behaviors is desired. On Dec 7, 2009, at 4:46 PM, Arthur Barstow wrote: This is a Call for Consensus (CfC) to publish a Last Call Working Draft of the following specs: 1. Server-Sent Events http://dev.w3.org/html5/eventsource/ 2. Web SQL Database http://dev.w3.org/html5/webdatabase/ 3. Web Sockets API http://dev.w3.org/html5/websockets/ 4. Web Storage http://dev.w3.org/html5/webstorage/ 5. Web Workers http://dev.w3.org/html5/workers/ This CfC satisfies the group's requirement to record the group's decision to request advancement to LCWD. Note that as specified in the Process Document [PD], a Working Group's Last Call announcement is a signal that: * the Working Group believes that it has satisfied its relevant technical requirements (e.g., of the charter or requirements document) in the Working Draft; * the Working Group believes that it has satisfied significant dependencies with other groups; * other groups SHOULD review the document to confirm that these dependencies have been satisfied. In general, a Last Call announcement is also a signal that the Working Group is planning to advance the technical report to later maturity levels. As with all of our CfCs, positive response is preferred and encouraged and silence will be assumed to be assent. The deadline for comments is 14 December. The comment period length will be 6 months (ending ~17 June 2010 if the LCWD is published ~ 17 December 2009) unless someone commits (by 15 December) to completing the review earlier including actively editing the spec, responding to comments, etc. -Regards, Art Barstow [PD]
