On Nov 9, 2009, at 3:12 AM, Robert O'Callahan wrote:
On Mon, Nov 9, 2009 at 9:58 PM, Maciej Stachowiak <m...@apple.com>
wrote:
On Nov 8, 2009, at 11:12 PM, Jonas Sicking wrote:
Indeed. I still personally wouldn't call it multiple independent
implementations though.
Would you call multiple implementations that use the standard C
library independent? Obviously there's a judgment call to be made
here.
Yes. Multiple implementations passing query strings (more or less)
verbatim to SQLite for parsing and interpretation would not pass
that judgement call ... IMHO, but wouldn't you agree?
If that were the extent of the implementation, I might agree. However,
that doesn't accurately characterize at least WebKit's WebDatabase
implementation. WebKit has around 15k lines of code which implement
asynchronicity, do checking and rewrites on the queries, export DOM
APIs, manage transactions, expose result sets, etc. It's true that
SQLite has a fair bit more code. But then again, the image decoding
library we use has more code than our <img> element implementation.
I think it's worth specifying and testing the layers above the query
language even if, for whatever reason, the query language does not get
specified.
* If we do specify a specific SQL dialect, that leaves us having to
implement it. It's very unlikely that the dialect would be compatible
with SQLite (especially given that SQLite uses a fairly unusual SQL
dialect with regards to datatypes) which likely leaves us implementing
our own SQL engine.
The SQL dialect could, of course, be specified in a way that it
could work on top of SQLite with some reasonable filtering and
preprocessing steps. I don't see a reason to expect otherwise, and
in particular that it would be so different that it would require
writing a new SQL engine.
I think the problem is rather coming up with a SQL definition that
can be implemented by anything other than SQLite (or from scratch,
of course). One weird thing about SQLite is that column types aren't
enforced. So either the spec requires something like SQLite's "type
affinity" (in which case it doesn't fit well with most other SQL
implementations, and precludes common performance optimizations), or
it requires strict type checking (which perhaps you could implement
in SQLite by adding CHECK constraints?). But the latter course is
probably incompatible with deployed content, so contrary to Jonas I
expect the spec would be implementable *only* on top of SQLite (or
from scratch, of course), or perhaps some unnatural embedding into
other engines where all values are text or variants. Experience with
alternative implementations would be important.
Indeed, it is a challenge to find the right design point. Right now it
looks like no one is going to take up this challenge.
Does SimpleDB give performance guarantees? Does any Web platform
spec give performance guarantees? Even DOM Core methods have
different Big-O complexity in Gecko and WebKit. So this doesn't seem
very persuasive.
You're right that precedent is weak here, but I think database APIs
aspire to work with larger data sets than the DOM does, so big-O
guarantees are more important.
Is SimpleDB going to give big-O performance guarantees? (The current
draft does not appear to.)
So basically, at this point the spec for the SQL dialect is blocked
on Mozilla, since no one is interested in doing that spec work if it
won't actually lead to wider implementation.
That doesn't seem logical to me. The spec work has to be done if Web
Database is to advance through the spec process. You want it to
advance even if Mozilla doesn't agree to implement it. Therefore,
the spec work has to be done. (And if there was a SQL dialect spec
that's simple and elegant and amenable to multiple implementations
and compatible with your deployed content, that would certainly be a
large point in favour of Web Database.)
At the Web Apps WG face-to-face meeting at TPAC, all parties agreed
(in the room at least) to let the spec continue without fully
specifying the SQL dialect. The reason is that all parties who
currently have or are in the process of developing implementations did
not appear to need it, and the parties that would be blocked (Mozilla,
Microsoft) said their decision would not be swayed by having a spec,
and would not implement regardless. Thus, it did not seem there would
be a practical benefit to specifying the SQL dialect. Thus, those
present said they were satisfied to specify that SQLite v3 is the
dialect.
Note: I would try to find Apple resources to help write a SQL dialect
spec if anyone says it will materially help them to have such a spec
(and the level of interest doesn't reach the threshold where Hixie
wants to write it himself). I don't think I could get resources if
it's just a busywork exercise.
I don't think SimpleDB is useless for mobile platforms. You
certainly *could* use it. But it does have three significant
downsides compared to the SQL database: (1) it's very different from
what developers have already (happily) been using on mobile; (2) the
target design point is that it's primarily expected to be used
through JavaScript libraries layered on top, and not directly (so
you have to ship more code over the wire); and (3) for more complex
queries, more of the work has to be done in JavaScript instead of in
the database engine (so performance will likely be poor on low-power
CPUs).
Do you have easy access to knowledge about the sort of complex
queries these mobile apps do? That would be very useful.
I don't have easy access to that knowledge, but I can attempt to
inquire.
We already ship SQLite and implementing Web Database using SQLite
would definitely be the path of least resistance for us. We're just
concerned it might not be the right thing for the Web.
My thoughts on this are as follows:
1) It seems that a database layer that's less "opinionated" than SQL
is a better target for library authors; by having less conceptual
baggage and policy, it makes it easier for different conceptual models
(such as object oriented stores) to be layered on top. Thus, library
developers are likely to want a lower-level solution.
2) It seems that a database layer with a good amount of high-level
concepts (including some kind of query language) is likely to be
easier to code against directly for many use cases. Thus, application
programmers, particularly in environments where extra abstraction
layers are particularly costly
3) Some mobile web developers have existing investment in SQL in
particular, and do not appear to have had problems with it as a model.
It would be a shame to abandon them, as in many ways they have been
better pioneers of offline Web apps than mainline desktop-focused Web
developers.
It seems plausible to me that SQL is not the best solution for all
storage use cases. But it seems like a pretty aggressive position to
say that, as a result, it should be out of the Web platform (and not
just augmented by other facilities). It seems like that would
underserve other use cases
Regards,
Maciej
