Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
In fairness, and perspective, GLSL is a big leap, adding one more language with neat features like swizzle and matrix math. SQL is too big a leap for the time being. I'd like to see a continued effort at 'borrowing' from our new web family member, glsl. Typed arrays are wonderful. There's been a little exploration into CSSMatrix for mat4 operations. Interesting stuff. That's my top-post opinion. I have a lot of respect for sqlite-- postgres devs gave me the impression they'd prefer a method based API over relexing or reusing the sqlite dialect. -Charles On Apr 6, 2011, at 10:24 PM, Boris Zbarsky bzbar...@mit.edu wrote: On 4/4/11 10:15 AM, Ryan Fugger wrote: That's not the only reason. Mozilla laid out others ten months ago: https://hacks.mozilla.org/2010/06/beyond-html5-database-apis-and-the-road-to-indexeddb/ Mozilla's plan appears to be to implement IndexedDB on top of SQLite, This is not a plan so much as a current stopgap. Why not just expose the thing and let developers worry about whether what they're using is standard or not, and suffer the consequences later, if in fact there are any? Because due to the incentive structure here the entities making the bad decisions are not the ones who end up having to suffer the consequences. It's a typical case of externalizing the negatives. It seems to me that the proper role for standards is to step in and help clear things up when they get messy Often it's too late by then. -Boris
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On Tue, Apr 5, 2011 at 9:33 PM, Joran Greef jo...@ronomon.com wrote: On 06 Apr 2011, at 2:53 AM, Pablo Castro wrote: The goal of IndexedDB has always been to enable things like RelationalDB and CouchDB to be built on top, while maintaining a reasonable level of functionality for those that wanted to use it directly. I really like the idea of thinking of RelationalDB as something that's built as a library on top of IndexedDB. Are there specific tweaks we can make to IndexedDB so it can be a good lower-layer for RelationalDB, such that RelationalDB could be built as a pure JavaScript library? Thanks -pablo 1. Treat object values as opaque (necessary to avoid deserialization/serialization overhead, this is mandatory for storing anything over 50,000 objects on a device like an iPad or iPhone). Please explain this in more detail as I have no idea what you mean by treat as opaque. Are you saying that we should not allow storing objects but rather only allow storing strings? If not, surely any type of object needs to be serialized upon storage. If you are simply suggesting forbidding storing objects, then this doesn't seem like a blocker. Simply store a string and we won't serialize anything. I'm also interested in what you are basing the claim on overhead on. Have you profiled a IndexedDB implementation? If so, which? And if Firefox, did you do so before or after we switched away from using a JSON serializer? 2. Enable indices to be modified at time of putting/deleting objects (index references provided by application at time of putObject/deleteObject call). I don't believe that this is a blocker. You can simply modify the object you are storing to add properties and then index of these properties. What you are suggesting only has the advantage that it allows storing objects without modifying them. While that can be important, it isn't a blocker to at least creating a prototype implementation. 3. Provide a simpler, more powerful locking mechanism, opaque to IndexedDB, to provide finer-grained application-specific locking (i.e. have we just entered into a sync process with the master database). Are you talking about supporting row-level locking? As far as I can tell we support as fine grained locking as possible without going to row-level locks. We already support more fine grained locking than for example WebSQL which only support whole-database locks. None of the issues you are listing seems like blockers for creating a RelationalDB implementation on top of IndexedDB. And there's still an abundance of lack of proposals. / Jonas
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 06 Apr 2011, at 8:56 AM, Jonas Sicking wrote: 1. Treat object values as opaque (necessary to avoid deserialization/serialization overhead, this is mandatory for storing anything over 50,000 objects on a device like an iPad or iPhone). Please explain this in more detail as I have no idea what you mean by treat as opaque. Are you saying that we should not allow storing objects but rather only allow storing strings? If not, surely any type of object needs to be serialized upon storage. If you are simply suggesting forbidding storing objects, then this doesn't seem like a blocker. Simply store a string and we won't serialize anything. I'm also interested in what you are basing the claim on overhead on. Have you profiled a IndexedDB implementation? If so, which? And if Firefox, did you do so before or after we switched away from using a JSON serializer? Yes, it must accept a string value and store that directly. The opaque terminology comes from some of the BDB papers. I tested both Chrome and Firefox implementations 3 weeks ago. Both were an order of magnitude slower than using SQLite as a key-value store (storing strings as blobs). You can use whatever serializer you like, but it will always be slower than avoiding serialization completely (this is possible by the way, my application does not deserialize objects received from the server before storing them). Even if your serializer takes only 1ms per serialize call, that's 50 seconds for 50,000 objects. For my use-case that is unacceptable, considering that SQLite is available in Chrome and Safari. I will encourage my users to use those browsers and continue developing for SQLite until IndexedDB resolves this issue. How would you support indices (see below) if you say Simply store a string and we won't serialize anything.? 2. Enable indices to be modified at time of putting/deleting objects (index references provided by application at time of putObject/deleteObject call). I don't believe that this is a blocker. You can simply modify the object you are storing to add properties and then index of these properties. What you are suggesting only has the advantage that it allows storing objects without modifying them. While that can be important, it isn't a blocker to at least creating a prototype implementation. How would you index objects passed to putObject as a string (see above)? Plus you have the unnecessary object creation overhead. How fast is it to create 50,000 objects on an iPad? What would that do to the GC and why would you want to do that if you don't need to? I would like to see Mozilla do as they say: re-implement a SQLite on IndexedDB themselves, that is just as fast and memory efficient as the original, before suggesting that this is possible, that the web therefore be deprived of SQLite. Furthermore, that Mozilla stop using SQLite for all internal use, and rely solely on IndexedDB instead. That is essentially the request that Mozilla are making of web developers today. It's clear that scores of web developers are upset with the decision to deprecate WebSQL. It's not clear that IndexedDB provides anything close in terms of actual raw performance. This surprised me greatly since I assumed IndexedDB would naturally leverage established indexed key-value ideas (for instance to quote BDB - In Berkeley DB, the key and value in a record are opaque to Berkeley DB) which would give it an edge over SQLite. Pragmatically speaking, would it really be so hard for Mozilla to join Chrome, Safari and Opera and provide an embedding of SQLite along with IndexedDB? If IndexedDB is as good as you suggest it is, then I am sure developers will flock to it, and you won't need to speculate as to whether or not SQLite will take over the web and then break backwards compatibility (despite a stated objective and proven track record of not doing so). And if SQLite did ever break backwards compatibility then developers would have IndexedDB. And if applications relying on SQLite are abandoned by their authors and broken as a result of not upgrading, then arguably those applications should be deprecated and not SQLite.
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On Wed, Apr 6, 2011 at 3:21 AM, Joran Greef jo...@ronomon.com wrote: On 06 Apr 2011, at 8:56 AM, Jonas Sicking wrote: 1. Treat object values as opaque (necessary to avoid deserialization/serialization overhead, this is mandatory for storing anything over 50,000 objects on a device like an iPad or iPhone). Please explain this in more detail as I have no idea what you mean by treat as opaque. Are you saying that we should not allow storing objects but rather only allow storing strings? If not, surely any type of object needs to be serialized upon storage. If you are simply suggesting forbidding storing objects, then this doesn't seem like a blocker. Simply store a string and we won't serialize anything. I'm also interested in what you are basing the claim on overhead on. Have you profiled a IndexedDB implementation? If so, which? And if Firefox, did you do so before or after we switched away from using a JSON serializer? Yes, it must accept a string value and store that directly. The opaque terminology comes from some of the BDB papers. This is already supported, so I don't understand why you list this as a blocker for implementing something like RelationalDB. I tested both Chrome and Firefox implementations 3 weeks ago. Both were an order of magnitude slower than using SQLite as a key-value store (storing strings as blobs). You can use whatever serializer you like, but it will always be slower than avoiding serialization completely (this is possible by the way, my application does not deserialize objects received from the server before storing them). Even if your serializer takes only 1ms per serialize call, that's 50 seconds for 50,000 objects. For my use-case that is unacceptable, considering that SQLite is available in Chrome and Safari. I will encourage my users to use those browsers and continue developing for SQLite until IndexedDB resolves this issue. Sounds like you have no basis for saying that serialization is an issue then. How would you support indices (see below) if you say Simply store a string and we won't serialize anything.? This however is a problem I agree. However shouldn't be a blocker for RelationalDB. You'd simply get a bit of overhead. In fact, writing a prototype would allow us to measure how big that overhead is and would make an excellent case for adding the API you are suggesting. / Jonas
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 4/4/2011 8:07 AM, Joran Greef wrote: SQLite has a fantastic track record of maintaining backwards compatibility. Sort of. They didn't between SQLite 2 and SQLite 3. There also have been some (albeit minor) backwards compatibility issues with SQLite 3.x releases. The most serious of which deal with performance characteristics changing because they changed how the optimizer works. These type of things are acceptable to deal with in browser code because you can change your code unlike on the web (unless you want to have different code for each browser, and then each browser version). It's that, or browsers can ship one version of SQLite for all eternity. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 4/4/2011 10:28 AM, Joran Greef wrote: Do you think it would be wise then to advocate doing away with SQLite before IndexedDB has had a chance to prove itself? Surely two competing APIs would be the fastest way to bring IndexedDB up to speed? Who is advocating doing away with it? Note that I would not consider vendors not implementing it as doing away with it. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 06 Apr 2011, at 6:26 PM, Shawn Wilsher wrote: On 4/4/2011 8:07 AM, Joran Greef wrote: SQLite has a fantastic track record of maintaining backwards compatibility. Sort of. They didn't between SQLite 2 and SQLite 3. There also have been some (albeit minor) backwards compatibility issues with SQLite 3.x releases. The most serious of which deal with performance characteristics changing because they changed how the optimizer works. These type of things are acceptable to deal with in browser code because you can change your code unlike on the web (unless you want to have different code for each browser, and then each browser version). It's that, or browsers can ship one version of SQLite for all eternity. Cheers, Shawn We only need one fixed version of SQLite to be shipped across Chrome, Safari, Opera, Firefox and IE. That in itself would represent a tremendous goal for IndexedDB to target and to try and achieve. When it actually does, and surpasses the fixed version of SQLite, those developers requiring the raw performance and reliability of SQLite could then switch over. It is too soon to deprecate SQLite in the browser. IndexedDB is only getting started. It is beta and nowhere near the performance and test coverage of SQLite. A fixed version of SQLite across browsers would be helpful at this stage. If Mozilla could lead the way on this it would be fantastic. Perhaps that would satisfy all parties on these issues? It would also give IndexedDB implementors sufficient incentive to optimize their implementations, and developers the safety net of SQLite until such time as they do.
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 4/4/2011 10:18 AM, Joran Greef wrote: How would you create an index on an existing object store in IndexedDB containing more than 50,000 objects on an iPad, without incurring any object deserialization/serialization overhead, without being an order of magnitude slower than SQLite, and without bringing the iPad to its knees? If you can do it with even one IndexedDB implementation out there then kudos and hats off to you. :) You keep bringing this point up, but only a naive implementation of IndexedDB would bring a device to it's knees (or a poorly implemented thread scheduler, which I don't expect the iPad to have). The API is asynchronous, which means it doesn't need to (nor should it) happen on any thread that the UI is being drawn on. You still have a point about it possibly taking longer, but even then, that will be implementation dependent. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 06 Apr 2011, at 6:49 PM, Shawn Wilsher wrote: On 4/4/2011 10:18 AM, Joran Greef wrote: How would you create an index on an existing object store in IndexedDB containing more than 50,000 objects on an iPad, without incurring any object deserialization/serialization overhead, without being an order of magnitude slower than SQLite, and without bringing the iPad to its knees? If you can do it with even one IndexedDB implementation out there then kudos and hats off to you. :) You keep bringing this point up, but only a naive implementation of IndexedDB would bring a device to it's knees (or a poorly implemented thread scheduler, which I don't expect the iPad to have). The API is asynchronous, which means it doesn't need to (nor should it) happen on any thread that the UI is being drawn on. You still have a point about it possibly taking longer, but even then, that will be implementation dependent. Cheers, Shawn I bring up the iPad example because I had experience with a LocalStorage implementation (I think it was Safari) loading the contents of LocalStorage into memory synchronously on first access, blocking the UI thread. I am probably wrong on this one but I think I remember reading on Web Apps that this was one of the motivations behind limiting LocalStorage quota to around 10mb. At the time I was one of those who believed that LocalStorage would support storage of at least 10 GB as a matter of course. I hope you can understand my slight distrust of subsequent storage APIs (other than those of proven track record) in this light. It would still take longer (easily 30-50 seconds per 50,000 objects more than an opaque key-value store built on SQLite) even if the IndexedDB implementation was asynchronous. The developer would also have a tough time reasoning about when index migrations would be finished, since IndexedDB offers no control over the migration process and provides no way to modify index memberships directly. For those that care about these things, IndexedDB does not provide sufficient low-level storage primitives.
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 4/6/2011 9:44 AM, Joran Greef wrote: We only need one fixed version of SQLite to be shipped across Chrome, Safari, Opera, Firefox and IE. That in itself would represent a tremendous goal for IndexedDB to target and to try and achieve. When it actually does, and surpasses the fixed version of SQLite, those developers requiring the raw performance and reliability of SQLite could then switch over. I don't believe any browser vendor would be interested in shipping two different version of SQLite (one for internal use, and one for the web). I can say, with certainty, that Mozilla is not. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 4/6/2011 10:06 AM, Joran Greef wrote: I bring up the iPad example because I had experience with a LocalStorage implementation (I think it was Safari) loading the contents of LocalStorage into memory synchronously on first access, blocking the UI thread. I am probably wrong on this one but I think I remember reading on Web Apps that this was one of the motivations behind limiting LocalStorage quota to around 10mb. At the time I was one of those who believed that LocalStorage would support storage of at least 10 GB as a matter of course. I hope you can understand my slight distrust of subsequent storage APIs (other than those of proven track record) in this light. localStorage had a major design flaw that shipped and was then used before it was identified (at least that's my understanding of the timeline). That flaw is that it requires disk I/O to happen before (in webkit's case onload) or during the call to the API. IndexedDB (and WebSQL) were specifically designed to avoid this by doing the work asynchronously. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On Wed, Apr 6, 2011 at 10:14 AM, Shawn Wilsher sdwi...@mozilla.com wrote: On 4/6/2011 9:44 AM, Joran Greef wrote: We only need one fixed version of SQLite to be shipped across Chrome, Safari, Opera, Firefox and IE. That in itself would represent a tremendous goal for IndexedDB to target and to try and achieve. When it actually does, and surpasses the fixed version of SQLite, those developers requiring the raw performance and reliability of SQLite could then switch over. I don't believe any browser vendor would be interested in shipping two different version of SQLite (one for internal use, and one for the web). I can say, with certainty, that Mozilla is not. In addition, as previously stated, the near certainty that there is, hidden somewhere in the code, some security bugs (there are *always* security bugs) means that browsers can not/will not ship a single fixed version. When a security bug is encountered, either the browsers update to a new version of sqlite (if it's already been fixed), thus potentially breaking sites, or they patch sqlite and then upgrade to the patched version, thus potentially breaking sites, or they fork sqlite and patch the error only in their forked version, still potentially breaking sites but also forking the project. The only thing that is *not* a valid possibility is the browsers staying on the single fixed version, thus continuing to expose their users to the security bug. ~TJ
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 06 Apr 2011, at 7:14 PM, Shawn Wilsher wrote: On 4/6/2011 9:44 AM, Joran Greef wrote: We only need one fixed version of SQLite to be shipped across Chrome, Safari, Opera, Firefox and IE. That in itself would represent a tremendous goal for IndexedDB to target and to try and achieve. When it actually does, and surpasses the fixed version of SQLite, those developers requiring the raw performance and reliability of SQLite could then switch over. I don't believe any browser vendor would be interested in shipping two different version of SQLite (one for internal use, and one for the web). I can say, with certainty, that Mozilla is not. Cheers, Shawn If Mozilla enjoys using the latest version of SQLite (and I assume they are not planning on replacing internal SQLite embeddings with IndexedDB - not at this stage at least), then web developers deserve the latest version. Ship the latest version of SQLite (even with the -moz prefix). Developers targeting HTML 5 are used to API changes, waiting on browsers and trying to reason about broken implementations. The library writers will quickly grow over any SQLite version changes should they even ever arise. Would you run the Mozilla production database on any browser's implementation of IndexedDB? How can you expect developers to run their production client code on IndexedDB? It's simply not ready and will not be for at least a year or two or three. How likely is it that SQLite (given it's history) will remove the SELECT, INSERT, UPDATE, DELETE statements before then?
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 06 Apr 2011, at 7:24 PM, Tab Atkins Jr. wrote: When a security bug is encountered, either the browsers update to a new version of sqlite (if it's already been fixed), thus potentially breaking sites, or they patch sqlite and then upgrade to the patched version, thus potentially breaking sites, or they fork sqlite and patch the error only in their forked version, still potentially breaking sites but also forking the project. The only thing that is *not* a valid possibility is the browsers staying on the single fixed version, thus continuing to expose their users to the security bug. ~TJ Browser vendors are moving to shorter and shorter release cycles. People have stopped viewing these things through the IE6-here-forever lens. Browsers are starting to update themselves automatically, even nightly. If a security issue were to be found, it would be highly unlikely that its patch would break any SQL interface of SQLite.
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 4/6/11 10:30 AM, Joran Greef wrote: If Mozilla enjoys using the latest version of SQLite (and I assume they are not planning on replacing internal SQLite embeddings with IndexedDB - not at this stage at least), then web developers deserve the latest version. This is not obvious a priori, for what it's worth. Mozilla also enjoys using self-modifying machine code [1], for example. Yet we have no plans to expose that capability directly to the web. Ship the latest version of SQLite (even with the -moz prefix). In case this was not clear, we think that this would be actively bad for the web, not least by increasing the barrier to entry into the web space for new browsers. -Boris [1] http://en.wikipedia.org/wiki/Inline_caching
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 06 Apr 2011, at 7:42 PM, Boris Zbarsky wrote: On 4/6/11 10:30 AM, Joran Greef wrote: If Mozilla enjoys using the latest version of SQLite (and I assume they are not planning on replacing internal SQLite embeddings with IndexedDB - not at this stage at least), then web developers deserve the latest version. This is not obvious a priori, for what it's worth. The point was made with reference to Mozilla expecting web developers to run production client code on IndexedDB, when Mozilla themselves run production code on SQLite. Boris, Jonas and Shaun, we could talk round and round in circles. It seems you're not too concerned by any of the performance and design problems re: indexedDB that I have raised. You ask for proposals but it's clear you're not sold on these issues. If you were, I am sure you would be among the first to provide them. Do you have real-world experience developing web-based applications, targeting mobile and desktop, with offline support for storing, indexing, migrating and synchronizing several million objects? Or are we all arguing in the realm of conjecture (it should be able to) without having encountered any of these issues ourselves, or having any basis for our claims?
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On Wed, Apr 6, 2011 at 10:42 AM, Boris Zbarsky bzbar...@mit.edu wrote: On 4/6/11 10:30 AM, Joran Greef wrote: If Mozilla enjoys using the latest version of SQLite (and I assume they are not planning on replacing internal SQLite embeddings with IndexedDB - not at this stage at least), then web developers deserve the latest version. This is not obvious a priori, for what it's worth. Mozilla also enjoys using self-modifying machine code [1], for example. Yet we have no plans to expose that capability directly to the web. Ship the latest version of SQLite (even with the -moz prefix). In case this was not clear, we think that this would be actively bad for the web, not least by increasing the barrier to entry into the web space for new browsers. For what it's worth, this thread is very much going in circles. I've listed several reasons why I'm not supporting shipping WebSQL in Firefox. Nothing in this thread has persuaded me otherwise. In particular some sort of competition between IndexedDB and WebSQL doesn't change that in the least. Further, there have have been claims of various blockers for implementing something like RelationalDB or even WebSQL on top of IndexedDB. So far these claims are unsubstantiated. I really don't think this thread is going to go much further than this. I'd love for someone to implement a higher-level API on top of IndexedDB. This will help us find shortcomes in the API, and bottlenecks in the implementation. But until someone actually does that, all we have is unsubstantiated guesses. Others should feel free to continue this thread, but I doubt I will. / Jonas
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
That's not the only reason. Mozilla laid out others ten months ago: https://hacks.mozilla.org/2010/06/beyond-html5-database-apis-and-the-road-to-indexeddb/ Mozilla's plan appears to be to implement IndexedDB on top of SQLite, and then encourage developers to build SQL in javascript on top of IndexedDB. That seems like a ludicrous hoop to jump through to avoid letting SQLite become an ad-hoc standard, not to mention a terrible performance killer for the sake of... what, exactly? Why not just expose the thing and let developers worry about whether what they're using is standard or not, and suffer the consequences later, if in fact there are any? It seems to me that the proper role for standards is to step in and help clear things up when they get messy, not to prevent developers from accessing useful tools. If SQLite offers a consistent SQL interface for the next decade, this will all seem very foolish, and if they don't, nothing's stopping a fork from maintaining whatever standard has evolved up to that point, or making improvements to make it more suitable for exposing to developers through the browser. Ryan
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 4/4/2011 10:15 AM, Ryan Fugger wrote: That's not the only reason. Mozilla laid out others ten months ago: https://hacks.mozilla.org/2010/06/beyond-html5-database-apis-and-the-road-to-indexeddb/ Mozilla's plan appears to be to implement IndexedDB on top of SQLite, and then encourage developers to build SQL in javascript on top of IndexedDB. That seems like a ludicrous hoop to jump through to avoi They have no intention of exposing SQLite to the untrusted environment, having it in a standards doc wouldn't change that. SQL has been done-before in JS, I don't see what Mozilla's enthusiasm about an SQL+IDB layer is about. http://code.google.com/p/trimpath/wiki/TrimQuery I'm a big fan of sqlite. I love public domain software. I think it's a great idea. And this is no slight on sqlite; it does not implement some exotic features that really make SQL powerful. Windowing functions really cut down on code: http://developer.postgresql.org/pgdocs/postgres/tutorial-window.html http://developer.postgresql.org/pgdocs/postgres/functions-window.html I do like the RelationalDB idea being floated, I'd like to see more energy put into that proposal. It looks like something that's easier to implement on top of JS objects, SQLite, or ArrayBuffer, than starting with the SQLite SQL dialect. It's still complicated, in that it may easily be fed large joins that will grind the computer to a halt, so that's something that needs to be taken into account. I'm happier working with IDB than SQLite. I'm happier with IDB than the FileSystem API, too. It's just easier for the items I work on. And I'm happy that webkit has SQLite available in their public api, because I've fallen back to it many times for data storage. But I just use it in key value pair. -Charles
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 4/4/11 10:15 AM, Ryan Fugger wrote: That's not the only reason. Mozilla laid out others ten months ago: https://hacks.mozilla.org/2010/06/beyond-html5-database-apis-and-the-road-to-indexeddb/ Mozilla's plan appears to be to implement IndexedDB on top of SQLite, This is not a plan so much as a current stopgap. Why not just expose the thing and let developers worry about whether what they're using is standard or not, and suffer the consequences later, if in fact there are any? Because due to the incentive structure here the entities making the bad decisions are not the ones who end up having to suffer the consequences. It's a typical case of externalizing the negatives. It seems to me that the proper role for standards is to step in and help clear things up when they get messy Often it's too late by then. -Boris
RE: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
From: public-webapps-requ...@w3.org [mailto:public-webapps-requ...@w3.org] On Behalf Of Keean Schupke Sent: Monday, April 04, 2011 10:17 PM Something like RelationalDB gives you the power of a relational-db with no dependence on a specific implementation of SQL, so it would be compatible enough for the web. It fixes all the problems with the standardisation of WebSQL that have been talked about so far. I think it would find no technical issues that block its standardisation. As a high level DB API it does not need all the low-level features of IndexedDB, so its API can be much simpler and cleaner. RelationalDB can at least be provided as a library on top of IndexedDB, and it can use WebSQL where it is supported. My concern with the library approach is performance when implemented on top of IndexedDB. The goal of IndexedDB has always been to enable things like RelationalDB and CouchDB to be built on top, while maintaining a reasonable level of functionality for those that wanted to use it directly. I really like the idea of thinking of RelationalDB as something that's built as a library on top of IndexedDB. Are there specific tweaks we can make to IndexedDB so it can be a good lower-layer for RelationalDB, such that RelationalDB could be built as a pure JavaScript library? Thanks -pablo
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 06 Apr 2011, at 2:53 AM, Pablo Castro wrote: The goal of IndexedDB has always been to enable things like RelationalDB and CouchDB to be built on top, while maintaining a reasonable level of functionality for those that wanted to use it directly. I really like the idea of thinking of RelationalDB as something that's built as a library on top of IndexedDB. Are there specific tweaks we can make to IndexedDB so it can be a good lower-layer for RelationalDB, such that RelationalDB could be built as a pure JavaScript library? Thanks -pablo 1. Treat object values as opaque (necessary to avoid deserialization/serialization overhead, this is mandatory for storing anything over 50,000 objects on a device like an iPad or iPhone). 2. Enable indices to be modified at time of putting/deleting objects (index references provided by application at time of putObject/deleteObject call). 3. Provide a simpler, more powerful locking mechanism, opaque to IndexedDB, to provide finer-grained application-specific locking (i.e. have we just entered into a sync process with the master database). If I may say so, it does seem odd that some would advocate the difficulties of speccing merely the interface of something like SQLite, and then advise others to suggest re-implementing it entirely. If there was a specific BTree API in the browser and a powerful asynchronous sLocalStorage mechanism this might be something for the brave, but IndexedDB is a little too tightly coupled to it's own interface agenda at the moment to make this goal possible.
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 04 Apr 2011, at 4:39 PM, Jonas Sicking wrote: Hence it would still be the case that we would be relying on the SQLite developers to maintain a stable SQL interpretation... SQLite has a fantastic track record of maintaining backwards compatibility. IndexedDB has as yet no track record, no consistent implementations, no widespread deployment, only measurably poor performance and a lukewarm indexing and querying API. If anything it's the other way round. You have yet to convince developers that IndexedDB will be faster, more stable, more powerful, more memory efficient than SQLite and with better test coverage at that.
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
This is ignoring the possibility that something like RelationalDB could be used, where a well defined common subset of SQL can be used (and I use well-defined in the formal sense). This would allow a relatively thin wrapper on top of most SQL implementations and would allow SQLite (or BDB) to be used as the backend. As a seasoned C++ programmer, I could even write a Firefox plugin using XPCOM as a reference implementation using the same API as the JavaScript RelationalDB implementation on my GitHub. Although I am not keen on putting in the time to do this if nobody is interested. To me is seems this thread is going in circles. RelationalDB does not have the standardisation problem that WebSQL has, but is still a relatively thin API layer that can be implemented over the top of a fast and well tested SQL implementation. It is based on sound theory and research defining the abstraction layer, and has a relationally complete API, so there should be no need to change the core API in the development of a standard. Cheers, Keean. On 4 April 2011 14:39, Jonas Sicking jo...@sicking.cc wrote: On Saturday, April 2, 2011, Joran Greef jo...@ronomon.com wrote: I am incredibly uncomfortable with the idea of putting the responsibility of the health of the web in the hands of one project. In fact, one of the main reasons I started working at Mozilla was to prevent this. / Jonas I agree with you. All the more reason to support both WebSQL and IndexedDB. It is not a case of either/or. It would be healthy to have competing APIs. Competition might be a great thing. But it doesn't address the issue in the least. It would still be the case that some developers would choose to use WebSQL, and browser makers would still have to support it, including support the SQL dialect it uses. Hence it would still be the case that we would be relying on the SQLite developers to maintain a stable SQL interpretation to keep a healthy and functional web. / Jonas
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 04 Apr 2011, at 5:26 PM, Keean Schupke wrote: This is ignoring the possibility that something like RelationalDB could be used, where a well defined common subset of SQL can be used (and I use well-defined in the formal sense). This would allow a relatively thin wrapper on top of most SQL implementations and would allow SQLite (or BDB) to be used as the backend. Yes, if an implementation of RelationalDB arrives which is solid and fast with support for set operations that would be great. The important thing is that we have two competing APIs (and preferably a strong API with a great track record).
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
Yes, it already has well defined set operations. Solid is a matter of
testing by enough people (and if you wanted to try it and feed back that
would be a start). Fast should not be a problem, as the SQL database does
all the heavy lifting.
In more detail, Codd's six primitive operators are project, restrict,
cross-product, union and difference. Relations are an extension of Sets, so
intersection and difference on compatible relations behave like they would
on sets.
RelationalDB already implements the following 5 methods making it
relationally-complete. Meaning it can do anything you could possibly want to
do with relations using combinations of these 5 methods.
Relation.prototype.project = function(attributes) { // this implements
rename as well.
Relation.prototype.restrict = function(exp)
Relation.prototype.join = function(relation, exp) {
Relation.prototype.union = function() {};
Relation.prototype.difference = function() {};
Of course some things can be made easier, so the following methods, although
they can be defined in terms of the above 5, will be provided (in future
implementations) to keep user code concise and implementations thin and
fast.
// derived methods
Relation.prototype.intersection = function() {};
Relation.prototype.thetajoin = function() {};
Relation.prototype.semijoin = function() {};
Relation.prototype.antijoin = function() {};
Relation.prototype.divide = function() {};
Relation.prototype.leftjoin = function() {};
Relation.prototype.rightjoin = function() {};
Relation.prototype.fulljoin = function() {};
We also hope to provide the lattice operators meet and join:
http://en.wikipedia.org/wiki/Lattice_(order)
Just these two operators can replace all 5 of Codd's primitives (including
all set operations). With just these two you can do anything that you can
with _all_ the above. Meet is actually the same as Codd's natural-join
(unfortunately terminology in different mathematical fields is not
consistent here) and Join is a generalised union operator.
See:
http://www.arxiv.com/pdf/cs/0501053v2
To see how Meet and Join can be used to construct each of the above
operators.
Cheers,
Keean.
On 4 April 2011 15:36, Joran Greef jo...@ronomon.com wrote:
On 04 Apr 2011, at 5:26 PM, Keean Schupke wrote:
This is ignoring the possibility that something like RelationalDB could
be used, where a well defined common subset of SQL can be used (and I use
well-defined in the formal sense). This would allow a relatively thin
wrapper on top of most SQL implementations and would allow SQLite (or BDB)
to be used as the backend.
Yes, if an implementation of RelationalDB arrives which is solid and fast
with support for set operations that would be great. The important thing is
that we have two competing APIs (and preferably a strong API with a great
track record).
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 4 April 2011 15:55, Keean Schupke ke...@fry-it.com wrote: Yes, it already has well defined set operations. Solid is a matter of testing by enough people (and if you wanted to try it and feed back that would be a start). Fast should not be a problem, as the SQL database does all the heavy lifting. In more detail, Codd's six primitive operators are project, restrict, cross-product, union and difference. Relations are an extension of Sets, so intersection and difference on compatible relations behave like they would on sets. I missed 'rename' from my list of Codd's operators. Our 'project' function provides both project rename, so I overlooked it. ... Cheers, Keean.
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
i've mostly stayed out of this thread because i felt like i'd just being fanning the flames but i really can't stay out anymore. databases are more that SQL, always have been. SQL is a DSL for relational database access. all implementations of SQL have a similar set of tools they implement first and layer SQL on top of. those tools tend to be a storage engine, btree, and some kind of transactional model between them. under the ugly covers, most databases look like berkeleydb and the layer you live in is just sugar on top. creating an in-browser specification/implementation on top of a given relational/SQL story is a terrible idea. it's unnecessarily limiting to a higher level api and can't be easily extended the way a simple set of tools like IndexedDB can. suggesting that other databases be implemented on top of SQL rather than on top of the tools in which SQL is built is just backwards to anyone who's built a database. it's not very hard to write the abstraction you're talking about on top of IndexedDB, and until you do it i'm going to have a hard time taking you seriously because it's clearly doable. i implemented a CouchDB compatible datastore on top of IndexedDB, it took me less than a week at a time when there was only one implementation that was still changing and still had bugs. it would be much easier now. https://github.com/mikeal/idbcouch it needs to be updated to use the latest version of the spec which is a day of work i just haven't gotten to yet. the constructs in IndexedDB are pretty low level but sufficient if you know how to implement databases. performance is definitely an issue, but making these constructs faster would be much easier than trying to tweak an off the shelf SQL implementation to your use case.
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On Mon, Apr 4, 2011 at 8:07 AM, Joran Greef jo...@ronomon.com wrote: On 04 Apr 2011, at 4:39 PM, Jonas Sicking wrote: Hence it would still be the case that we would be relying on the SQLite developers to maintain a stable SQL interpretation... SQLite has a fantastic track record of maintaining backwards compatibility. IndexedDB has as yet no track record, no consistent implementations, no widespread deployment, It's new. only measurably poor performance Ironically, the poor performance is because it's using sqlite as a backing-store in the current implementation. That's being fixed by replacing sqlite. and a lukewarm indexing and querying API. Kinda the point, in that the power/complexity of SQL confuses a huge number of develoeprs, who end up coding something which doesn't actually use the relational model in any significant way, but still pays the cost of it in syntax. (I found normalization forms and similar things completely trivial when I was learning SQL, but for some reason almost every codebase I've looked at has a horribly-structured db. As far as I can tell, the majority of developers just hack SQL into being a linear object store and do the rest in their application code. We can reduce the friction here by actually giving them a linear object store, which is what IndexedDB is.) ~TJ
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
Some thoughts: On 4 April 2011 16:10, Mikeal Rogers mikeal.rog...@gmail.com wrote: i've mostly stayed out of this thread because i felt like i'd just being fanning the flames but i really can't stay out anymore. databases are more that SQL, always have been. SQL is a DSL for relational database access. all implementations of SQL have a similar set of tools they implement first and layer SQL on top of. those tools tend to be a storage engine, btree, and some kind of transactional model between them. under the ugly covers, most databases look like berkeleydb and the layer you live in is just sugar on top. SQL is a standard language (or API) for talking to databases. Why should a developer need to learn a different API for each database. W3 is about standardising APIs. SQL is just an API standardised as a DSL. It is good for all the reasons any standard is good. Add to that the sound mathematical theory of relational-algebra, means it has a lot going for it. Although like any standard is has its problems, most of those seem to be where it has deviated away from the pure relational-algebra. creating an in-browser specification/implementation on top of a given relational/SQL story is a terrible idea. it's unnecessarily limiting to a higher level api and can't be easily extended the way a simple set of tools like IndexedDB can. Its not limiting it provides a more powerful (higher level) interface, that allows developers to concentrate on what to do with the data not how to do it. suggesting that other databases be implemented on top of SQL rather than on top of the tools in which SQL is built is just backwards to anyone who's built a database. RelationalDB is not a database its a relational-data model. it's not very hard to write the abstraction you're talking about on top of IndexedDB, and until you do it i'm going to have a hard time taking you seriously because it's clearly doable. Surely its the API that is important, not how it is implemented? You can try the API now implemented on top of WebSQL. The API will stay the same no matter what underlying technology is used. i implemented a CouchDB compatible datastore on top of IndexedDB, it took me less than a week at a time when there was only one implementation that was still changing and still had bugs. it would be much easier now. https://github.com/mikeal/idbcouch it needs to be updated to use the latest version of the spec which is a day of work i just haven't gotten to yet. I am not overly impressed by CouchDB. the constructs in IndexedDB are pretty low level but sufficient if you know how to implement databases. performance is definitely an issue, but making these constructs faster would be much easier than trying to tweak an off the shelf SQL implementation to your use case. I look at the amount of man hours that have gone into developing SQLite, and BDB and I think, hey if its so easy to write a high performance database, those guys must have been wasting a lot of time? Cheers, Keean.
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 4 April 2011 16:04, Tab Atkins Jr. jackalm...@gmail.com wrote: On Mon, Apr 4, 2011 at 8:07 AM, Joran Greef jo...@ronomon.com wrote: On 04 Apr 2011, at 4:39 PM, Jonas Sicking wrote: Hence it would still be the case that we would be relying on the SQLite developers to maintain a stable SQL interpretation... SQLite has a fantastic track record of maintaining backwards compatibility. IndexedDB has as yet no track record, no consistent implementations, no widespread deployment, It's new. only measurably poor performance Ironically, the poor performance is because it's using sqlite as a backing-store in the current implementation. That's being fixed by replacing sqlite. and a lukewarm indexing and querying API. Kinda the point, in that the power/complexity of SQL confuses a huge number of develoeprs, who end up coding something which doesn't actually use the relational model in any significant way, but still pays the cost of it in syntax. (I found normalization forms and similar things completely trivial when I was learning SQL, but for some reason almost every codebase I've looked at has a horribly-structured db. As far as I can tell, the majority of developers just hack SQL into being a linear object store and do the rest in their application code. We can reduce the friction here by actually giving them a linear object store, which is what IndexedDB is.) ~TJ SQLite has seen really good use in the mobile app community on both iPhone and Android. I would have thought that if we wanted the same kind of thriving app developer community around HTML5 web-apps, taking a few leaves from the mobile developers book would not be a bad idea? IMHO its those kind of developers HTML5 should be trying to attract, in addition to the current web developers. Cheers, Keean.
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
I would point out that RelationalDB is relationally complete and the api does not depend on the sqlite spec at all. Cheers Keean On Apr 1, 2011 8:58 PM, Jonas Sicking jo...@sicking.cc wrote: On Fri, Apr 1, 2011 at 12:39 PM, Glenn Maynard gl...@zewt.org wrote: Lastly, some vendors have expressed unwillingness to embed SQLite for legal reasons. Embedding other peoples code definitely exposes you to risk of copyright and patent lawsuits. While I can't say that I fully agree with this reasoning, I'm also not the one that would be on the receiving end of a lawsuit. Nor am I a lawyer and so ultimately will have to defer to people that know better. In the end it doesn't really matter as if a browser won't embed SQLite then it doesn't matter why, the end result is that the same SQL dialect won't be available cross browser which is bad for the web. If SQLite was to be used as a web standard, I'd hope that it wouldn't show up in a spec as simply do what SQLite does, but as a complete spec of SQLite's behavior. Browser vendors could then, if their lawyers insisted, implement their own compatible implementation, just as they do with other web APIs. I'd expect large portions of SQLite's test suite to be adaptable as a major starting point for spec tests, too. Have you read the WebSQL spec? Creating such a spec would be a formidable task, of course. Indeed. One that the SQL community has failed in doing so far. And they have a lot more experience with SQL than we do. / Jonas
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 04 Apr 2011, at 6:10 PM, Mikeal Rogers wrote: it's not very hard to write the abstraction you're talking about on top of IndexedDB, and until you do it i'm going to have a hard time taking you seriously because it's clearly doable. You assume I have not written the abstraction I am talking about on top of IndexedDB? the constructs in IndexedDB are pretty low level but sufficient if you know how to implement databases. performance is definitely an issue, but making these constructs faster would be much easier than trying to tweak an off the shelf SQL implementation to your use case. How exactly would you make a schema-enforcing interface faster than a stateless interface? How would you implement application-managed indices on top of IndexedDB without being slower than SQLite? How would you implement set operations on indices in IndexedDB without being slower or less memory efficient than SQLite? How would you create an index on an existing object store in IndexedDB containing more than 50,000 objects on an iPad, without incurring any object deserialization/serialization overhead, without being an order of magnitude slower than SQLite, and without bringing the iPad to its knees? If you can do it with even one IndexedDB implementation out there then kudos and hats off to you. :) I understand your point of view. I once thought the same. You would think that IndexedDB would be more than satisfactory for these things. The question is whether IndexedDB provides adequate and performant database primitives, to the same degree as SQLite (and of course SQL is merely an interface to database storage primitives, I do not recalling saying otherwise). You can build IndexedDB on top of SQLite (as some browsers are indeed doing), but you cannot build SQLite on IndexedDB.
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
Joran Greef April 4, 2011 April 4, 201110:18 AM On 04 Apr 2011, at 6:10 PM, Mikeal Rogers wrote: it's not very hard to write the abstraction you're talking about on top of IndexedDB, and until you do it i'm going to have a hard time taking you seriously because it's clearly doable. You assume I have not written the abstraction I am talking about on top of IndexedDB? the constructs in IndexedDB are pretty low level but sufficient if you know how to implement databases. performance is definitely an issue, but making these constructs faster would be much easier than trying to tweak an off the shelf SQL implementation to your use case. How exactly would you make a schema-enforcing interface faster than a stateless interface? How would you implement application-managed indices on top of IndexedDB without being slower than SQLite? this assumes your indexes can be describe well in sqlite and that you want them generated at write time. one of the performance optimizations couchdb makes is to generate secondary indexes at read time which is much more efficient as you can batch the processing and do a single bulk transaction which limits IO. if you need, exactly, the indexes sqlite provides then i'm sure they will be well optimized in sqlite, but for everyone else. How would you implement set operations on indices in IndexedDB without being slower or less memory efficient than SQLite? you can do pretty much anything if you write a good abstraction and leverage transactions. if it's too slow complain to the vendor and it'll get improved, that's how we've been improving performance in the browser for many, many years and it seems to be working. How would you create an index on an existing object store in IndexedDB containing more than 50,000 objects on an iPad, without incurring any object deserialization/serialization overhead, without being an order of magnitude slower than SQLite, and without bringing the iPad to its knees? If you can do it with even one IndexedDB implementation out there then kudos and hats off to you. :) the biggest bottleneck here in the current implementation would be the transaction overhead on a database this size, which is because of performance problems in sqlite which is underlying the implementation. sqlite can't fix this, it's currently the problem. the object serialization is not a huge performance issue, performance issues in databases are almost always do to IO or transaction locks. I understand your point of view. I once thought the same. You would think that IndexedDB would be more than satisfactory for these things. The question is whether IndexedDB provides adequate and performant database primitives, to the same degree as SQLite (and of course SQL is merely an interface to database storage primitives, I do not recalling saying otherwise). sqlite doesn't expose it's primitives well, it exposes it's abstractions. berkeleydb exposes it's primitives well, and you could make the case that it would be a better thing to standardize on, although i wouldn't think that would be a great idea either. You can build IndexedDB on top of SQLite (as some browsers are indeed doing), but you cannot build SQLite on IndexedDB. you should most definitely be able build sqlite on top of IDB, there would be a performance penalty of course, which we can address, but you should be able to do it. if you can't then we need to extend the specification. Keean Schupke April 4, 2011 April 4, 20118:55 AM Yes, it already has well defined set operations. Solid is a matter of testing by enough people (and if you wanted to try it and feed back that would be a start). Fast should not be a problem, as the SQL database does all the heavy lifting. In more detail, Codd's six primitive operators are project, restrict, cross-product, union and difference. Relations are an extension of Sets, so intersection and difference on compatible relations behave like they would on sets. RelationalDB already implements the following 5 methods making it relationally-complete. Meaning it can do anything you could possibly want to do with relations using combinations of these 5 methods.
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 04 Apr 2011, at 6:04 PM, Tab Atkins Jr. wrote: It's new. Do you think it would be wise then to advocate doing away with SQLite before IndexedDB has had a chance to prove itself? Surely two competing APIs would be the fastest way to bring IndexedDB up to speed? Ironically, the poor performance is because it's using sqlite as a backing-store in the current implementation. That's being fixed by replacing sqlite. Yes I am aware of this. There are some design flaws in IndexedDB. For instance, it does not regard objects as opaque (as would a typical key-value store), which means that creating an index on an existing object store would require deserializing/serializing every object therein. Doing that for 50,000 objects on an iPad would be breathtaking. I have written object stores on top of SQLite and they are already an order of magnitude faster than IndexedDB with a more powerful and memory efficient API to boot. Kinda the point, in that the power/complexity of SQL confuses a huge number of develoeprs, who end up coding something which doesn't actually use the relational model in any significant way, but still pays the cost of it in syntax. I was not referring to SQL but to the underlying primitives exposed through the SQL interface. For example, set operations on indices, or the ability to index objects with array values.
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 4 April 2011 22:55, Aryeh Gregor simetrical+...@gmail.com wrote: On Fri, Apr 1, 2011 at 2:39 PM, Jonas Sicking jo...@sicking.cc wrote: There are several reasons why we don't want to rely exclusively on SQLite, other than solely W3C formalia. First of all, what should we do once the SQLite team releases a new version which has some modifications in its SQL dialect? We generally always need to embed the latest version of the library since it contains critical bug fixes, however SQLite makes no guarantees that it will always support exactly the same SQL statements. . . . These are good reasons, and I have no problem with them. SQLite is designed with very different compatibility and security needs than the web platform has, and its performance goals might be different in some respects as well. There are various ways that you could address this short of making up something completely different, but I'm not sure whether it would be a good idea. Anyway, I didn't intend to reignite this whole discussion. The decision has been made, now we get to see what comes of it. On Mon, Apr 4, 2011 at 11:07 AM, Joran Greef jo...@ronomon.com wrote: SQLite has a fantastic track record of maintaining backwards compatibility. SQLite does not face anything close to the compatibility requirements that web browsers face. There are perhaps billions of independent web pages, which don't have any control over what browser versions they're being run in. These pages are expected to work in all browsers even if they were written ten years ago and no one has looked at them since, and even if they were written incompetently. Just because something has an excellent compatibility track record by the standards of application libraries doesn't mean it's compatible enough for the web. Something like RelationalDB gives you the power of a relational-db with no dependence on a specific implementation of SQL, so it would be compatible enough for the web. It fixes all the problems with the standardisation of WebSQL that have been talked about so far. I think it would find no technical issues that block its standardisation. As a high level DB API it does not need all the low-level features of IndexedDB, so its API can be much simpler and cleaner. RelationalDB can at least be provided as a library on top of IndexedDB, and it can use WebSQL where it is supported. My concern with the library approach is performance when implemented on top of IndexedDB. Cheers, Keean.
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
I agree that it'd be best to have a spec independent of database platform, which is why I was asking about an idea along the lines of RelationalDB https://github.com/keean/RelationalDBor the example I gave in the email which initiated this discussion, both of which are entirely abstracted from the underlying database (SQLite or otherwise - doesn't even have to be SQL). Was anything like this considered before WebSQL was canned? Also, +1 on looking at a FireFox plugin to drum up some interest on this. On 2 April 2011 05:32, Ian Hickson i...@hixie.ch wrote: On Fri, 1 Apr 2011, Arthur Barstow wrote: On Apr/1/2011 3:39 PM, ext Glenn Maynard wrote: If SQLite was to be used as a web standard, I'd hope that it wouldn't show up in a spec as simply do what SQLite does, but as a complete spec of SQLite's behavior. FYI, the Web SQL Database NOTE says: [[ http://www.w3.org/TR/2010/NOTE-webdatabase-20101118/#web-sql 5. Web SQL User agents must implement the SQL dialect supported by Sqlite 3.6.19. ]] and I don't recall anyone ever committing to creating the later. Actually I did, multiple times. But nobody was interested in reimplementing that dialect independently, so all the points Jonas raised still apply. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On Sat, Apr 2, 2011 at 00:42:40, Glenn Maynard wrote: You can certainly ask if they're interested in doing so, not for our benefit (whoever our means), but for the benefit of the Web as a whole, and there's nothing at all rude in asking. I'd say the opposite: it's rude to assume they wouldn't be interested, rather than asking and letting them come to their own decision. (I don't know where the notion of forcing them to do anything came from.) I have been reading up more on the history of SQLite. It is a stellar implementation, just to highlight a few points: 1. Most of the SQLite source code is devoted purely to testing and verification. An automated test suite runs millions and millions of test cases involving hundreds of millions of individual SQL statements and achieves 100% branch test coverage. 2. SQLite can also be made to run in minimal stack space (4KiB) and very little heap (100KiB), making SQLite a popular database engine choice on memory constrained gadgets such as cellphones, PDAs, and MP3 players. 3. Faster than popular client/server database engines for most common operations. 4. Supports terabyte-sized databases and gigabyte-sized strings and blobs. 5. The developers continue to expand the capabilities of SQLite and enhance its reliability and performance while maintaining backwards compatibility with the published interface spec, SQL syntax, and database file format. It is easier to build a performant IndexedDB on SQLite than to build a performant SQLite on IndexedDB. Maybe that is something to think about. Developers need working database primitives, more than they need convenience. There may be conjectural reasons for Mozilla not implementing WebSQL, but the track history of SQLite is hard to ignore. Mozilla is already embedding SQLite for other uses, and appears to be a sponsor of the project. SQLite may not be a specification in our sense of the word, but in a Web sense of the word, it is so widely deployed already that it would be hard not to call it a standard.
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
I am incredibly uncomfortable with the idea of putting the responsibility of the health of the web in the hands of one project. In fact, one of the main reasons I started working at Mozilla was to prevent this. / Jonas I agree with you. All the more reason to support both WebSQL and IndexedDB. It is not a case of either/or. It would be healthy to have competing APIs.
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
Pity. Anyway RelationalDB defines its API without reference to the underlying SQL or non-SQL database... So as a candidate for replacing WebSQL, it does not suffer from that problem. Cheers, Keean. On 2 April 2011 14:56, Glenn Maynard gl...@zewt.org wrote: On Sat, Apr 2, 2011 at 5:24 AM, Keean Schupke ke...@fry-it.com wrote: Infact now BDB supports the SQLite-3.0 API, you can have two implementation that conform to the same API. So the original reason for abandoning WebSQL seems no longer valid. As there are now more than one implementation of the SQLite-3.0 API it is a de-facto (open) standard. Based on http://download.oracle.com/docs/cd/E17076_02/html/installation/upgrade_11gr2_51_sqlite_ver.html, it's not like an independent implementation. -- Glenn Maynard
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On Thu, Mar 31, 2011 at 12:28 PM, Boris Zbarsky bzbar...@mit.edu wrote: No, it actually sounds like a success; it prevented a specification being created which would have been tied to a particular implementation, no matter how widely-deployed. For comparison, IE6 was very widely deployed circa 2002. And yet specifying CSS, say, by saying just do exactly what IE6 does would not have been a good idea. Neither is defining WebSQL to do exactly what some particular version of SQLite does, and no one stepped up to define it better. IE6 is closed-source software written for a single platform. SQLite is in the public domain, works for all major operating systems and lots of minor ones, and is already used (I think?) by every major browser except IE. That makes all the difference. There's some benefit to having multiple interoperable implementations even if the reference implementation is public-domain, but enormously less than when the only implementations are controlled by particular parties. I think SQLite isn't a great long-term solution, because in the long term a single implementation stifles innovation. But it's also very powerful, pretty familiar to web developers (who are used to MySQL), requires vastly less implementation effort than making up a new system, and is in practice going to be much more interoperable than anything written separately by different browsers. So refusing to consider it just because there isn't a written standard or multiple interoperable implementations is not sound. When the Wikimedia Foundation was considering an official Board-approved policy for what file formats they'd allow for upload on sites like Wikimedia, the draft http://meta.wikimedia.org/wiki/File_format_policy required that the format be Defined by an open standard, implementation, or specification not under proprietary control. Notice that an open *standard* was not required -- an open *implementation* was enough. .doc need not apply, but LaTeX or MediaWiki wikitext are perfectly fine. The proposal was never formally adopted, but I think this philosophy makes a lot of sense. Many (although certainly not all) of the reasons for standards evaporate when you have a reference implementation under a BSD-style license or better. So if the only objection to WebSQL is there's no way we're going to get a formal spec or two interoperable implementations, I'd really encourage objectors to step back and ask themselves why they *want* a formal spec and two interoperable implementations. Those requirements are not axiomatic, they're means to obtain practical ends like allowing competitions and avoiding user lock-in. How many of those ends are really contrary to using SQLite as a de facto standard, and do the remaining ones really outweigh the practical advantages? (I came to this discussion very late, so apologies if there are additional objections to WebSQL that no one's mentioned in this specific thread, but have mentioned in the dozens of threads about this before now.)
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 4/1/2011 5:40 AM, Nathan Kitchen wrote: Are there any browser vendor representatives on the mailing list who would care to comment on the criteria for implementing something akin to Keean's RelationalDBhttps://github.com/keean/RelationalDB idea? What would need to be in place to start work on such an implementation? It wouldn't be terribly difficult to prototype this as an add-on for Firefox, I don't think (and I'd be happy to provide technical assistance to anyone wishing to do so). Doing this would allow web developers to install the add-on and play with it, which can give us useful feedback. I'm not saying we'd move it into the tree at that point, but it's a good first step to building a case to take it. 1. Opportunity to explore more solutions to offline data than *just * IndexedDB. There is also http://dev.w3.org/html5/spec/offline.html and http://dev.w3.org/html5/webstorage/ (even if you don't like them, they are other solutions to the offline problem). Browser vendors are not just looking at IndexedDB. 2. Many web developers have a working knowledge of SQL, so the concepts of a relational database may be more familiar. If adoption could be considered a proxy for the success of a standard, I'd suggest that aiming for something the web development community understands would be a large factor in adoption. I don't really think IndexedDB is that dissimilar to a relational database. There are a lot of one-to-one mappings of concepts of one to the other. 3. It's probably (!) easier to implement RelationalDB than IndexedDB, as it maps fairly cleanly to existing relational database technologies. This would allow vendors to implement it using Sqlite, Access, etc independent of the spec. Given that most vendors already have working implementations of IndexedDB, I don't think this is a good argument ;) Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 4/1/2011 9:39 AM, Aryeh Gregor wrote: IE6 is closed-source software written for a single platform. SQLite is in the public domain, works for all major operating systems and lots of minor ones, and is already used (I think?) by every major browser except IE. That makes all the difference. There's some benefit to having multiple interoperable implementations even if the reference implementation is public-domain, but enormously less than when the only implementations are controlled by particular parties. How, exactly, does it make all the difference? I sure hope you aren't suggesting that the spec say do what this code does. So if the only objection to WebSQL is there's no way we're going to get a formal spec or two interoperable implementations, I'd really encourage objectors to step back and ask themselves why they *want* a formal spec and two interoperable implementations. Those requirements are not axiomatic, they're means to obtain practical ends like allowing competitions and avoiding user lock-in. How many of those ends are really contrary to using SQLite as a de facto standard, and do the remaining ones really outweigh the practical advantages? That's not the only reason. Mozilla laid out others ten months ago: https://hacks.mozilla.org/2010/06/beyond-html5-database-apis-and-the-road-to-indexeddb/ Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On Fri, Apr 1, 2011 at 9:39 AM, Aryeh Gregor simetrical+...@gmail.com wrote: On Thu, Mar 31, 2011 at 12:28 PM, Boris Zbarsky bzbar...@mit.edu wrote: No, it actually sounds like a success; it prevented a specification being created which would have been tied to a particular implementation, no matter how widely-deployed. For comparison, IE6 was very widely deployed circa 2002. And yet specifying CSS, say, by saying just do exactly what IE6 does would not have been a good idea. Neither is defining WebSQL to do exactly what some particular version of SQLite does, and no one stepped up to define it better. IE6 is closed-source software written for a single platform. SQLite is in the public domain, works for all major operating systems and lots of minor ones, and is already used (I think?) by every major browser except IE. That makes all the difference. There's some benefit to having multiple interoperable implementations even if the reference implementation is public-domain, but enormously less than when the only implementations are controlled by particular parties. I think SQLite isn't a great long-term solution, because in the long term a single implementation stifles innovation. But it's also very powerful, pretty familiar to web developers (who are used to MySQL), requires vastly less implementation effort than making up a new system, and is in practice going to be much more interoperable than anything written separately by different browsers. So refusing to consider it just because there isn't a written standard or multiple interoperable implementations is not sound. When the Wikimedia Foundation was considering an official Board-approved policy for what file formats they'd allow for upload on sites like Wikimedia, the draft http://meta.wikimedia.org/wiki/File_format_policy required that the format be Defined by an open standard, implementation, or specification not under proprietary control. Notice that an open *standard* was not required -- an open *implementation* was enough. .doc need not apply, but LaTeX or MediaWiki wikitext are perfectly fine. The proposal was never formally adopted, but I think this philosophy makes a lot of sense. Many (although certainly not all) of the reasons for standards evaporate when you have a reference implementation under a BSD-style license or better. So if the only objection to WebSQL is there's no way we're going to get a formal spec or two interoperable implementations, I'd really encourage objectors to step back and ask themselves why they *want* a formal spec and two interoperable implementations. Those requirements are not axiomatic, they're means to obtain practical ends like allowing competitions and avoiding user lock-in. How many of those ends are really contrary to using SQLite as a de facto standard, and do the remaining ones really outweigh the practical advantages? (I came to this discussion very late, so apologies if there are additional objections to WebSQL that no one's mentioned in this specific thread, but have mentioned in the dozens of threads about this before now.) There are several reasons why we don't want to rely exclusively on SQLite, other than solely W3C formalia. First of all, what should we do once the SQLite team releases a new version which has some modifications in its SQL dialect? We generally always need to embed the latest version of the library since it contains critical bug fixes, however SQLite makes no guarantees that it will always support exactly the same SQL statements. For normal software this is fine. You simply retest your software after upgrading to the latest version and if it treats any of your queries differently, you simply adjust your query or your code to account for this. However web apps are afforded no such luxury. They have no control over when users upgrade their browsers to one with a new version of SQLite. This is why it is critically important on the web to maintain stable APIs. Unfortunately SQLite does not provide such a stable API. To make matters worse, since we are exposing the SQLite engine directly to potentially harmful web content, it is extra important to be tracking the latest version of SQLite as to pick up any and all security fixes. Second, as a browser developer, I am not at all excited about the idea of locking myself in to shipping SQLite forever. What if a new better embeddable SQL engine comes along and I want to switch to using that? Just because SQLite is popular now doesn't mean that in 10 years it couldn't be a unmaintained project largely replaced by some other embeddable SQL engine. Lastly, some vendors have expressed unwillingness to embed SQLite for legal reasons. Embedding other peoples code definitely exposes you to risk of copyright and patent lawsuits. While I can't say that I fully agree with this reasoning, I'm also not the one that would be on the receiving end of a lawsuit. Nor am I a lawyer and so
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On Apr/1/2011 3:39 PM, ext Glenn Maynard wrote: If SQLite was to be used as a web standard, I'd hope that it wouldn't show up in a spec as simply do what SQLite does, but as a complete spec of SQLite's behavior. FYI, the Web SQL Database NOTE says: [[ http://www.w3.org/TR/2010/NOTE-webdatabase-20101118/#web-sql 5. Web SQL User agents must implement the SQL dialect supported by Sqlite 3.6.19. ]] and I don't recall anyone ever committing to creating the later. -AB
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On Fri, Apr 1, 2011 at 12:39 PM, Glenn Maynard gl...@zewt.org wrote: Lastly, some vendors have expressed unwillingness to embed SQLite for legal reasons. Embedding other peoples code definitely exposes you to risk of copyright and patent lawsuits. While I can't say that I fully agree with this reasoning, I'm also not the one that would be on the receiving end of a lawsuit. Nor am I a lawyer and so ultimately will have to defer to people that know better. In the end it doesn't really matter as if a browser won't embed SQLite then it doesn't matter why, the end result is that the same SQL dialect won't be available cross browser which is bad for the web. If SQLite was to be used as a web standard, I'd hope that it wouldn't show up in a spec as simply do what SQLite does, but as a complete spec of SQLite's behavior. Browser vendors could then, if their lawyers insisted, implement their own compatible implementation, just as they do with other web APIs. I'd expect large portions of SQLite's test suite to be adaptable as a major starting point for spec tests, too. Have you read the WebSQL spec? Creating such a spec would be a formidable task, of course. Indeed. One that the SQL community has failed in doing so far. And they have a lot more experience with SQL than we do. / Jonas
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On Fri, Apr 1, 2011 at 3:53 PM, Jonas Sicking jo...@sicking.cc wrote: Creating such a spec would be a formidable task, of course. Indeed. One that the SQL community has failed in doing so far. And they have a lot more experience with SQL than we do. That suggests a very different rationale for not using SQL: it's too complex to specify interoperably. If true, it is, in my opinion, a much more compelling line of argument than the others. -- Glenn Maynard
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
Hi Shawn I would be interested in this. What would need to be done to make this a Firefox plugin? I've done XPCOM stuff before in xulrunner if that's any help. Cheers, Keean On Apr 1, 2011 6:09 PM, Shawn Wilsher sdwi...@mozilla.com wrote: On 4/1/2011 5:40 AM, Nathan Kitchen wrote: Are there any browser vendor representatives on the mailing list who would care to comment on the criteria for implementing something akin to Keean's RelationalDBhttps://github.com/keean/RelationalDB idea? What would need to be in place to start work on such an implementation? It wouldn't be terribly difficult to prototype this as an add-on for Firefox, I don't think (and I'd be happy to provide technical assistance to anyone wishing to do so). Doing this would allow web developers to install the add-on and play with it, which can give us useful feedback. I'm not saying we'd move it into the tree at that point, but it's a good first step to building a case to take it. 1. Opportunity to explore more solutions to offline data than *just * IndexedDB. There is also http://dev.w3.org/html5/spec/offline.html and http://dev.w3.org/html5/webstorage/ (even if you don't like them, they are other solutions to the offline problem). Browser vendors are not just looking at IndexedDB. 2. Many web developers have a working knowledge of SQL, so the concepts of a relational database may be more familiar. If adoption could be considered a proxy for the success of a standard, I'd suggest that aiming for something the web development community understands would be a large factor in adoption. I don't really think IndexedDB is that dissimilar to a relational database. There are a lot of one-to-one mappings of concepts of one to the other. 3. It's probably (!) easier to implement RelationalDB than IndexedDB, as it maps fairly cleanly to existing relational database technologies. This would allow vendors to implement it using Sqlite, Access, etc independent of the spec. Given that most vendors already have working implementations of IndexedDB, I don't think this is a good argument ;) Cheers, Shawn
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On Fri, 1 Apr 2011, Arthur Barstow wrote: On Apr/1/2011 3:39 PM, ext Glenn Maynard wrote: If SQLite was to be used as a web standard, I'd hope that it wouldn't show up in a spec as simply do what SQLite does, but as a complete spec of SQLite's behavior. FYI, the Web SQL Database NOTE says: [[ http://www.w3.org/TR/2010/NOTE-webdatabase-20101118/#web-sql 5. Web SQL User agents must implement the SQL dialect supported by Sqlite 3.6.19. ]] and I don't recall anyone ever committing to creating the later. Actually I did, multiple times. But nobody was interested in reimplementing that dialect independently, so all the points Jonas raised still apply. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On Fri, 1 Apr 2011, Glenn Maynard wrote: On Fri, Apr 1, 2011 at 2:39 PM, Jonas Sicking jo...@sicking.cc wrote: First of all, what should we do once the SQLite team releases a new version which has some modifications in its SQL dialect? We generally always need to embed the latest version of the library since it contains critical bug fixes, however SQLite makes no guarantees that it will always support exactly the same SQL statements. I don't find this compelling, because it assumes that the release methodology of SQLite is fixed in stone. It would be incredibly rude of us to force an independent team of developers to change development practices for our benefit. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On Sat, Apr 2, 2011 at 12:33 AM, Ian Hickson i...@hixie.ch wrote: I don't find this compelling, because it assumes that the release methodology of SQLite is fixed in stone. It would be incredibly rude of us to force an independent team of developers to change development practices for our benefit. You can certainly ask if they're interested in doing so, not for our benefit (whoever our means), but for the benefit of the Web as a whole, and there's nothing at all rude in asking. I'd say the opposite: it's rude to assume they wouldn't be interested, rather than asking and letting them come to their own decision. (I don't know where the notion of forcing them to do anything came from.) -- Glenn Maynard
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On Friday, April 1, 2011, Glenn Maynard gl...@zewt.org wrote: On Sat, Apr 2, 2011 at 12:33 AM, Ian Hickson i...@hixie.ch wrote: I don't find this compelling, because it assumes that the release methodology of SQLite is fixed in stone. It would be incredibly rude of us to force an independent team of developers to change development practices for our benefit. You can certainly ask if they're interested in doing so, not for our benefit (whoever our means), but for the benefit of the Web as a whole, and there's nothing at all rude in asking. I'd say the opposite: it's rude to assume they wouldn't be interested, rather than asking and letting them come to their own decision. (I don't know where the notion of forcing them to do anything came from.) I am incredibly uncomfortable with the idea of putting the responsibility of the health of the web in the hands of one project. In fact, one of the main reasons I started working at Mozilla was to prevent this. / Jonas
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 03/31/2011 05:19 PM, ext Nathan Kitchen wrote: I've been watching discussions on IndexedDB for a while now, and wondered if anyone would mind spending a few moments to explain how IndexedDB is related (or not) to WebSQL. Is IndexedDB seen as replacing the functionality originally offered by WebSQL? If not, are there any plans to make a cross-platform variant of Web SQL? WebSQL in its current form is pretty dead, see http://www.w3.org/TR/webdatabase/ : quoteBeware. This specification is no longer in active maintenance and the Web Applications Working Group does not intend to maintain it further./quote And: quoteThis document was on the W3C Recommendation track but specification work has stopped. The specification reached an impasse: all interested implementors have used the same SQL backend (Sqlite), but we need multiple independent implementations to proceed along a standardisation path./quote I suggest you to look at the mailing list archive if you want all the details. cheers, Benjamin
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
That's nice, pretty much what I was thinking but somewhat more complete : )
Is there not a w3 group progressing something like this? And if not, who
would need to be lobbied to get one started?!
As an aside, I note you didn't implement date as a supported data type.
Was that a conscious decision, and if so what was the reasoning behind it?
N
On 31 March 2011 16:33, Keean Schupke ke...@fry-it.com wrote:
Have a look at my RelationalDB API
https://github.com/keean/RelationalDB
In particular examples/candy.html
A lot of work went into the underlying concepts - Its work originally
published by myself and others at the 2004 Haskell Workshop, and follows on
from HaskellDB which was the original inspiration behind C#s Linq
functionality).
It implements the relational-algebra operators as methods that operate on
relation objects.
Let me know what you think.
Cheers,
Keean.
On 31 March 2011 15:19, Nathan Kitchen w...@nathankitchen.com wrote:
Hi.
I've been watching discussions on IndexedDB for a while now, and wondered
if anyone would mind spending a few moments to explain how IndexedDB is
related (or not) to WebSQL. Is IndexedDB seen as replacing the functionality
originally offered by WebSQL? If not, are there any plans to make a
cross-platform variant of Web SQL?
If (?) most web developers know SQL, is there a case to be made for
abstracting SQL into JSON/JavaScript rather than moving to IndexedDB
document storage? Reasons for asking this:
- Many of the posts appearing to come from the dev community rather
than W3C seem to expect more SQL-esque functionality from IndexedDB. If
the
enthusiasts who get involved enough to post to the board are expecting
SQL/query type experience, maybe there is a driver for a native database
API
supporting this.
- Several people have noted that third-party frameworks could
implement this functionality. This might be a daft question, but isn't it
easier to implement an IndexedDB-like framework on top of WebSQL,
than a
WebSQL-like framework on top of IndexedDB (overuse of quotes to
indicate
the general concept).
I had a ponder on how I'd like to see such a framework implemented (in
both Access SQLite :p ), and came up with a stack of pseudo-code below in
my lunch break. Might make an interesting discussion point. It's not really
IndexedDB, it's WebSQL v2. Or maybe WebJSQL or something. I'd be really
interested to understand what advantages IndexedDB has over an
implementation like the one below though.
// DATABASE
// First, open a database with the specified name. The number at the
// end denotes the version of the specification that the application
// plans to use. This allows forward-compatibility with vNext.
var db = window.openDatabase(shoppinglist, 1.0);
// MIGRATIONS
// Next, create some migrations. These are predefined structures which
// are validated by the browser database engine. A migration consists
// of two actions: one up, one down. Each action specifies some
// operations and parameters. It's up to the browser database to read
// these and perform the appropriate action, as defined in the spec.
// Other actions may include a batch add for static data. It could
// also be valid to have key and index creation and removal as separate
// actions.
// SHOPPING TRIP
var createTripTableAction = {
action: create-table,
params: {
name: trip,
columns: [
{ name: id, type: whole-number, primaryKey: true },
{ name: name, type: string, length: 32, regex: [A-Z]{1,32}
} // regex: wouldn't that be nice...
],
indexes: [
{
columns: [
{ name: name, type: full-text }
]
} // More indexes here if required
]
}
};
var removeTripTableAction = {
action: remove-table,
params: {
name: shopping,
cleardata: true
}
};
// SHOPPING
var createShoppingTableAction = {
action: create-table,
params: {
name: shopping,
columns: [
{ name: id, type: whole-number, primaryKey: true },
{ name: tripid, type: whole-number },
{ name: name, type: string, length: 128, nillable: false },
{ name: cost, type: decimal-number }
],
keys: [
{ local: tripid, foreign: { table: trips, column: id },
cascade-delete: true } }
],
indexes: [
{
columns: [
{ name: tripid, type: asc },
{ name: name, type: full-text }
]
}
]
}
};
var removeShoppingTableAction = {
action: remove-table,
params: {
name: shopping,
cleardata: true
}
};
var migTrip = { up: createTripTableAction, down: removeTripTableAction };
var migShopping = { up: createShoppingTableAction, down:
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On Thu, Mar 31, 2011 at 11:43 AM, Benjamin Poulain benjamin.poul...@nokia.com wrote: WebSQL in its current form is pretty dead, see http://www.w3.org/TR/webdatabase/ : quoteBeware. This specification is no longer in active maintenance and the Web Applications Working Group does not intend to maintain it further./quote And: quoteThis document was on the W3C Recommendation track but specification work has stopped. The specification reached an impasse: all interested implementors have used the same SQL backend (Sqlite), but we need multiple independent implementations to proceed along a standardisation path./quote This is painful to read. WebSQL development died because SQLite, the most widely-deployed database software in the world, was too good? That sounds like a catastrophic failure of the W3C process. -- Glenn Maynard
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 3/31/11 12:06 PM, Glenn Maynard wrote: This is painful to read. WebSQL development died because SQLite, the most widely-deployed database software in the world, was too good? That sounds like a catastrophic failure of the W3C process. No, it actually sounds like a success; it prevented a specification being created which would have been tied to a particular implementation, no matter how widely-deployed. For comparison, IE6 was very widely deployed circa 2002. And yet specifying CSS, say, by saying just do exactly what IE6 does would not have been a good idea. Neither is defining WebSQL to do exactly what some particular version of SQLite does, and no one stepped up to define it better. -Boris
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
No real reason - just trying to implement a minimal framework. Date objects
would be a definite must have going forward.
I was interested in trying to get something like this standardised, as I
believe it has none of the issues that stopped WebSQL, as it defines a
complete relational API independent of the implementation of SQL behind it.
The key thing is to get the browser implementors interested in implementing
it. If even one of the main browser implementors is not interested in
implementing it, then it will suffer the same fate as WebSQL.
Independent of standardisation (which I would like) I intend to try and
implement the same API on top of WebSQL and IndexedDB as a library. So
people are free to use the backend with the best performance without
changing their code. It aims to be as stateless as possible, and to
implement relational algebra on relations.
Cheers,
Keean.
On 31 March 2011 15:54, Nathan Kitchen w...@nathankitchen.com wrote:
That's nice, pretty much what I was thinking but somewhat more complete : )
Is there not a w3 group progressing something like this? And if not, who
would need to be lobbied to get one started?!
As an aside, I note you didn't implement date as a supported data type.
Was that a conscious decision, and if so what was the reasoning behind it?
N
On 31 March 2011 16:33, Keean Schupke ke...@fry-it.com wrote:
Have a look at my RelationalDB API
https://github.com/keean/RelationalDB
In particular examples/candy.html
A lot of work went into the underlying concepts - Its work originally
published by myself and others at the 2004 Haskell Workshop, and follows on
from HaskellDB which was the original inspiration behind C#s Linq
functionality).
It implements the relational-algebra operators as methods that operate on
relation objects.
Let me know what you think.
Cheers,
Keean.
On 31 March 2011 15:19, Nathan Kitchen w...@nathankitchen.com wrote:
Hi.
I've been watching discussions on IndexedDB for a while now, and wondered
if anyone would mind spending a few moments to explain how IndexedDB is
related (or not) to WebSQL. Is IndexedDB seen as replacing the functionality
originally offered by WebSQL? If not, are there any plans to make a
cross-platform variant of Web SQL?
If (?) most web developers know SQL, is there a case to be made for
abstracting SQL into JSON/JavaScript rather than moving to IndexedDB
document storage? Reasons for asking this:
- Many of the posts appearing to come from the dev community rather
than W3C seem to expect more SQL-esque functionality from IndexedDB. If
the
enthusiasts who get involved enough to post to the board are expecting
SQL/query type experience, maybe there is a driver for a native database
API
supporting this.
- Several people have noted that third-party frameworks could
implement this functionality. This might be a daft question, but isn't it
easier to implement an IndexedDB-like framework on top of WebSQL,
than a
WebSQL-like framework on top of IndexedDB (overuse of quotes to
indicate
the general concept).
I had a ponder on how I'd like to see such a framework implemented (in
both Access SQLite :p ), and came up with a stack of pseudo-code below in
my lunch break. Might make an interesting discussion point. It's not really
IndexedDB, it's WebSQL v2. Or maybe WebJSQL or something. I'd be really
interested to understand what advantages IndexedDB has over an
implementation like the one below though.
// DATABASE
// First, open a database with the specified name. The number at the
// end denotes the version of the specification that the application
// plans to use. This allows forward-compatibility with vNext.
var db = window.openDatabase(shoppinglist, 1.0);
// MIGRATIONS
// Next, create some migrations. These are predefined structures which
// are validated by the browser database engine. A migration consists
// of two actions: one up, one down. Each action specifies some
// operations and parameters. It's up to the browser database to read
// these and perform the appropriate action, as defined in the spec.
// Other actions may include a batch add for static data. It could
// also be valid to have key and index creation and removal as separate
// actions.
// SHOPPING TRIP
var createTripTableAction = {
action: create-table,
params: {
name: trip,
columns: [
{ name: id, type: whole-number, primaryKey: true },
{ name: name, type: string, length: 32, regex:
[A-Z]{1,32} } // regex: wouldn't that be nice...
],
indexes: [
{
columns: [
{ name: name, type: full-text }
]
} // More indexes here if required
]
}
};
var removeTripTableAction = {
action: remove-table,
params: {
name: shopping,
cleardata: true
}
};
// SHOPPING
var
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
This is painful to read. WebSQL development died because SQLite, the most widely-deployed database software in the world, was too good? That sounds like a catastrophic failure of the W3C process. -- Glenn Maynard Hear. I am starting to think that Mozilla will step up and provide an embedding of SQLite, even if it has to only think of it as such. It will have to. People would rather use a working database than something crippled albeit specced (see LocalStorage or IndexedDB). It was things like XHR in all their unspecced glory that brought the web to where it is today.
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 31 March 2011 19:08, Joran Greef jo...@ronomon.com wrote: This is painful to read. WebSQL development died because SQLite, the most widely-deployed database software in the world, was too good? That sounds like a catastrophic failure of the W3C process. -- Glenn Maynard Hear. I am starting to think that Mozilla will step up and provide an embedding of SQLite, even if it has to only think of it as such. It will have to. People would rather use a working database than something crippled albeit specced (see LocalStorage or IndexedDB). It was things like XHR in all their unspecced glory that brought the web to where it is today. Do you want to take a look at my RelationalDB library - it could form the basis of a replacement for WebSQL, and as it is based on relational algebra not SQL, it has not user visible dependencies on the particular SQL implementation? Have a look at: https://github.com/keean/RelationalDB/blob/master/examples/candy.html For a usage example. This should run in chrome right now (using WebSQL as a backend). I would appreciate any thoughts, comments etc. Cheers, Keean.
