date:20121112


On 11/12/12 2:59 PM, Florian Bösch wrote:

A-ha. Filled payload with random. Now it's FF 4.4s put, 5.5s get. In
line with chromes perf.


Hmm.  This I can't reproduce.  I tried filling the array with randoms, 
and the times end up about 30% higher, but not 20x higher, for me.


-Boris

Re: Making offline apps easier?

2012-11-12 Thread Ian Hickson

On Mon, 12 Nov 2012, Charles McCathie Nevile wrote:
> 
> 1. Appcache.
> Yeah, we know. But if nobody makes a proposal, nothing will get better.
> (There is also a "fixing-appcache" community group taht might come here
> with proposals).

There are already several proposals, and they are already making their way 
through to the spec. If anyone has any use cases that they haven't filed 
as bugs on the WHATWG spec or e-mailed the WHATWG list, please don't 
hesitate to do so, so that they can be considered. It's use cases that are 
needed, not proposals, to make additional progress here beyond what will 
happen already given the currently-filed bugs.

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: Two years on and still no sensible web storage solutions exist

A-ha. Filled payload with random. Now it's FF 4.4s put, 5.5s get. In line
with chromes perf.


On Mon, Nov 12, 2012 at 11:46 PM, Boris Zbarsky  wrote:

> On 11/12/12 1:56 PM, Florian Bösch wrote:
>
>> In FF 16.0.2 on linux, 50mb put takes about 100ms and get takes about
>> 500ms.
>>
>
> For what it's worth, I did some measuring, and the main performance cost
> centers for the write (the CPU-intensive ones, not the disk access) seem to
> be:
>
> 1)  Making a copy of the data to put in the database (because the database
> write happens on a background thread and we want to return control to JS
> before we do that to avoid blocking the main thread, so have to copy the
> data).
>
> 2)  Compressing the data (we store it compressed on disk).
>
> For the read it's decompressing and then copying out of the resulting data
> structure into the JS engine's heap for the typed array.
>
> It might be possible to mitigate the copying costs, especially if
> indexeddb had a way to indicate that things should be transferred, not
> copied, similar to postMessage.
>
> -Boris
>

Re: Two years on and still no sensible web storage solutions exist


On 11/12/12 1:56 PM, Florian Bösch wrote:

In FF 16.0.2 on linux, 50mb put takes about 100ms and get takes about 500ms.


For what it's worth, I did some measuring, and the main performance cost 
centers for the write (the CPU-intensive ones, not the disk access) seem 
to be:


1)  Making a copy of the data to put in the database (because the 
database write happens on a background thread and we want to return 
control to JS before we do that to avoid blocking the main thread, so 
have to copy the data).


2)  Compressing the data (we store it compressed on disk).

For the read it's decompressing and then copying out of the resulting 
data structure into the JS engine's heap for the typed array.


It might be possible to mitigate the copying costs, especially if 
indexeddb had a way to indicate that things should be transferred, not 
copied, similar to postMessage.


-Boris

Re: Two years on and still no sensible web storage solutions exist

In FF 16.0.2 on linux, 50mb put takes about 100ms and get takes about 500ms.


On Mon, Nov 12, 2012 at 10:50 PM, Boris Zbarsky  wrote:

> On 11/12/12 12:31 PM, Florian Bösch wrote:
>
>> Care to paste that one? I'll test it in firefox.
>>
>
> http://pastebin.mozilla.org/**1937689
>
> -Boris
>

Re: Two years on and still no sensible web storage solutions exist


On 11/12/12 12:31 PM, Florian Bösch wrote:

Care to paste that one? I'll test it in firefox.


http://pastebin.mozilla.org/1937689

-Boris

Re: Two years on and still no sensible web storage solutions exist

Care to paste that one? I'll test it in firefox.


On Mon, Nov 12, 2012 at 9:25 PM, Boris Zbarsky  wrote:

> On 11/12/12 8:17 AM, Florian Bösch wrote:
>
>> This testsite: 
>> http://codeflow.org/issues/**indexeddb.html
>> Putting 50mb takes between 2.5 to 8 seconds. Getting it between 2 to 4
>> seconds.
>> Putting 100mb doesn't work in Chrome at all and throws an exception on
>> the put after half a second.
>>
>
> After fixing your testcase to work in Gecko (by updating to current
> indexeddb spec and fixing all the webkit-specific things in there), I see
> it take about 60ms to put 50MB of data.  About 150ms to read it. Obvious
> your mileage may vary depending on your machine's I/O subsystem.
>
> -Boris
>
>

Re: Two years on and still no sensible web storage solutions exist


On 11/12/12 8:17 AM, Florian Bösch wrote:

This testsite: http://codeflow.org/issues/indexeddb.html
Putting 50mb takes between 2.5 to 8 seconds. Getting it between 2 to 4
seconds.
Putting 100mb doesn't work in Chrome at all and throws an exception on
the put after half a second.


After fixing your testcase to work in Gecko (by updating to current 
indexeddb spec and fixing all the webkit-specific things in there), I 
see it take about 60ms to put 50MB of data.  About 150ms to read it. 
Obvious your mileage may vary depending on your machine's I/O subsystem.


-Boris

Re: Two years on and still no sensible web storage solutions exist

Similarly the vendors deciding not wanting to support WebSQL isn't a spec
issue and it should be filed in the bug tracker of your favorite vendor.


On Mon, Nov 12, 2012 at 8:02 PM, João Eiras  wrote:

> On Mon, 12 Nov 2012 19:50:20 +0100, Florian Bösch 
> wrote:
>
>  On Mon, Nov 12, 2012 at 7:42 PM, João Eiras  wrote:
>>
>>  You're again confusing implementation details with the specification.
>>> User
>>> agents are free to implement whatever quota management they want, as long
>>> as it's transparent and respects the visible effects on the webpage side.
>>>
>>>  That may be so, it still means WebSQL is universally hard limited
>> without
>> escape to 5mb. So it doesn't really matter if that's what's specced or
>> not,
>> it makes it equally unsuitable for data beyond 5mb.
>>
>
> Again, not a spec issue. Go to your favorite browser bug tracker and make
> a feature request.
>
>

Re: Two years on and still no sensible web storage solutions exist

2012-11-12 Thread João Eiras


On Mon, 12 Nov 2012 19:50:20 +0100, Florian Bösch  wrote:


On Mon, Nov 12, 2012 at 7:42 PM, João Eiras  wrote:

You're again confusing implementation details with the specification.  
User
agents are free to implement whatever quota management they want, as  
long
as it's transparent and respects the visible effects on the webpage  
side.



That may be so, it still means WebSQL is universally hard limited without
escape to 5mb. So it doesn't really matter if that's what's specced or  
not,

it makes it equally unsuitable for data beyond 5mb.


Again, not a spec issue. Go to your favorite browser bug tracker and make  
a feature request.

Re: Two years on and still no sensible web storage solutions exist

On Mon, Nov 12, 2012 at 7:42 PM, João Eiras  wrote:

> You're again confusing implementation details with the specification. User
> agents are free to implement whatever quota management they want, as long
> as it's transparent and respects the visible effects on the webpage side.
>
That may be so, it still means WebSQL is universally hard limited without
escape to 5mb. So it doesn't really matter if that's what's specced or not,
it makes it equally unsuitable for data beyond 5mb.

Re: Two years on and still no sensible web storage solutions exist

On Nov 12, 2012, at 10:27 AM, Florian Bösch wrote:

> The hard quota is not my idea. It's what those browsers who do implement 
> WebSQL implement. Unlike IndexedDB which triggers a "do you want to allow 
> yadda yadda" or whatever, at least sometimes, WebSQL is hard limited to 5mb.

Actually, asking for more than 5M triggers a "I can haz mor?" dialog as well.  
The limit is equally soft.  At least IME.

But I don't get the philosophy disconnect. If the user is going to object to 5M 
of user data in a database, he certainly isn't going to be happy with giant 
files all over the place.  I certainly wouldn't be happy about it.  Disk bytes 
is disk bytes.

Re: Two years on and still no sensible web storage solutions exist

2012-11-12 Thread João Eiras


On Mon, 12 Nov 2012 19:27:33 +0100, Florian Bösch  wrote:


The hard quota is not my idea. It's what those browsers who do implement
WebSQL implement. Unlike IndexedDB which triggers a "do you want to allow
yadda yadda" or whatever, at least sometimes, WebSQL is hard limited to  
5mb.


You're again confusing implementation details with the specification. User  
agents are free to implement whatever quota management they want, as long  
as it's transparent and respects the visible effects on the webpage side.

Re: Two years on and still no sensible web storage solutions exist

On Mon, Nov 12, 2012 at 7:19 PM, Todd Blanchard wrote:

> Gigabytes of data?  Why would one subject WebSQL to a hard quote of 5M and
> think using gigabytes of user disk space in a different scenario is OK?
>  Something seems out of balance here.  Why limit one but not the other?
>
The hard quota is not my idea. It's what those browsers who do implement
WebSQL implement. Unlike IndexedDB which triggers a "do you want to allow
yadda yadda" or whatever, at least sometimes, WebSQL is hard limited to 5mb.

And as to why large sizes do matter. Let's take the sponza atrium in binary
form including normals, positions, texcoords and material IDs (100mb). If
you chuck out a bunch of things you can compute, structure and otherwise
make smaller, you can get out at some 12mb. But you can't send that to the
GPU. You've got to convert it to something that's usable for the GPU, which
takes up around 1-2 seconds of CPU time in JS if you wrote a
super-optimized piece of code. Now reading 100mb from disk to a binary
representation shouldn't take more than about 80ms or so worst case. So It
would be kind of really desirable to be able to store temporary calculated
assets. Another usecase for large asset storage is productivity software
that deals with images/meshes. The amount of data you quickly generate with
that kind of software is staggering, and in the interest to keep things
easy to use it would be kinda nice to be able to manage assets on the
client other than file drop.

Re: Two years on and still no sensible web storage solutions exist

On Nov 12, 2012, at 4:43 AM, Florian Bösch wrote:

> On Mon, Nov 12, 2012 at 1:36 PM, Todd Blanchard  
> wrote:
> Wondering what is wrong with the blob type in SQLite?
> A couple of things
> 1) Array buffers can't be passed to websql

Could be fixed.

> 2) websql is subject to the hard quota (5mb) and I'm talking about gigabytes 
> of data

Gigabytes of data?  Why would one subject WebSQL to a hard quote of 5M and 
think using gigabytes of user disk space in a different scenario is OK?  
Something seems out of balance here.  Why limit one but not the other?

Re: Two years on and still no sensible web storage solutions exist

2012-11-12 Thread Joshua Bell

For anyone that's confused, I sent from the wrong email address so non-list
recipients received my reply but the list did not.

And Kyle's right, as I realized when following up before re-sending.

On Mon, Nov 12, 2012 at 9:56 AM, Kyle Huey  wrote:

> On Mon, Nov 12, 2012 at 9:52 AM, Joshua Bell  wrote:
>
>> Per the spec, anything the structured cloning algorithm [1] handles can
>> be used as record values in IndexedDB. ArrayBuffers are not on that list,
>> but Chrome does support them in IndexedDB.
>>
>
> The TypedArray spec specifies how to structured clone ArrayBuffers.
>
> http://www.khronos.org/registry/typedarray/specs/latest/#9.1
>
> - Kyle
>

Re: Two years on and still no sensible web storage solutions exist

2012-11-12 Thread Kyle Huey

On Mon, Nov 12, 2012 at 9:52 AM, Joshua Bell  wrote:

> Per the spec, anything the structured cloning algorithm [1] handles can be
> used as record values in IndexedDB. ArrayBuffers are not on that list, but
> Chrome does support them in IndexedDB.
>

The TypedArray spec specifies how to structured clone ArrayBuffers.

http://www.khronos.org/registry/typedarray/specs/latest/#9.1

- Kyle

Re: Two years on and still no sensible web storage solutions exist

2012-11-12 Thread Andrew Wilson

Your best bet is to report implementation/performance issues to the browser
vendors, rather than here, unless you believe that there is something
inherent in the spec that results in slow implementations.

I'd say in the specific case of Chrome, we probably have not optimized for
the case of writing out a 100MB ArrayBuffer - logging a bug at
http://crbug.com describing your use case would be a useful way to have
this addressed, at least on that platform.

On Mon, Nov 12, 2012 at 5:17 PM, Florian Bösch  wrote:

> On Mon, Nov 12, 2012 at 3:49 PM, Kyle Huey  wrote:
>
>> Er, IndexedDB should handle ArrayBuffers just fine.  If you're seeing
>> problems with that that's an implementation bug, not a case that the spec
>> failed to handle.
>>
> You're right it does, I just checked. Nevertheless there are some issues.
>
> This testsite: http://codeflow.org/issues/indexeddb.html
> Putting 50mb takes between 2.5 to 8 seconds. Getting it between 2 to 4
> seconds.
> Putting 100mb doesn't work in Chrome at all and throws an exception on the
> put after half a second.
>
> For comparison, in python:
> Writing 50mb takes between 40-80ms. Reading takes around 50ms.
>
>
>
>

RE: Making offline apps easier?

2012-11-12 Thread arnaud.braud

Hello,
Some comments inline,

> -Message d'origine-
> De : Charles McCathie Nevile [mailto:cha...@yandex-team.ru]
> Envoyé : lundi 12 novembre 2012 17:11
> À : public-webapps WG
> Objet : Making offline apps easier?
> 
> Hi folks
> 
> We have a handful of things that could be on our plate here.
> 
> 1. Appcache.
> Yeah, we know. But if nobody makes a proposal, nothing will get better.
> (There is also a "fixing-appcache" community group taht might come here
> with proposals).
> 2. Offline apps breakout at TPAC
> http://www.w3.org/wiki/TPAC2012/Offline_Apps Ashok Malhotra (Oracle,
> who aren't webapps members), ran a session. Where we agreed to divide
> the thoughts into "the app shell" and "the app data" - either of which
> could be brought from offline, or online, independently. Plus I made a
> half-page note about different ways of having apps offline 3. File APIs.
> The plain File reader seems almost done. Still. There are apparently
> people interested in a File System (Opera implemented one for Opera
> Unite, Chrome and Yandex have one, and there are discussions about
> making a simpler one).
> 4. Databases
> IndexedDB seems to be getting traction across all platforms. Unlike at
> the face to face, when literally nobody spoke in favour of WebSQL,
> there are people who think it is worthwhile. At the same time Ashok has
> been hinting at using something more seriously SQL to help with issues
> like synch (which always seems like it won't be that hard, but never
> actually works entirely properly after all).
> 5. Widgets and packaged apps
> There is the "official" widget stack. At the face to face the only
> people who spoke up in favour of keeping this alive and connected to
> other app systems were Zynga. There is the Mozilla proposal for apps
> packaged with a JSON manifest, and Yandex has the same functionality
> but probably slightly different syntax for the JS. And there may be
> others...


I'd also add a 
6) Zip Archive : which was discussed at last TPAC.

It might be worth trying to flatten our use cases and see where we have huge 
gaps and avoid running too many specs for one given use case.


> 
> 
> cheers
> 
> Chaals
> 
> --
> Charles McCathie Nevile - Consultant (web standards) CTO Office, Yandex
>  cha...@yandex-team.ru Find more at http://yandex.com


_

Ce message et ses pieces jointes peuvent contenir des informations 
confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce 
message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages 
electroniques etant susceptibles d'alteration,
France Telecom - Orange decline toute responsabilite si ce message a ete 
altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged 
information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete 
this message and its attachments.
As emails may be altered, France Telecom - Orange is not liable for messages 
that have been modified, changed or falsified.
Thank you.

Re: Two years on and still no sensible web storage solutions exist

On Mon, Nov 12, 2012 at 3:49 PM, Kyle Huey  wrote:

> Er, IndexedDB should handle ArrayBuffers just fine.  If you're seeing
> problems with that that's an implementation bug, not a case that the spec
> failed to handle.
>
You're right it does, I just checked. Nevertheless there are some issues.

This testsite: http://codeflow.org/issues/indexeddb.html
Putting 50mb takes between 2.5 to 8 seconds. Getting it between 2 to 4
seconds.
Putting 100mb doesn't work in Chrome at all and throws an exception on the
put after half a second.

For comparison, in python:
Writing 50mb takes between 40-80ms. Reading takes around 50ms.

Making offline apps easier?

2012-11-12 Thread Charles McCathie Nevile


Hi folks

We have a handful of things that could be on our plate here.

1. Appcache.
Yeah, we know. But if nobody makes a proposal, nothing will get better.
(There is also a "fixing-appcache" community group taht might come here
with proposals).
2. Offline apps breakout at TPAC
http://www.w3.org/wiki/TPAC2012/Offline_Apps Ashok Malhotra (Oracle, who
aren't webapps members), ran a session. Where we agreed to divide the
thoughts into "the app shell" and "the app data" - either of which could
be brought from offline, or online, independently. Plus I made a half-page
note about different ways of having apps offline
3. File APIs.
The plain File reader seems almost done. Still. There are apparently
people interested in a File System (Opera implemented one for Opera Unite,
Chrome and Yandex have one, and there are discussions about making a
simpler one).
4. Databases
IndexedDB seems to be getting traction across all platforms. Unlike at the
face to face, when literally nobody spoke in favour of WebSQL, there are
people who think it is worthwhile. At the same time Ashok has been hinting
at using something more seriously SQL to help with issues like synch
(which always seems like it won't be that hard, but never actually works
entirely properly after all).
5. Widgets and packaged apps
There is the "official" widget stack. At the face to face the only people  
who spoke up in favour of keeping this alive and connected to other app  
systems were Zynga. There is the Mozilla proposal for apps packaged with a  
JSON manifest, and Yandex has the same functionality but probably slightly  
different syntax for the JS. And there may be others...



cheers

Chaals

--
Charles McCathie Nevile - Consultant (web standards) CTO Office, Yandex
cha...@yandex-team.ru Find more at http://yandex.com

Re: Two years on and still no sensible web storage solutions exist

2012-11-12 Thread Kyle Huey

On Mon, Nov 12, 2012 at 4:30 AM, Florian Bösch  wrote:

> On Mon, Nov 12, 2012 at 1:12 PM, Charles McCathie Nevile <
> cha...@yandex-team.ru> wrote:
>
>> The Webapps group will continue to try and reach industry agreement on
>> ways to handle data storage for offline applications. Help is appreciated.
>> Trying to help is generally appreciated too.
>>
> A slight tangent, but coming from WebGL where we're dealing with a lot of
> binary data, the inability of WebSQL and IndexedDB to deal with Array
> Buffers is quite an obstacle.
>

Er, IndexedDB should handle ArrayBuffers just fine.  If you're seeing
problems with that that's an implementation bug, not a case that the spec
failed to handle.

- Kyle

Re: Two years on and still no sensible web storage solutions exist

On Mon, Nov 12, 2012 at 1:36 PM, Todd Blanchard wrote:

> Wondering what is wrong with the blob type in SQLite?
>
A couple of things
1) Array buffers can't be passed to websql
2) websql is subject to the hard quota (5mb) and I'm talking about
gigabytes of data
3) traditionally blobs in RDBMSes are slow

Of course you could use base64 encoding, unfortunately atob btoa do not
support array buffers, and the methods are also not standardized, and there
don't exist native methods for base64 encoding/decoding to array buffers.
Base64 encoding in JS of hundreds of megabytes or a couple gigabytes is
unfortunately not quite a feasible prospect, especially not on weaker
machines.

Re: Two years on and still no sensible web storage solutions exist


On Nov 12, 2012, at 4:30 AM, Florian Bösch wrote:

> On Mon, Nov 12, 2012 at 1:12 PM, Charles McCathie Nevile 
>  wrote:
> The Webapps group will continue to try and reach industry agreement on ways 
> to handle data storage for offline applications. Help is appreciated. Trying 
> to help is generally appreciated too.
> A slight tangent, but coming from WebGL where we're dealing with a lot of 
> binary data, the inability of WebSQL and IndexedDB to deal with Array Buffers 
> is quite an obstacle. 

Wondering what is wrong with the blob type in SQLite?

http://www.sqlite.org/datatype3.html

Re: Two years on and still no sensible web storage solutions exist

On Mon, Nov 12, 2012 at 1:12 PM, Charles McCathie Nevile <
cha...@yandex-team.ru> wrote:

> The Webapps group will continue to try and reach industry agreement on
> ways to handle data storage for offline applications. Help is appreciated.
> Trying to help is generally appreciated too.
>
A slight tangent, but coming from WebGL where we're dealing with a lot of
binary data, the inability of WebSQL and IndexedDB to deal with Array
Buffers is quite an obstacle. The File storage API however isn't supported
anywhere but Chrome. I don't think it's mandatory for a query based API to
also support binary blobs. Traditionally this has often been solved in
similar situations by storing keys (paths) to an appropriate provider of
binary storage (such as a filesystem). Similar uses could be made if
browsers supported both a query based DB and a filesystem like API for
lookups by key/value.

On a related note, key/value backing stores might not be terribly fancy
compared to query based backing stores. But a surprising number of data
modelling problems does not neatly fit into relational logic well (usually
graphs, see also the popularity of key/value backing stores and the whole
NoSQL stuff). So for me a query based backing store would be nice to have.
But a good key/value backing store with the ability to store binary isn't
really optional.

Re: Two years on and still no sensible web storage solutions exist

2012-11-12 Thread Charles McCathie Nevile

On Mon, 12 Nov 2012 10:14:47 +0100, Andrew Wilson wrote:My recollection is that some vendors refused to ever support SQLite, making a SQL-based standard not really feasible. You can undoubtedly review the w3c archives to find out more details if you want to know the rationale they expressed at the time.Right. You'll find a lot of the discussion took place at the TPAC meeting in November 2008 and again in 2010. Note that when implementors say "never", you can be pretty sure it means "at least not for a while, then we might re-evaluate".It's unfortunate that IndexedDB is still not widely supported by browsers - I share your frustration.Likewise. It is unfortunate that the industry has not managed to agree on any widely-supported standard for offline data storage beyond key-value pairs. However, as you say, you can write to the IndexedDB APIs, and use a shim to get support on platforms that do not support it yet.
-atwOn Mon, Nov 12, 2012 at 9:50 AM, Florian Bösch wrote:
I'd like to propose as a constructive strategy not to flame/offend everybody right off the bat.As co-chair of the working group I'd like to point out that flaming is generally considered an unacceptable way to communicate in this group, and request a more civil tone in future messages.
On Fri, Nov 9, 2012 at 7:24 PM, Todd Blanchard wrote:

It has been two years since the following little note was attached to the Web SQL SpecThis document was on the W3C Recommendation track but specification work has stopped. The specification reached an impasse: all interested implementors have used the same SQL backend (Sqlite), but we need multiple independent implementations to proceed along a standardisation path.

This move has left the web browser world in disarray and has been widely misconstrued by readers to mean "Web SQL is deprecated and will not be supported in the future - better port to IndexedDB". I don't think that is misconstruing what the general sense of the gathered implementors was. As you note elsewhere, in your own case you are no more forced to agree with a standard than any other implementor. We're just documenting a view of reality here.Today, TWO YEARS LATER, we have SQLite on iOS, Android, Chrome, and Safari but no IndexedDB. On Firefox we have IndexedDB with SQLite available only via a browser extension https://addons.mozilla.org/en-US/firefox/addon/html5-websql-for-firefox/ (annoying but liveable), and on IE[89] only DOM storage with IndexedDB expected on IE 10.

Sources: http://caniuse.com/indexeddb http://caniuse.com/sql-storage

As someone who is trying to build an offline web app the works both on browsers and smart phones and needs to store a lot of client side complex data that will require lots of joins - let me just say WTF?I understand your frustration. I will point out that many people volunteer their own time, on top of time that many companies pay for, in order to help provide standards that make it easier for you to make a living or just do something useful. But this is not a guarantee that other people are required to solve your problems. Why do we have standards again? You're not helping.To be clear, standards are documents that describe what the industry has agreed on. There is no formal power of enforcement beyond market adoption. If your preferred tool developer doesn't support your preferred standard, then you as a customer probably have a lot more power than we as a forum where they can listen to what others are doing and make their own proposals.If you are providing solutions to their customers, then your ability to conI look around at information on the state of storage options and I read stuff like this:"Since November 18, 2010, the W3C announced that Web SQL database is a deprecated specification. This is a recommendation for web developers to no longer use the technology as effectively, the spec will receive no new updates and browser vendors aren't encouraged to support this technology. The new alternative is IndexedDB which is already available on Chrome 12+ and Firefox 5+, and, soon, in IE 10 as well.

"Was it really the intent to abandon SQL as a concept because everybody is using the same well tested and portable library?No. It was the intent to abandon SQL because significant parts (that is of course a udegement call, and you're entitled to judge it differently) of the industry were not using it and not interested in using it - and because most of those who *did* implement it agreed that it was a bad idea.
Incidentally, a half-proposal was recently floated by Ashok Malhotra, to introduce another SQL specification which as well as using SQL for local storage is intended to simplify the problem of synchronising data between online and offline applications. I don't know what the follow-up for that is.The Webapps group will continue to try and reach industry agreement on ways to handle data storage for offline applications. Help is appreciated. Trying

Mail List Etiquette [Was: Re: Two years on and still no sensible web storage solutions exist]

2012-11-12 Thread Arthur Barstow

Todd - your e-mail fails the "respectful manner" requirement of the 
group's e-mail policy (#Mail-List-Etiqutte).


Please restrict the content of your e-mails to respectful technical 
comments.


If you continue to fail to abide with this policy, your posting rights 
to this list will be removed.


-Thanks, AB

#Mail-List-Etiqutte 



On 11/9/12 1:24 PM, ext Todd Blanchard wrote:

Re: Two years on and still no sensible web storage solutions exist

On Mon, Nov 12, 2012 at 10:45 AM, Todd Blanchard
wrote:

> I've settled on using SQLite on smart phones (Android and iOS - we will
> not support phones lacking this capability) and I'm going to just rely on
> calls to the server for desktop apps.  I think fetching from the server
> will be no worse for the average desktop user than iterating a key-value
> store trying to calculate joins in javascript.
>
My observation of websql on phones is that it is very, very slow. I use it,
for some things, but in order to make it usable I had to denormalize my
database to form 1 hierarchical.

Re: Two years on and still no sensible web storage solutions exist

Don't care if it is SQLite so long as it is a real relational database.  
IndexedDB or some other key-value store is no substitute for the power of a 
relational database and is unlikely to be performant given the number of 
different views we support.  I don't mind too much if IE used mssql, Firefox 
settled on mysql, or postgresql or anything else so long as it is a proper 
relational database - switching sql dialects is already a fact of life.

I've settled on using SQLite on smart phones (Android and iOS - we will not 
support phones lacking this capability) and I'm going to just rely on calls to 
the server for desktop apps.  I think fetching from the server will be no worse 
for the average desktop user than iterating a key-value store trying to 
calculate joins in javascript.  

-Todd Blanchard

On Nov 12, 2012, at 1:14 AM, Andrew Wilson wrote:

> My recollection is that some vendors refused to ever support SQLite, making a 
> SQL-based standard not really feasible. You can undoubtedly review the w3c 
> archives to find out more details if you want to know the rationale they 
> expressed at the time.
> 
> It's unfortunate that IndexedDB is still not widely supported by browsers - I 
> share your frustration. However, as you say, you can write to the IndexedDB 
> APIs, and use a shim to get support on platforms that do not support it yet.
> 
> -atw
>

Re: Two years on and still no sensible web storage solutions exist

2012-11-12 Thread Andrew Wilson

My recollection is that some vendors refused to ever support SQLite, making
a SQL-based standard not really feasible. You can undoubtedly review the
w3c archives to find out more details if you want to know the rationale
they expressed at the time.

It's unfortunate that IndexedDB is still not widely supported by browsers -
I share your frustration. However, as you say, you can write to the
IndexedDB APIs, and use a shim to get support on platforms that do not
support it yet.

-atw


On Mon, Nov 12, 2012 at 9:50 AM, Florian Bösch  wrote:

> I'd like to propose as a constructive strategy not to flame/offend
> everybody right off the bat. I'm sure there's reasons, I'd like to hear
> them, too.
>
>
> On Fri, Nov 9, 2012 at 7:24 PM, Todd Blanchard 
> wrote:
>
>> It has been two years since the following little note was attached to the
>> Web SQL Spec
>>
>> This document was on the W3C Recommendation track but specification work
>> has stopped. The specification reached an impasse: all interested
>> implementors have used the same SQL backend (Sqlite), but we need multiple
>> independent implementations to proceed along a standardisation path.
>>
>> This move has left the web browser world in disarray and has been widely
>> misconstrued by readers to mean "Web SQL is deprecated and will not be
>> supported in the future - better port to IndexedDB".
>>
>> Today, TWO YEARS LATER, we have SQLite on iOS, Android, Chrome, and
>> Safari but no IndexedDB.  On Firefox we have IndexedDB with SQLite
>> available only via a browser extension
>> https://addons.mozilla.org/en-US/firefox/addon/html5-websql-for-firefox/ 
>> (annoying
>> but liveable), and on IE[89] only DOM storage with IndexedDB expected on IE
>> 10.
>>
>> Sources: http://caniuse.com/indexeddb http://caniuse.com/sql-storage
>>
>> As someone who is trying to build an offline web app the works both on
>> browsers and smart phones and needs to store a lot of client side complex
>> data that will require lots of joins - let me just say WTF?
>>
>> Why do we have standards again?  You're not helping.
>>
>> I look around at information on the state of storage options and I read
>> stuff like this:
>> "Since November 18, 2010, the W3C announced that Web SQL database is a
>> deprecated specification. This is a recommendation for web developers to no
>> longer use the technology as effectively, the spec will receive no new
>> updates and browser vendors aren't encouraged to support this technology.
>> The new alternative is IndexedDB which is already available on Chrome 12+
>> and Firefox 5+, and, soon, in IE 10 as well.
>> "
>> Was it really the intent to abandon SQL as a concept because everybody is
>> using the same well tested and portable library?  Are we doomed to never
>> ever having a stable and consistent platform to work on?Because there are
>> no competing implementations for browser accessible SQL database access -
>> everybody settled on one nice bit of code to fulfill this requirement - the
>> specification is dropped and the browser developers drop even trying to
>> implement SQL database access and there is even talk of removing it?
>>
>> WTF is wrong with you people?
>>
>> IndexedDB is fine - add it.  But don't for a second try to tell me it is
>> anywhere near as powerful as having a real SQL database on hand.
>>
>> Also, the hand waving about how it should be possible to add SQL on top
>> of IndexedDB rings hollow.  If it were easy we would have it.  OTOH, going
>> the other way seems pretty easy.
>> https://github.com/axemclion/IndexedDBShim maybe the developers should
>> just pursue this approach with SQLite and call it a day.
>>
>> Absolutely disgusted.
>>
>> -Todd Blanchard
>>
>
>

Re: Two years on and still no sensible web storage solutions exist