subject:"Points of order on this WG"



On Jun 26, 2009, at 3:33 PM, Nikunj R. Mehta wrote:

I have a tutorial available to understand how one can use Berkeley  
DB to store data with multiple fields [1]. If you are only  
interested in understanding how to do look up by one or more of  
them, please skip to slide 51.


If this doesn't help, I can write up another explanation for the  
issues that are outstanding.


It sounds like the answer is to make multiple tables with additional  
tables allowing secondary keys to map to the master key. Did I  
understand that correctly? (I'm not sure I got the right idea from the  
pictures).


Can you clarify how a Berkley DB style API would differ from  
LocalStorage in interface or capabilities? What would it be able to do  
that LocalStorage can't?


Regards,
Maciej

Re: Points of order on this WG



On Jun 26, 2009, at 10:51 AM, Nikunj R. Mehta wrote:

Secondly, Oracle proposes adding request interception and  
programmable http cache to the WG's charter. Oracle will provide  
resources for editing and reviewing proposals for all three  
deliverables.


We already have a broad charter and quite a few deliverables. Before  
we add more to the charter, I'd like to understand the degree of  
interest in request interception and programmable http cache. Is  
anyone besides Oracle interested in pursuing this technology? Are any  
implementors interested in implementing it?


Regards,
Maciej

Re: Berkeley DB license (was Re: Points of order on this WG)



On Jun 26, 2009, at 3:46 PM, Nikunj R. Mehta wrote:

FWIW, I came across two pieces about Oracle's open source licensing  
of Berkeley DB that might help clear the air around the licensing  
issues.


First, Oracle's license [1] is word-for-word identical to the  
erstwhile SleepyCat license [2]. Secondly, SleepyCat license  
"qualifies as a free software license, and is compatible with the  
GNU General Public License." [3]. Thirdly, the license is OSI  
approved [4].


I am not sure if this resolves issues. It would help if you had  
comments on the above so that I can keep that in my context while  
discussing with our legal staff.


The issue I see with using Berkeley DB for implementation (which I  
think is only a side issue to design of the spec itself) is as  
follows: Clause 3 of the first license (the one with the Oracle  
copyright notice) appears to have stricter source release requirements  
than LGPL. It's not clear to me what exactly the scope of the  
requirement is, but it doesn't seem to have the dynamic linking or  
relinkable object file exceptions of LGPL. That would be a problem for  
projects like WebKit or Gecko that don't want to impost any  
constraints that go beyond the LGPL in their license terms.


I don't want to start a huge debate over this, I just wanted to  
clarify the issue I see.


Regards,
Maciej

Re: Berkeley DB license (was Re: Points of order on this WG)


On Jun 26, 2009, at 4:06 PM, Maciej Stachowiak wrote:



On Jun 26, 2009, at 3:40 PM, L. David Baron wrote:


On Friday 2009-06-26 15:27 -0700, Nikunj R. Mehta wrote:

I understand the interest in using Berkeley DB in browsers provided
appropriate licensing freedom were available. I am beginning to
understand your concerns vis-à-vis Berkeley DB's license.


To be clear, I wasn't expressing any interest (or disinterest); I
was just commenting on the licensing issues.  I don't have any
opinion on whether we'd want to use it if there weren't licensing
issues (nor would I be the right person to do so).

(I'm just sending this clarification to avoid anyone being under the
incorrect impression that if the license were changed the software
would promptly be incorporated into browsers.  There's still the
issue of convincing browser makers that doing so is important enough
that they'd be willing to support it.)


That's roughly our position for WebKit as well. I did not mean to  
raise the license issue as a showstopper, merely to point out the  
following:


I agree with Maciej - we have gotten far ahead of ourselves here on  
licensing terms.




- If we propose an API modeled on Berkeley DB, it likely could not  
be implemented by the popular open source browser engines using  
Berkeley DB itself.


I don't buy this but...



- If we propose an API modeled on Berkeley DB, it likely could not  
be implemented by proprietary browser engines using Berkeley DB  
itself, unless the developers paid licensing fees to oracle.


there is no free lunch for commercial browsers, at least not one  
that's catered by Oracle,




- Therefore, if we design such an API, we need to be clear and  
detailed enough that it can be implemented interoperably from scratch.


and, regardless of Berkeley DB, this should be the design goal. We  
have all been burned by SQLite and SQL storage, and I am not going to  
lead another train down the same path. I was quite clear in my very  
first message on this topic that we are talking about a B-tree based  
database and not a W3C stamp of approval for Berkeley DB to be  
embedded in browsers.




- We also need to be clear that the implementation cost for any  
browser will likely involve implementation from scratch, not just  
plugging in an existing library.


This is not correct. You and I can disagree, but really we should let  
our lawyers examine the matter.




(If Oracle changed the license terms, things would be different, but  
I'm not asking for that and I don't think it's appropriate to ask at  
this early stage.)


Regards,
Maciej

Re: Berkeley DB license (was Re: Points of order on this WG)



On Jun 26, 2009, at 3:40 PM, L. David Baron wrote:


On Friday 2009-06-26 15:27 -0700, Nikunj R. Mehta wrote:

I understand the interest in using Berkeley DB in browsers provided
appropriate licensing freedom were available. I am beginning to
understand your concerns vis-à-vis Berkeley DB's license.


To be clear, I wasn't expressing any interest (or disinterest); I
was just commenting on the licensing issues.  I don't have any
opinion on whether we'd want to use it if there weren't licensing
issues (nor would I be the right person to do so).

(I'm just sending this clarification to avoid anyone being under the
incorrect impression that if the license were changed the software
would promptly be incorporated into browsers.  There's still the
issue of convincing browser makers that doing so is important enough
that they'd be willing to support it.)


That's roughly our position for WebKit as well. I did not mean to  
raise the license issue as a showstopper, merely to point out the  
following:


- If we propose an API modeled on Berkeley DB, it likely could not be  
implemented by the popular open source browser engines using Berkeley  
DB itself.


- If we propose an API modeled on Berkeley DB, it likely could not be  
implemented by proprietary browser engines using Berkeley DB itself,  
unless the developers paid licensing fees to oracle.


- Therefore, if we design such an API, we need to be clear and  
detailed enough that it can be implemented interoperably from scratch.


- We also need to be clear that the implementation cost for any  
browser will likely involve implementation from scratch, not just  
plugging in an existing library.


(If Oracle changed the license terms, things would be different, but  
I'm not asking for that and I don't think it's appropriate to ask at  
this early stage.)


Regards,
Maciej

Re: Berkeley DB license (was Re: Points of order on this WG)

2009-06-26 Thread Jonas Sicking

On Fri, Jun 26, 2009 at 3:46 PM, Nikunj R. Mehta wrote:
> FWIW, I came across two pieces about Oracle's open source licensing of
> Berkeley DB that might help clear the air around the licensing issues.
>
> First, Oracle's license [1] is word-for-word identical to the erstwhile
> SleepyCat license [2]. Secondly, SleepyCat license "qualifies as a free
> software license, and is compatible with the GNU General Public License."
> [3]. Thirdly, the license is OSI approved [4].
>
> I am not sure if this resolves issues. It would help if you had comments on
> the above so that I can keep that in my context while discussing with our
> legal staff.

Unfortunately this does not resolve the issue. "OSI approved" is
entirely different from compatible with any specific license (GPL,
LGPL, MPL or anything else).

Also, I'm not sure it's entirely fair to simply exclude non
open-source browsers. We want the browser space to be competative,
both for open source browsers and for proprietary browsers. If the API
we come up with is prohibitively complex to implement without either
releasing the browser as open source, or by licensing software from
Oracle or any other party, then I think we haven't designed a good
API.

/ Jonas

Re: Berkeley DB license (was Re: Points of order on this WG)

FWIW, I came across two pieces about Oracle's open source licensing of  
Berkeley DB that might help clear the air around the licensing issues.


First, Oracle's license [1] is word-for-word identical to the  
erstwhile SleepyCat license [2]. Secondly, SleepyCat license  
"qualifies as a free software license, and is compatible with the GNU  
General Public License." [3]. Thirdly, the license is OSI approved [4].


I am not sure if this resolves issues. It would help if you had  
comments on the above so that I can keep that in my context while  
discussing with our legal staff.


Nikunj
http://o-micron.blogspot.com

[1] 
http://www.oracle.com/technology/software/products/berkeley-db/htdocs/oslicense.html
[2] http://opensource.org/licenses/sleepycat.php
[3] http://en.wikipedia.org/wiki/Sleepycat_License
[4] http://opensource.org/trademark

On Jun 26, 2009, at 3:27 PM, Nikunj R. Mehta wrote:


Maciej, David, Jeremy, Doug, others,

I understand the interest in using Berkeley DB in browsers provided  
appropriate licensing freedom were available. I am beginning to  
understand your concerns vis-à-vis Berkeley DB's license.


I have asked our legal team to clarify what they mean by the last  
para of the 3rd clause of the first license. As I understand it, it  
is the following text that appears problematic:


For an executable file, complete source code means the source code  
for all modules it contains.



Although it might be ideal, at this time, I cannot commit to having  
Berkeley DB be offered under a third (besides commercial and its  
current "open source") license. I can only suggest that we move  
forward one step at a time. I will try my best to get this issue  
clarified in the quickest possible time. I also reaffirm the  
approach that it should not be necessary to use Berkeley DB to  
implement the structured storage API Oracle is proposing.


I hope this helps. Feel free to suggest other licensing terms that  
appear problematic.


Nikunj
http://o-micron.blogspot.com

On Jun 26, 2009, at 12:42 PM, L. David Baron wrote:


On Friday 2009-06-26 11:27 -0700, Jonas Sicking wrote:

Note that mozilla has since long made a commitment not to ship code
that is not compatible with all of GPL, LGPL *and* MPL. So unless  
the

BDB license is compatible with all those three we couldn't use BDB.


I think our (Mozilla's) requirement is actually slightly stronger
than license compatibility, at least as defined by
http://en.wikipedia.org/wiki/License_compatibility .  Rather, I
think we require that the licenses don't impose any restrictions in
addition to those imposed by the MPL, the LGPL, or the GPL.  (In
other words, that the license is less restrictive than *each* of
those licenses.)

For what it's worth, the license document in question, located at
http://www.oracle.com/technology/software/products/berkeley-db/htdocs/oslicense.html
appears to suggest that the files in the source code are covered
under three different licenses (although it's not entirely clear to
me what is meant by the concatenation of three licenses, my initial
guess is that it means different parts are covered under different
licenses).  The second and third given appear to me to be the
three-part BSD license (varying by whether the copyright holder is
the UC Regents or Harvard University).  If my quick glance is
correct and this is identical to the three-part BSD license, then I
suspect the second and third licenses are unlikely to be a problem
for Mozilla; we already include code licensed under the three-part
BSD license (see http://opensource.org/licenses/bsd-license.php ).

The first license, on the other hand, appears to be a modified
version of the BSD license, with the third claused replaced by an
entirely different one.  I don't recognize this clause, and I
suspect it would require legal analysis to determine whether it's
less restrictive than the MPL, LGPL, and GPL.  (Though the part that
says "For an executable file, complete source code means the source
code for all modules it contains." seems pretty restrictive to my
untrained eyes.)

-David

--
L. David Baron http://dbaron.org/
Mozilla Corporation   http://www.mozilla.com/

Re: Berkeley DB license (was Re: Points of order on this WG)

2009-06-26 Thread L. David Baron

On Friday 2009-06-26 15:27 -0700, Nikunj R. Mehta wrote:
> I understand the interest in using Berkeley DB in browsers provided  
> appropriate licensing freedom were available. I am beginning to  
> understand your concerns vis-à-vis Berkeley DB's license.

To be clear, I wasn't expressing any interest (or disinterest); I
was just commenting on the licensing issues.  I don't have any
opinion on whether we'd want to use it if there weren't licensing
issues (nor would I be the right person to do so).

(I'm just sending this clarification to avoid anyone being under the
incorrect impression that if the license were changed the software
would promptly be incorporated into browsers.  There's still the
issue of convincing browser makers that doing so is important enough
that they'd be willing to support it.)

-David

-- 
L. David Baron http://dbaron.org/
Mozilla Corporation   http://www.mozilla.com/

Re: Points of order on this WG

I have a tutorial available to understand how one can use Berkeley DB  
to store data with multiple fields [1]. If you are only interested in  
understanding how to do look up by one or more of them, please skip to  
slide 51.


If this doesn't help, I can write up another explanation for the  
issues that are outstanding.


Hope this helps.

Nikunj
http://o-micron.blogspot.com

[1] 
http://www.oracle.com/technology/products/berkeley-db/tutorial-berkeleydb-ds.html

On Jun 26, 2009, at 1:13 PM, Maciej Stachowiak wrote:




It's also not clear to me if a BDB-level API is sufficient for  
developer needs. As I understand it, it's basically a giant  
dictionary with unstructured keys and values. So it's not  
providing much over LocalStorage, except for prefix matching and  
the ability to hold large amounts of records or records that are  
individually large. There's no way to efficiently query by one of  
several fields, as I understand it.


I trust that you are relatively new to storing data with B-trees.  
They are at the heart of Oracle's indices so efficiency is out of  
question. If you are wondering how can people store complex data  
items with multiple fields and repeating values, look at Berkeley  
DB Java Edition, which supports the EJB 3 persistence model [5].  
FYI, there is no significant difference between the APIs of BDB  
Java Edition and the original BDB. They also have identical  
licensing requirements.


Your references do not appear to explain on a technical level how  
one stores data with multiple fields in a way that you can query  
efficiently by more than one of them. I would appreciate a brief  
explanation.

Berkeley DB license (was Re: Points of order on this WG)


Maciej, David, Jeremy, Doug, others,

I understand the interest in using Berkeley DB in browsers provided  
appropriate licensing freedom were available. I am beginning to  
understand your concerns vis-à-vis Berkeley DB's license.


I have asked our legal team to clarify what they mean by the last para  
of the 3rd clause of the first license. As I understand it, it is the  
following text that appears problematic:


For an executable file, complete source code means the source code  
for all modules it contains.



Although it might be ideal, at this time, I cannot commit to having  
Berkeley DB be offered under a third (besides commercial and its  
current "open source") license. I can only suggest that we move  
forward one step at a time. I will try my best to get this issue  
clarified in the quickest possible time. I also reaffirm the approach  
that it should not be necessary to use Berkeley DB to implement the  
structured storage API Oracle is proposing.


I hope this helps. Feel free to suggest other licensing terms that  
appear problematic.


Nikunj
http://o-micron.blogspot.com

On Jun 26, 2009, at 12:42 PM, L. David Baron wrote:


On Friday 2009-06-26 11:27 -0700, Jonas Sicking wrote:

Note that mozilla has since long made a commitment not to ship code
that is not compatible with all of GPL, LGPL *and* MPL. So unless the
BDB license is compatible with all those three we couldn't use BDB.


I think our (Mozilla's) requirement is actually slightly stronger
than license compatibility, at least as defined by
http://en.wikipedia.org/wiki/License_compatibility .  Rather, I
think we require that the licenses don't impose any restrictions in
addition to those imposed by the MPL, the LGPL, or the GPL.  (In
other words, that the license is less restrictive than *each* of
those licenses.)

For what it's worth, the license document in question, located at
http://www.oracle.com/technology/software/products/berkeley-db/htdocs/oslicense.html
appears to suggest that the files in the source code are covered
under three different licenses (although it's not entirely clear to
me what is meant by the concatenation of three licenses, my initial
guess is that it means different parts are covered under different
licenses).  The second and third given appear to me to be the
three-part BSD license (varying by whether the copyright holder is
the UC Regents or Harvard University).  If my quick glance is
correct and this is identical to the three-part BSD license, then I
suspect the second and third licenses are unlikely to be a problem
for Mozilla; we already include code licensed under the three-part
BSD license (see http://opensource.org/licenses/bsd-license.php ).

The first license, on the other hand, appears to be a modified
version of the BSD license, with the third claused replaced by an
entirely different one.  I don't recognize this clause, and I
suspect it would require legal analysis to determine whether it's
less restrictive than the MPL, LGPL, and GPL.  (Though the part that
says "For an executable file, complete source code means the source
code for all modules it contains." seems pretty restrictive to my
untrained eyes.)

-David

--
L. David Baron http://dbaron.org/
Mozilla Corporation   http://www.mozilla.com/

Re: Points of order on this WG

On Jun 26, 2009, at 10:26 AM, Nikunj R. Mehta wrote:

As a side note, it should be noted Berkeley DB itself could not be
used by WebKit or Gecko to implement the spec, because even though
it is open source, the license is not compatible with the LGPL. It
seems unlikely that non-open-source browser engines could use it
either, unless they are willing to pay Oracle for a commercial
license. So it's very important for the spec to be clear and
detailed, because everyone will have to implement it from scratch.

Huh? what? I hope you had read Oracle's BDB license document [3] and
open source FAQ [4]. IANAL, but I can get answers for your specific
concerns in the context of open source Berkeley DB. AFAICT, someone
like Mozilla would not face any trouble with the open source license
of Berkeley DB. YMMV.

I read the license. By my reading, it imposes requirements that go
beyond WebKit's LGPL license or Gecko's BSD/GPL/LGPL tri-license: . Specifically clause 3 of the license.

It's also not clear to me if a BDB-level API is sufficient for
developer needs. As I understand it, it's basically a giant
dictionary with unstructured keys and values. So it's not providing
much over LocalStorage, except for prefix matching and the ability
to hold large amounts of records or records that are individually
large. There's no way to efficiently query by one of several
fields, as I understand it.

I trust that you are relatively new to storing data with B-trees.
They are at the heart of Oracle's indices so efficiency is out of
question. If you are wondering how can people store complex data
items with multiple fields and repeating values, look at Berkeley DB
Java Edition, which supports the EJB 3 persistence model [5]. FYI,
there is no significant difference between the APIs of BDB Java
Edition and the original BDB. They also have identical licensing
requirements.

Your references do not appear to explain on a technical level how one
stores data with multiple fields in a way that you can query
efficiently by more than one of them. I would appreciate a brief
explanation.

Regards,
Maciej

P.S. I would appreciate if you could discuss technical matters without
mock incredulity or condescension.

Re: Points of order on this WG

2009-06-26 Thread L. David Baron

On Friday 2009-06-26 11:27 -0700, Jonas Sicking wrote:
> Note that mozilla has since long made a commitment not to ship code
> that is not compatible with all of GPL, LGPL *and* MPL. So unless the
> BDB license is compatible with all those three we couldn't use BDB.

I think our (Mozilla's) requirement is actually slightly stronger
than license compatibility, at least as defined by
http://en.wikipedia.org/wiki/License_compatibility .  Rather, I
think we require that the licenses don't impose any restrictions in
addition to those imposed by the MPL, the LGPL, or the GPL.  (In
other words, that the license is less restrictive than *each* of
those licenses.)

For what it's worth, the license document in question, located at
http://www.oracle.com/technology/software/products/berkeley-db/htdocs/oslicense.html
appears to suggest that the files in the source code are covered
under three different licenses (although it's not entirely clear to
me what is meant by the concatenation of three licenses, my initial
guess is that it means different parts are covered under different
licenses).  The second and third given appear to me to be the
three-part BSD license (varying by whether the copyright holder is
the UC Regents or Harvard University).  If my quick glance is
correct and this is identical to the three-part BSD license, then I
suspect the second and third licenses are unlikely to be a problem
for Mozilla; we already include code licensed under the three-part
BSD license (see http://opensource.org/licenses/bsd-license.php ).

The first license, on the other hand, appears to be a modified
version of the BSD license, with the third claused replaced by an
entirely different one.  I don't recognize this clause, and I
suspect it would require legal analysis to determine whether it's
less restrictive than the MPL, LGPL, and GPL.  (Though the part that
says "For an executable file, complete source code means the source
code for all modules it contains." seems pretty restrictive to my
untrained eyes.)

-David

-- 
L. David Baron http://dbaron.org/
Mozilla Corporation   http://www.mozilla.com/

Re: Points of order on this WG

2009-06-26 Thread Jonas Sicking

On Fri, Jun 26, 2009 at 11:16 AM, Nikunj R.
Mehta wrote:
 As a side note, it should be noted Berkeley DB itself could not be used
 by WebKit or Gecko to implement the spec, because even though it is open
 source, the license is not compatible with the LGPL. It seems unlikely that
 non-open-source browser engines could use it either, unless they are 
 willing
 to pay Oracle for a commercial license. So it's very important for the spec
 to be clear and detailed, because everyone will have to implement it from
 scratch.
>>>
>>> Huh? what? I hope you had read Oracle's BDB license document [3] and open
>>> source FAQ [4].
>>
>> This is the type of thing I'd expect you to say had those links clearly
>> stated GPL, LGPL, etc compatibility.  Am I missing it? Because I don't see
>> anything that makes it clear.
>
> Oracle's license is what it is. I don't see why I should commit to offering
> it under GPL or LGPL.

Note that mozilla has since long made a commitment not to ship code
that is not compatible with all of GPL, LGPL *and* MPL. So unless the
BDB license is compatible with all those three we couldn't use BDB.

That is what Maciej was saying, and it seems you are confirming that?

Open sources licenses are complicated.

/ Jonas

Re: Points of order on this WG


On Jun 26, 2009, at 10:56 AM, Jeremy Orlow wrote:

On Fri, Jun 26, 2009 at 10:26 AM, Nikunj R. Mehta > wrote:
Please don't skimp on due diligence before making such strong  
statements. It creates unnecessary friction. More details below.


Similarly, I'd ask you to make your points clearer and to read what  
others say more closely.  You didn't address Maciej's points very  
well, and I think they're definitely worth addressing.


I understand your point, even though I don't think I missed anything  
in Maciej's comments, unless you want me to give sentence by sentence  
and word by word clarifications. Still, I can try one more time. If I  
miss this time, it is not intentional, and you can just ask me to  
clarify.




On Jun 25, 2009, at 10:49 PM, Maciej Stachowiak wrote:

On Jun 25, 2009, at 5:23 PM, Jonas Sicking wrote:

On Thu, Jun 25, 2009 at 12:42 PM, Nikunj R.
Mehta wrote:
On Jun 24, 2009, at 11:35 PM, Ian Hickson wrote:
I have proposed to Mozilla a solution that provides access to an  
organized
key-value database such as that provided in the (open source)  
Berkeley DB.
In essence, a database is a simple B-tree - it keeps keys sorted and  
permits
duplicate keys. It is able to find a key or a key prefix, which  
enables

efficient searching through a very large number of items. If we are
ambitious (i.e., need more functionality), we can add indexes and  
joins to
this spec. There is unlikely to be an interoperability nightmare,  
such as
that which is the most likely outcome with SQL, it does not mandate  
the use
of any query language, and there is at least 40 years of experience  
with it,

including in highly resource-constrained environments. (There are 200
million copies of Berkeley DB in deployment [1]).

This is what so far seems like the best solution to me. I.e. something
that is more backend-ish than what a SQL API would be.

I'd love to see something that allows you to implement a SQL API on
top of. But that also allows you to implement something like MungoDB
very effectively.

I doubt you can efficiently or correctly implement SQL on top of a  
Berkeley-DB-style API.


If you are worrying, that's great; we can address your worries.

Just take a look at the top two hits for "MySQL Berkeley DB". The  
first one talks about MySQL with the BDB storage engine - so yes,  
you can correctly implement SQL on top of Berkeley DB [1]. The  
second one talks about MySQL discontinuing BDB support due to extra- 
technical reasons, so there are no efficiency reasons either [2].




As a side note, it should be noted Berkeley DB itself could not be  
used by WebKit or Gecko to implement the spec, because even though  
it is open source, the license is not compatible with the LGPL. It  
seems unlikely that non-open-source browser engines could use it  
either, unless they are willing to pay Oracle for a commercial  
license. So it's very important for the spec to be clear and  
detailed, because everyone will have to implement it from scratch.


Huh? what? I hope you had read Oracle's BDB license document [3] and  
open source FAQ [4].


This is the type of thing I'd expect you to say had those links  
clearly stated GPL, LGPL, etc compatibility.  Am I missing it?  
Because I don't see anything that makes it clear.


Oracle's license is what it is. I don't see why I should commit to  
offering it under GPL or LGPL.




IANAL, but I can get answers for your specific concerns in the  
context of open source Berkeley DB. AFAICT, someone like Mozilla  
would not face any trouble with the open source license of Berkeley  
DB. YMMV.


What do you mean by your mileage may vary?  Are you saying that  
perhaps it is exactly as Maciej said?


If you have a closed source browser then Berkeley DB will require  
commercial licensing. If your product is open source, then Berkeley DB  
is free for you. If you are somewhere in between, then you need to ask  
your lawyers. If you need something from Oracle, then by all means I  
can help you with that.


Before believing Maciej, you might do well to read the links I sent in  
my previous email, or ask your lawyers to review the license. I  
understand that it is more work for you, but Oracle's at least helping  
to the extent it can.







It's also not clear to me if a BDB-level API is sufficient for  
developer needs. As I understand it, it's basically a giant  
dictionary with unstructured keys and values. So it's not providing  
much over LocalStorage, except for prefix matching and the ability  
to hold large amounts of records or records that are individually  
large. There's no way to efficiently query by one of several fields,  
as I understand it.


I trust that you are relatively new to storing data with B-trees.  
They are at the heart of Oracle's indices so efficiency is out of  
question. If you are wondering how can people store complex data  
items with multiple fields and repeating values, look at Berkeley DB  
Java Edition, which supports the EJB

Re: Berkeley DB (was: Points of order on this WG)


On Jun 26, 2009, at 1:07 AM, Jonas Sicking wrote:

On Fri, Jun 26, 2009 at 12:58 AM, Doug Schepers  
wrote:

Hi, Maciej-

Maciej Stachowiak wrote (on 6/26/09 1:49 AM):


As a side note, it should be noted Berkeley DB itself could not be  
used
by WebKit or Gecko to implement the spec, because even though it  
is open
source, the license is not compatible with the LGPL. It seems  
unlikely
that non-open-source browser engines could use it either, unless  
they

are willing to pay Oracle for a commercial license. So it's very
important for the spec to be clear and detailed, because everyone  
will

have to implement it from scratch.


I wonder if Oracle would be willing to back the Berkeley DB option by
changing the license?


I don't think we should tie a web API to a specific library. Just as I
think specifying a SQL storage to exactly follow a given version of
SQLite, I would think it's a bad idea to follow a given version of
Berkeley DB.



I agree with the principle of not tying the API to a specific library.  
To the extent that this WG provides feedback, I would be inclined to  
keep the API independent of Berkeley DB. That said, there is inherent  
benefit to starting with an API which could be easily implemented and  
be mature enough from the beginning.


Therefore, my approach would be to use a subset of Berkeley DB's API  
that is justified based on the requirements considered appropriate by  
this WG.



The idea isn't to use BDB specifically. The idea is to provide the
type of data structures that SQL databases use to implement their
tables.


You got it right.



Berkeley DB used to be available as a backend to MySQL, so clearly it
is possible to implement SQL on top of BDB. However it appears that
MySQL no longer is able to run on top of BDB, so there is probably a
reason for that too.


To be precise, MySQL has deprecated support for BDB as a storage  
engine starting with v5.1.12. Who know what will happen in the future  
though?


Nikunj
http://o-micron.blogspot.com

Re: Points of order on this WG


On Jun 26, 2009, at 12:56 AM, Doug Schepers wrote:


Hi, Folks-

Maciej Stachowiak wrote (on 6/25/09 7:20 PM):


On Jun 24, 2009, at 11:35 PM, Ian Hickson wrote:

I think Nikunj's proposal definitely is worthy of being persued,  
just

like
the working group is persuing dozens of other proposals like XHR,  
CORS,
Selectors API, Workers, Server-Sent Events, Web Sockets, etc. I  
don't
believe it really fits into the Web Storage spec (if anything, I  
think we
should split Web Storage into two further specs, not add a third  
wholly
independent feature to it). However, I would definitely support an  
FPWD

publication of Nikunj's proposal, as I have for other proposals.


I strongly agree on these points. I would prefer to see SQL Storage
split out of the rest of Web Storage. We seem to have rough consensus
and strong multilateral implementor interest on LocalStorage and
SessionStorage, and they should be allowed to move forward on the
standards track quickly. SQL Storage remains contentious, and only  
Apple
and Google have shown strong implementor interest so far. And it  
has no

technical tie to the other storage drafts. I also think Nikunj's
proposal should be yet a third separate orthogonal draft.


Art, Chaals, Mike, and I discussed this yesterday, and we agreed  
that this seems like the best solution.  Like the Widgets work, a  
deliverable doesn't necessarily have to be in a single spec, so we  
believe there is sufficient justification for this in the charter.


The plan of record would be to split out the SQL Storage section  
into its own spec, with an alternate spec edited by Nikunj, and to  
publish an updated draft of Web Storage that points to both those  
other drafts. This way, all parts of the web storage deliverable are  
put on a level playing field to be judged on their individual  
merits, and subject to being edited and updated individually.


Nikunj, would this suit you?  Does anyone else have any thoughts?


I would be pleased to edit an alternate spec for structured storage.  
After the charter snafu earlier in April, we discussed the need for  
WG's charter to state a desire to standardize structured storage (as  
opposed to limiting to SQL storage). Still it would be good if the  
charter clarifies this.


Secondly, Oracle proposes adding request interception and programmable  
http cache to the WG's charter. Oracle will provide resources for  
editing and reviewing proposals for all three deliverables.


That being said, I am really glad that the WG was able to arrive at a  
swift and positive conclusion on this matter. I want to take a moment  
and appreciate the help of all those who weighed in on Oracle's  
concerns and look forward to working together and constructively to  
remove the concerns.


Best regards,
Nikunj
http://o-micron.blogspot.com

Re: Points of order on this WG


On Jun 26, 2009, at 12:15 AM, Robin Berjon wrote:


On Jun 26, 2009, at 07:49 , Maciej Stachowiak wrote:
It's also not clear to me if a BDB-level API is sufficient for  
developer needs.


That's something that we should nail down early this time around. I  
tend to think that "sufficient for developer needs" means good  
enough that one can implement something like Persevere / MongoDB /  
KiokuDB on top of it, and several others have made similar comments.  
I'm unsure as to how exactly that translates into requirements  
without tying us to the implementation strategy of a couple products.


I agree. This is why I am making an attempt to write down requirements  
ahead of time. I would suggest formally publishing and taking comments  
on the requirements, so we can avoid trouble.


Nikunj
http://o-micron.blogspot.com

Re: Points of order on this WG


On Jun 25, 2009, at 5:23 PM, Jonas Sicking wrote:


On Thu, Jun 25, 2009 at 12:42 PM, Nikunj R.
Mehta wrote:

On Jun 24, 2009, at 11:35 PM, Ian Hickson wrote:
I have proposed to Mozilla a solution that provides access to an  
organized
key-value database such as that provided in the (open source)  
Berkeley DB.
In essence, a database is a simple B-tree - it keeps keys sorted  
and permits
duplicate keys. It is able to find a key or a key prefix, which  
enables

efficient searching through a very large number of items. If we are
ambitious (i.e., need more functionality), we can add indexes and  
joins to
this spec. There is unlikely to be an interoperability nightmare,  
such as
that which is the most likely outcome with SQL, it does not mandate  
the use
of any query language, and there is at least 40 years of experience  
with it,

including in highly resource-constrained environments. (There are 200
million copies of Berkeley DB in deployment [1]).


This is what so far seems like the best solution to me. I.e. something
that is more backend-ish than what a SQL API would be.

I'd love to see something that allows you to implement a SQL API on
top of. But that also allows you to implement something like MungoDB
very effectively.


You could implement SQL API on top of Berkeley DB as some have in the  
past. I am not super confident that the whole thing can work out with  
JavaScript overhead, but that may be just my perception of JS. This  
may be one of those conclusions that can only be made after a painful  
implementation exercise.

Re: Points of order on this WG


On Jun 25, 2009, at 4:25 PM, Maciej Stachowiak wrote:



On Jun 25, 2009, at 12:42 PM, Nikunj R. Mehta wrote:



I think Nikunj's proposal definitely is worthy of being persued,  
just like
the working group is persuing dozens of other proposals like XHR,  
CORS,
Selectors API, Workers, Server-Sent Events, Web Sockets, etc. I  
don't
believe it really fits into the Web Storage spec (if anything, I  
think we
should split Web Storage into two further specs, not add a third  
wholly
independent feature to it). However, I would definitely support an  
FPWD

publication of Nikunj's proposal, as I have for other proposals.


That is encouraging. I will be glad to edit an FPWD that includes B- 
tree, interception, and programmable cache, if the WG so prefers.




It seems to me that Berkley DB style database storage, and request  
interception / programmable cache are orthogonal ideas and should  
arguably be separate drafts. I would assume request interception and  
programmable cache are usable regardless of what client-side storage  
APIs are available, much as HTML5 Application Cache is independent  
of these APIs. If anything, it seems more closely related to  
AppCache than to any proposed storage solution.




I don't have a preference either way. Request interception and  
programmable cache belong in a single spec. We could put Structured  
storage in a separate spec on its own.



Regards,
Maciej

Re: Points of order on this WG

Please don't skimp on due diligence before making such strong  
statements. It creates unnecessary friction. More details below.


On Jun 25, 2009, at 10:49 PM, Maciej Stachowiak wrote:


On Jun 25, 2009, at 5:23 PM, Jonas Sicking wrote:


On Thu, Jun 25, 2009 at 12:42 PM, Nikunj R.
Mehta wrote:

On Jun 24, 2009, at 11:35 PM, Ian Hickson wrote:
I have proposed to Mozilla a solution that provides access to an  
organized
key-value database such as that provided in the (open source)  
Berkeley DB.
In essence, a database is a simple B-tree - it keeps keys sorted  
and permits
duplicate keys. It is able to find a key or a key prefix, which  
enables

efficient searching through a very large number of items. If we are
ambitious (i.e., need more functionality), we can add indexes and  
joins to
this spec. There is unlikely to be an interoperability nightmare,  
such as
that which is the most likely outcome with SQL, it does not  
mandate the use
of any query language, and there is at least 40 years of  
experience with it,
including in highly resource-constrained environments. (There are  
200

million copies of Berkeley DB in deployment [1]).


This is what so far seems like the best solution to me. I.e.  
something

that is more backend-ish than what a SQL API would be.

I'd love to see something that allows you to implement a SQL API on
top of. But that also allows you to implement something like MungoDB
very effectively.


I doubt you can efficiently or correctly implement SQL on top of a  
Berkeley-DB-style API.


If you are worrying, that's great; we can address your worries.

Just take a look at the top two hits for "MySQL Berkeley DB". The  
first one talks about MySQL with the BDB storage engine - so yes, you  
can correctly implement SQL on top of Berkeley DB [1]. The second one  
talks about MySQL discontinuing BDB support due to extra-technical  
reasons, so there are no efficiency reasons either [2].




As a side note, it should be noted Berkeley DB itself could not be  
used by WebKit or Gecko to implement the spec, because even though  
it is open source, the license is not compatible with the LGPL. It  
seems unlikely that non-open-source browser engines could use it  
either, unless they are willing to pay Oracle for a commercial  
license. So it's very important for the spec to be clear and  
detailed, because everyone will have to implement it from scratch.


Huh? what? I hope you had read Oracle's BDB license document [3] and  
open source FAQ [4]. IANAL, but I can get answers for your specific  
concerns in the context of open source Berkeley DB. AFAICT, someone  
like Mozilla would not face any trouble with the open source license  
of Berkeley DB. YMMV.




It's also not clear to me if a BDB-level API is sufficient for  
developer needs. As I understand it, it's basically a giant  
dictionary with unstructured keys and values. So it's not providing  
much over LocalStorage, except for prefix matching and the ability  
to hold large amounts of records or records that are individually  
large. There's no way to efficiently query by one of several fields,  
as I understand it.


I trust that you are relatively new to storing data with B-trees. They  
are at the heart of Oracle's indices so efficiency is out of question.  
If you are wondering how can people store complex data items with  
multiple fields and repeating values, look at Berkeley DB Java  
Edition, which supports the EJB 3 persistence model [5]. FYI, there is  
no significant difference between the APIs of BDB Java Edition and the  
original BDB. They also have identical licensing requirements.


[1] http://dev.mysql.com/doc/refman/5.0/en/bdb-storage-engine.html
[2] http://www.linux.com/archive/articles/56835
[3] 
http://www.oracle.com/technology/software/products/berkeley-db/htdocs/oslicense.html
[4] 
http://www.oracle.com/technology/software/products/berkeley-db/htdocs/licensing.html
[5] http://www.oracle.com/technology/products/berkeley-db/je/index.html

RE: Points of order on this WG

2009-06-26 Thread Marcin Hanclik

>>Where it is specified did not really have an
>>impact on whether we would do it or not.
Agreed. The place does not matter. Stability does, IMHO.

>>I would also be somewhat hesitant
>>to say that separate drafts necessarily move faster.
At least there is a chance.
It seems more feasible to implement a set of interfaces one by one - if they 
are independent - then to do it all at once.

Separation on the spec level may also result in better specs, since the 
interface interdependencies would be avoided for practical reasons.

Thanks,
Marcin

Marcin Hanclik
ACCESS Systems Germany GmbH
Tel: +49-208-8290-6452  |  Fax: +49-208-8290-6465
Mobile: +49-163-8290-646
E-Mail: marcin.hanc...@access-company.com

-Original Message-
From: Anne van Kesteren [mailto:ann...@opera.com]
Sent: Friday, June 26, 2009 11:57 AM
To: Marcin Hanclik; Maciej Stachowiak; Ian Hickson
Cc: Doug Schepers; Nikunj R. Mehta; public-webapps WG; Charles McCathieNevile; 
Arthur Barstow; Jeff Mischkinsky
Subject: Re: Points of order on this WG

On Fri, 26 Jun 2009 10:09:43 +0200, Marcin Hanclik
 wrote:
> +1
> Stable specification moving faster in the standards track will
> definitely bring more implementations.

To be clear, when we decided to implement this feature it was still part
of the HTML5 specification. Where it is specified did not really have an
impact on whether we would do it or not. I would also be somewhat hesitant
to say that separate drafts necessarily move faster.

--
Anne van Kesteren
http://annevankesteren.nl/

Access Systems Germany GmbH
Essener Strasse 5  |  D-46047 Oberhausen
HRB 13548 Amtsgericht Duisburg
Geschaeftsfuehrer: Michel Piquemal, Tomonori Watanabe, Yusuke Kanda

www.access-company.com

CONFIDENTIALITY NOTICE
This e-mail and any attachments hereto may contain information that is 
privileged or confidential, and is intended for use only by the
individual or entity to which it is addressed. Any disclosure, copying or 
distribution of the information by anyone else is strictly prohibited.
If you have received this document in error, please notify us promptly by 
responding to this e-mail. Thank you.

Re: Points of order on this WG

2009-06-26 Thread Anne van Kesteren

On Fri, 26 Jun 2009 10:09:43 +0200, Marcin Hanclik  
 wrote:

+1
Stable specification moving faster in the standards track will  
definitely bring more implementations.


To be clear, when we decided to implement this feature it was still part  
of the HTML5 specification. Where it is specified did not really have an  
impact on whether we would do it or not. I would also be somewhat hesitant  
to say that separate drafts necessarily move faster.



--
Anne van Kesteren
http://annevankesteren.nl/

Re: Points of order on this WG

2009-06-26 Thread Anne van Kesteren

On Fri, 26 Jun 2009 10:43:10 +0200, Maciej Stachowiak   
wrote:

On Jun 26, 2009, at 12:23 AM, Anne van Kesteren wrote:

FWIW, Opera is implementing SQL storage too.


That's great news! Having multiple independent implementations will, I  
hope, provide more reason to advance the spec.


However, I still think SQL storage should be split from LocalStorage/ 
SessionStorage, since there is no technical reason for them to be tied  
together, and they enjoy different levels of consensus and implementor  
support.


Yeah, that seems fine. That way localStorage/sessionStorage can move ahead  
while Web SQL is properly defined.



--
Anne van Kesteren
http://annevankesteren.nl/

Re: Points of order on this WG

2009-06-26 Thread Robin Berjon


On Jun 26, 2009, at 10:54 , Maciej Stachowiak wrote:
I don't think the Web Storage draft (I assume by this you mean the  
remaining draft that would define LocalStorage and SessionStorage)  
needs to link to either of the other drafts.


It is customary, when something is split out of a draft, to link to it  
so that people can find it using their old bookmarks. I think that's  
all that Doug meant here — not linking in the normative sense.


I should add that I still think SQL Storage is a good technical  
solution to the problem of structured client-side storage. Web  
developers who are specifically targeting mobile devices, or in  
particular iPhone, have given extremely positive feedback about both  
LocalStorage and SQL Storage, as well as the HTML5 Application  
Cache. On general-purpose Web sites, of course, uptake is limited by  
the lack of other implementations so far. But Web developers seem  
positive about it as a technology, based on feedback from  
presentations. All of this makes me doubt that a fundamentally  
different model for structured storage is needed or would be  
significantly better.


Well, the advantage of SQL is that developers know it, and have been  
wanting something like that on the client for ages (I know I have).  
There are non-negligible issues however in defining it interoperably.  
Also, I think there's a possibility that it's popular with developers  
simply because it's the only option. That's why ideally I'd like to  
give us the leeway to experiment a little around various options  
before committing completely to SQL.


--
Robin Berjon - http://berjon.com/
Feel like hiring me? Go to http://robineko.com/

Re: Points of order on this WG



On Jun 26, 2009, at 12:56 AM, Doug Schepers wrote:



Art, Chaals, Mike, and I discussed this yesterday, and we agreed  
that this seems like the best solution.  Like the Widgets work, a  
deliverable doesn't necessarily have to be in a single spec, so we  
believe there is sufficient justification for this in the charter.


The plan of record would be to split out the SQL Storage section  
into its own spec, with an alternate spec edited by Nikunj, and to  
publish an updated draft of Web Storage that points to both those  
other drafts. This way, all parts of the web storage deliverable are  
put on a level playing field to be judged on their individual  
merits, and subject to being edited and updated individually.


I don't think the Web Storage draft (I assume by this you mean the  
remaining draft that would define LocalStorage and SessionStorage)  
needs to link to either of the other drafts. I also don't think  
proposals for other storage APIs need to be "alternate"; they could be  
orthogonal non-overlapping specs and implementors could choose which  
make sense for them.


I should add that I still think SQL Storage is a good technical  
solution to the problem of structured client-side storage. Web  
developers who are specifically targeting mobile devices, or in  
particular iPhone, have given extremely positive feedback about both  
LocalStorage and SQL Storage, as well as the HTML5 Application Cache.  
On general-purpose Web sites, of course, uptake is limited by the lack  
of other implementations so far. But Web developers seem positive  
about it as a technology, based on feedback from presentations. All of  
this makes me doubt that a fundamentally different model for  
structured storage is needed or would be significantly better.


That being said, I would like to see the technology succeed on its own  
merits and not by being tie to the so far more widely deployed  
technology of LocalStorage/SessionStorage.


Regards,
Maciej

Re: Points of order on this WG



On Jun 26, 2009, at 12:23 AM, Anne van Kesteren wrote:

On Fri, 26 Jun 2009 01:20:43 +0200, Maciej Stachowiak  
 wrote:
I strongly agree on these points. I would prefer to see SQL Storage  
split out of the rest of Web Storage. We seem to have rough  
consensus and strong multilateral implementor interest on  
LocalStorage and SessionStorage, and they should be allowed to move  
forward on the standards track quickly. SQL Storage remains  
contentious, and only Apple and Google have shown strong  
implementor interest so far. And it has no technical tie to the  
other storage drafts. I also think Nikunj's proposal should be yet  
a third separate orthogonal draft.


FWIW, Opera is implementing SQL storage too.


That's great news! Having multiple independent implementations will, I  
hope, provide more reason to advance the spec.


However, I still think SQL storage should be split from LocalStorage/ 
SessionStorage, since there is no technical reason for them to be tied  
together, and they enjoy different levels of consensus and implementor  
support.


Regards,
Maciej

Re: Points of order on this WG

2009-06-26 Thread Ian Hickson

On Fri, 26 Jun 2009, Doug Schepers wrote:
> 
> The plan of record would be to split out the SQL Storage section into 
> its own spec, with an alternate spec edited by Nikunj, and to publish an 
> updated draft of Web Storage that points to both those other drafts. 
> This way, all parts of the web storage deliverable are put on a level 
> playing field to be judged on their individual merits, and subject to 
> being edited and updated individually.

I've added "split the Web Storage spec" to my list of things to do and 
will hopefully get to that within the next few weeks. It shouldn't be much 
work.

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

RE: Points of order on this WG

2009-06-26 Thread Marcin Hanclik

+1
Stable specification moving faster in the standards track will definitely bring 
more implementations.

Marcin Hanclik
ACCESS Systems Germany GmbH
Tel: +49-208-8290-6452  |  Fax: +49-208-8290-6465
Mobile: +49-163-8290-646
E-Mail: marcin.hanc...@access-company.com

-Original Message-
From: public-webapps-requ...@w3.org [mailto:public-webapps-requ...@w3.org] On 
Behalf Of Anne van Kesteren
Sent: Friday, June 26, 2009 9:23 AM
To: Maciej Stachowiak; Ian Hickson
Cc: Doug Schepers; Nikunj R. Mehta; public-webapps WG; Charles McCathieNevile; 
Arthur Barstow; Jeff Mischkinsky
Subject: Re: Points of order on this WG

On Fri, 26 Jun 2009 01:20:43 +0200, Maciej Stachowiak 
wrote:
> I strongly agree on these points. I would prefer to see SQL Storage
> split out of the rest of Web Storage. We seem to have rough consensus
> and strong multilateral implementor interest on LocalStorage and
> SessionStorage, and they should be allowed to move forward on the
> standards track quickly. SQL Storage remains contentious, and only Apple
> and Google have shown strong implementor interest so far. And it has no
> technical tie to the other storage drafts. I also think Nikunj's
> proposal should be yet a third separate orthogonal draft.

FWIW, Opera is implementing SQL storage too.


--
Anne van Kesteren
http://annevankesteren.nl/




Access Systems Germany GmbH
Essener Strasse 5  |  D-46047 Oberhausen
HRB 13548 Amtsgericht Duisburg
Geschaeftsfuehrer: Michel Piquemal, Tomonori Watanabe, Yusuke Kanda

www.access-company.com

CONFIDENTIALITY NOTICE
This e-mail and any attachments hereto may contain information that is 
privileged or confidential, and is intended for use only by the
individual or entity to which it is addressed. Any disclosure, copying or 
distribution of the information by anyone else is strictly prohibited.
If you have received this document in error, please notify us promptly by 
responding to this e-mail. Thank you.

Re: Berkeley DB (was: Points of order on this WG)

2009-06-26 Thread Jonas Sicking

On Fri, Jun 26, 2009 at 12:58 AM, Doug Schepers wrote:
> Hi, Maciej-
>
> Maciej Stachowiak wrote (on 6/26/09 1:49 AM):
>>
>> As a side note, it should be noted Berkeley DB itself could not be used
>> by WebKit or Gecko to implement the spec, because even though it is open
>> source, the license is not compatible with the LGPL. It seems unlikely
>> that non-open-source browser engines could use it either, unless they
>> are willing to pay Oracle for a commercial license. So it's very
>> important for the spec to be clear and detailed, because everyone will
>> have to implement it from scratch.
>
> I wonder if Oracle would be willing to back the Berkeley DB option by
> changing the license?

I don't think we should tie a web API to a specific library. Just as I
think specifying a SQL storage to exactly follow a given version of
SQLite, I would think it's a bad idea to follow a given version of
Berkeley DB.

The idea isn't to use BDB specifically. The idea is to provide the
type of data structures that SQL databases use to implement their
tables.

Berkeley DB used to be available as a backend to MySQL, so clearly it
is possible to implement SQL on top of BDB. However it appears that
MySQL no longer is able to run on top of BDB, so there is probably a
reason for that too.

/ Jonas

Berkeley DB (was: Points of order on this WG)

2009-06-26 Thread Doug Schepers


Hi, Maciej-

Maciej Stachowiak wrote (on 6/26/09 1:49 AM):


As a side note, it should be noted Berkeley DB itself could not be used
by WebKit or Gecko to implement the spec, because even though it is open
source, the license is not compatible with the LGPL. It seems unlikely
that non-open-source browser engines could use it either, unless they
are willing to pay Oracle for a commercial license. So it's very
important for the spec to be clear and detailed, because everyone will
have to implement it from scratch.


I wonder if Oracle would be willing to back the Berkeley DB option by 
changing the license?


Regards-
-Doug Schepers
W3C Team Contact, SVG and WebApps WGs

Re: Points of order on this WG

2009-06-26 Thread Doug Schepers


Hi, Folks-

Maciej Stachowiak wrote (on 6/25/09 7:20 PM):


On Jun 24, 2009, at 11:35 PM, Ian Hickson wrote:


I think Nikunj's proposal definitely is worthy of being persued, just
like
the working group is persuing dozens of other proposals like XHR, CORS,
Selectors API, Workers, Server-Sent Events, Web Sockets, etc. I don't
believe it really fits into the Web Storage spec (if anything, I think we
should split Web Storage into two further specs, not add a third wholly
independent feature to it). However, I would definitely support an FPWD
publication of Nikunj's proposal, as I have for other proposals.


I strongly agree on these points. I would prefer to see SQL Storage
split out of the rest of Web Storage. We seem to have rough consensus
and strong multilateral implementor interest on LocalStorage and
SessionStorage, and they should be allowed to move forward on the
standards track quickly. SQL Storage remains contentious, and only Apple
and Google have shown strong implementor interest so far. And it has no
technical tie to the other storage drafts. I also think Nikunj's
proposal should be yet a third separate orthogonal draft.


Art, Chaals, Mike, and I discussed this yesterday, and we agreed that 
this seems like the best solution.  Like the Widgets work, a deliverable 
doesn't necessarily have to be in a single spec, so we believe there is 
sufficient justification for this in the charter.


The plan of record would be to split out the SQL Storage section into 
its own spec, with an alternate spec edited by Nikunj, and to publish an 
updated draft of Web Storage that points to both those other drafts. 
This way, all parts of the web storage deliverable are put on a level 
playing field to be judged on their individual merits, and subject to 
being edited and updated individually.


Nikunj, would this suit you?  Does anyone else have any thoughts?

Regards-
-Doug Schepers
W3C Team Contact, SVG and WebApps WGs

Re: Points of order on this WG

2009-06-26 Thread Anne van Kesteren

On Fri, 26 Jun 2009 01:20:43 +0200, Maciej Stachowiak   
wrote:
I strongly agree on these points. I would prefer to see SQL Storage  
split out of the rest of Web Storage. We seem to have rough consensus  
and strong multilateral implementor interest on LocalStorage and  
SessionStorage, and they should be allowed to move forward on the  
standards track quickly. SQL Storage remains contentious, and only Apple  
and Google have shown strong implementor interest so far. And it has no  
technical tie to the other storage drafts. I also think Nikunj's  
proposal should be yet a third separate orthogonal draft.


FWIW, Opera is implementing SQL storage too.


--
Anne van Kesteren
http://annevankesteren.nl/

Re: Points of order on this WG

2009-06-26 Thread Robin Berjon


On Jun 26, 2009, at 07:49 , Maciej Stachowiak wrote:
It's also not clear to me if a BDB-level API is sufficient for  
developer needs.


That's something that we should nail down early this time around. I  
tend to think that "sufficient for developer needs" means good enough  
that one can implement something like Persevere / MongoDB / KiokuDB on  
top of it, and several others have made similar comments. I'm unsure  
as to how exactly that translates into requirements without tying us  
to the implementation strategy of a couple products.


--
Robin Berjon - http://berjon.com/
Feel like hiring me? Go to http://robineko.com/

Re: Points of order on this WG

2009-06-25 Thread Maciej Stachowiak



On Jun 25, 2009, at 5:23 PM, Jonas Sicking wrote:


On Thu, Jun 25, 2009 at 12:42 PM, Nikunj R.
Mehta wrote:

On Jun 24, 2009, at 11:35 PM, Ian Hickson wrote:
I have proposed to Mozilla a solution that provides access to an  
organized
key-value database such as that provided in the (open source)  
Berkeley DB.
In essence, a database is a simple B-tree - it keeps keys sorted  
and permits
duplicate keys. It is able to find a key or a key prefix, which  
enables

efficient searching through a very large number of items. If we are
ambitious (i.e., need more functionality), we can add indexes and  
joins to
this spec. There is unlikely to be an interoperability nightmare,  
such as
that which is the most likely outcome with SQL, it does not mandate  
the use
of any query language, and there is at least 40 years of experience  
with it,

including in highly resource-constrained environments. (There are 200
million copies of Berkeley DB in deployment [1]).


This is what so far seems like the best solution to me. I.e. something
that is more backend-ish than what a SQL API would be.

I'd love to see something that allows you to implement a SQL API on
top of. But that also allows you to implement something like MungoDB
very effectively.


I doubt you can efficiently or correctly implement SQL on top of a  
Berkeley-DB-style API.


As a side note, it should be noted Berkeley DB itself could not be  
used by WebKit or Gecko to implement the spec, because even though it  
is open source, the license is not compatible with the LGPL. It seems  
unlikely that non-open-source browser engines could use it either,  
unless they are willing to pay Oracle for a commercial license. So  
it's very important for the spec to be clear and detailed, because  
everyone will have to implement it from scratch.


It's also not clear to me if a BDB-level API is sufficient for  
developer needs. As I understand it, it's basically a giant dictionary  
with unstructured keys and values. So it's not providing much over  
LocalStorage, except for prefix matching and the ability to hold large  
amounts of records or records that are individually large. There's no  
way to efficiently query by one of several fields, as I understand it.


Regards,
Maciej

Re: Points of order on this WG

2009-06-25 Thread Jonas Sicking

On Thu, Jun 25, 2009 at 12:42 PM, Nikunj R.
Mehta wrote:
> On Jun 24, 2009, at 11:35 PM, Ian Hickson wrote:
> I have proposed to Mozilla a solution that provides access to an organized
> key-value database such as that provided in the (open source) Berkeley DB.
> In essence, a database is a simple B-tree - it keeps keys sorted and permits
> duplicate keys. It is able to find a key or a key prefix, which enables
> efficient searching through a very large number of items. If we are
> ambitious (i.e., need more functionality), we can add indexes and joins to
> this spec. There is unlikely to be an interoperability nightmare, such as
> that which is the most likely outcome with SQL, it does not mandate the use
> of any query language, and there is at least 40 years of experience with it,
> including in highly resource-constrained environments. (There are 200
> million copies of Berkeley DB in deployment [1]).

This is what so far seems like the best solution to me. I.e. something
that is more backend-ish than what a SQL API would be.

I'd love to see something that allows you to implement a SQL API on
top of. But that also allows you to implement something like MungoDB
very effectively.

>> I think Nikunj's proposal definitely is worthy of being persued, just like
>> the working group is persuing dozens of other proposals like XHR, CORS,
>> Selectors API, Workers, Server-Sent Events, Web Sockets, etc. I don't
>> believe it really fits into the Web Storage spec (if anything, I think we
>> should split Web Storage into two further specs, not add a third wholly
>> independent feature to it). However, I would definitely support an FPWD
>> publication of Nikunj's proposal, as I have for other proposals.
>
> That is encouraging. I will be glad to edit an FPWD that includes B-tree,
> interception, and programmable cache, if the WG so prefers.

What I don't understand is, why does interception (I assume you mean
HTTP interception?) and programmable cache, belong in the same spec as
a B-tree?

/ Jonas

Re: Points of order on this WG

2009-06-25 Thread Maciej Stachowiak



On Jun 25, 2009, at 12:42 PM, Nikunj R. Mehta wrote:



I think Nikunj's proposal definitely is worthy of being persued,  
just like
the working group is persuing dozens of other proposals like XHR,  
CORS,

Selectors API, Workers, Server-Sent Events, Web Sockets, etc. I don't
believe it really fits into the Web Storage spec (if anything, I  
think we
should split Web Storage into two further specs, not add a third  
wholly
independent feature to it). However, I would definitely support an  
FPWD

publication of Nikunj's proposal, as I have for other proposals.


That is encouraging. I will be glad to edit an FPWD that includes B- 
tree, interception, and programmable cache, if the WG so prefers.




It seems to me that Berkley DB style database storage, and request  
interception / programmable cache are orthogonal ideas and should  
arguably be separate drafts. I would assume request interception and  
programmable cache are usable regardless of what client-side storage  
APIs are available, much as HTML5 Application Cache is independent of  
these APIs. If anything, it seems more closely related to AppCache  
than to any proposed storage solution.


Regards,
Maciej

Re: Points of order on this WG

2009-06-25 Thread Maciej Stachowiak



On Jun 24, 2009, at 11:35 PM, Ian Hickson wrote:



In any case, adding a new feature to a spec whose future is uncertain
isn't a good idea because it means that the new feature's progress  
is tied
to the uncertain future of the rest of the spec. Thus, my  
recommendation

to Nikunj would be to create a new WG deliverable, not one tied to the
fate of the SQL Database section.


[...]

I think Nikunj's proposal definitely is worthy of being persued,  
just like
the working group is persuing dozens of other proposals like XHR,  
CORS,

Selectors API, Workers, Server-Sent Events, Web Sockets, etc. I don't
believe it really fits into the Web Storage spec (if anything, I  
think we
should split Web Storage into two further specs, not add a third  
wholly
independent feature to it). However, I would definitely support an  
FPWD

publication of Nikunj's proposal, as I have for other proposals.


I strongly agree on these points. I would prefer to see SQL Storage  
split out of the rest of Web Storage. We seem to have rough consensus  
and strong multilateral implementor interest on LocalStorage and  
SessionStorage, and they should be allowed to move forward on the  
standards track quickly. SQL Storage remains contentious, and only  
Apple and Google have shown strong implementor interest so far. And it  
has no technical tie to the other storage drafts. I also think  
Nikunj's proposal should be yet a third separate orthogonal draft.


Regards,
Maciej

Re: Points of order on this WG


On Jun 25, 2009, at 9:34 AM, Arthur Barstow wrote:


Nikunj, All,

Charles will respond separately regarding a way forward but I want  
to respond to the false accusation below.


On Jun 24, 2009, at 8:13 PM, ext Nikunj R. Mehta wrote:


The WG chair went ahead with the publication of the Web Storage draft
overriding serious objections about it's direction and emphasis.


The record actually shows Nikunj saying:

[[
http://lists.w3.org/Archives/Public/public-webapps/2009AprJun/ 
0145.html


Oracle conditionally supports the publishing this draft as FPWD
provided that the abstract is worded appropriately.

...

Here's what Oracle would like to see in the abstract:

This specification defines two APIs for persistent data storage in Web
clients: one for accessing key-value pair data and another for
accessing structured data.
]]

Ian agreed [1] to make the requested change above (it is included in  
the FPWD [2]) and thus addressed the only concern you raised re  
publishing the FPWD.


Seeing the way things were, there was no way to stop the publication  
[1]. To mitigate the negative effects of publication, Oracle made its  
assent conditional. In reality, the chairs should have taken in to  
account the prior reluctance to continue with the draft [2] and asked  
the author to seek requirements and provide cautionary text in  
prominent places in the FPWD.


[1] http://lists.w3.org/Archives/Public/public-webapps/2009AprJun/0143.html
[2] http://lists.w3.org/Archives/Public/public-webapps/2009AprJun/0106.html

Re: Points of order on this WG


On Jun 24, 2009, at 11:35 PM, Ian Hickson wrote:


On Thu, 25 Jun 2009, Doug Schepers wrote:

On Jun 23, 2009, at 5:10 PM, Ian Hickson wrote:
The Web Storage specification is someone dead-locked right now due  
to the

lack of consensus on whether to use SQL or not.


I don't buy this argument for an instant, and I'd be very surprised  
if

anyone in the WebApps WG did.  This specification was published as
specified because it matched the behavior (more or less) of an
implementation (WebKit), and it's disingenuous to pretend that that
doesn't set a precedent for the future development of the  
specification.


Let's be frank: there is good reason to specify and standardize
something that exists in a browser, because there is implementation
experience, and opportunity for widespread adoption, which is often
good; this is especially so with an implementation in a widespread
open-source engine like WebKit, because it can be reused.  I don't  
find

fault with that premise.

But just because it's been implemented doesn't mean it's the  
correct or

best (or even a good) solution.  There is less justification for
insisting on a single solution when it's only been implemented in one
browser engine, and only just recently.  There's little evidence that
this will work well for other implementers, nor that this is the
solution that works best for content developers.

I cannot take seriously a claim that a spec can't be changed due to a
"lack of consensus" when the claimant has openly and repeatedly
indicated a disinterest in consensus. So, the only conclusion I can  
draw

is that the spec is currently in a holding pattern to allow the
currently specified solution to gain defacto weight through real- 
world
content, and possibly garner premature implementations before it is  
even
in LC, thus making it all but impossible to change.  As Kyle Weems  
put

it: Deny, Delay, Too Late.


I think there may have been some misunderstanding about what I said  
above

(possibly because of my typo; s/someone/somewhat/).

When I say that the Web Storage spec is deadlocked, I mean that as it
stands, it isn't acceptable, since it doesn't represent what at  
least one
major vendor (Mozilla) wants, but that there haven't been any  
alternative
proposals that have gained widespread approval either, and so I  
don't know
how to move the spec forward. (I've been working with Mozilla off- 
line to

try to resolve this, but do not yet have a solution.)


I have proposed to Mozilla a solution that provides access to an  
organized key-value database such as that provided in the (open  
source) Berkeley DB. In essence, a database is a simple B-tree - it  
keeps keys sorted and permits duplicate keys. It is able to find a key  
or a key prefix, which enables efficient searching through a very  
large number of items. If we are ambitious (i.e., need more  
functionality), we can add indexes and joins to this spec. There is  
unlikely to be an interoperability nightmare, such as that which is  
the most likely outcome with SQL, it does not mandate the use of any  
query language, and there is at least 40 years of experience with it,  
including in highly resource-constrained environments. (There are 200  
million copies of Berkeley DB in deployment [1]).




There is no attempt on my part to force anything through de-facto
implementations; it is in fact the lack of vendors willing to  
implement
what is in the spec that is keeping it in a holding pattern. There  
is no
claim that Web Storage can't be changed; indeed, I claim that it  
_must_ be

changed, it's just that I'm not sure what it should be changed _to_.

In any case, adding a new feature to a spec whose future is uncertain
isn't a good idea because it means that the new feature's progress  
is tied
to the uncertain future of the rest of the spec. Thus, my  
recommendation

to Nikunj would be to create a new WG deliverable, not one tied to the
fate of the SQL Database section.



Nikunj has asked that his proposal be given equal weight and
consideration. While I'm not sure that's possible even now, because  
of
the existing implementation, I personally think it is reasonable to  
give

him a platform to demonstrate the relative merits of his alternate
proposal.


I think Nikunj's proposal definitely is worthy of being persued,  
just like
the working group is persuing dozens of other proposals like XHR,  
CORS,

Selectors API, Workers, Server-Sent Events, Web Sockets, etc. I don't
believe it really fits into the Web Storage spec (if anything, I  
think we
should split Web Storage into two further specs, not add a third  
wholly
independent feature to it). However, I would definitely support an  
FPWD

publication of Nikunj's proposal, as I have for other proposals.


That is encouraging. I will be glad to edit an FPWD that includes B- 
tree, interception, and programmable cache, if the WG so prefers.


[1] 
http://www.oracle.com/technology/products/berkeley-db/pdf/berkeley-db-famil

Re: Points of order on this WG


On Jun 24, 2009, at 10:24 PM, Doug Schepers wrote:


Hi, Nikunj-

I think Mike was overly blunt, but essentially correct in his  
response, but I'd like to add a specific comment inline...


Nikunj R. Mehta wrote (on 6/24/09 8:13 PM):


On Jun 23, 2009, at 5:10 PM, Ian Hickson wrote:
The Web Storage specification is someone dead-locked right now due  
to the

lack of consensus on whether to use SQL or not.




As Kyle Weems put it: Deny, Delay, Too Late.



I would endorse you, Nikunj, to edit the Web Storage spec to include  
your proposal.  However, I will also say that the burden of proving  
that your solution is better lies on you.  I'm not going to pretend  
this is not an uphill battle.  If you or someone on the Oracle team  
wants to demonstrate an implementation of your proposal, or even  
better, contribute that code to the WebKit or Mozilla codebase, that  
would be a compelling way of demonstrating relative merits...  
cutting-edge authors could experiment with both and provide feedback  
about what aspects of each they prefer.  That would be an effective  
argument in favor of one or the other.


You bet.



I will say that Hixie's proposal (which, if I understand it, comes  
from Apple's implementation) has an advantage, because he has been  
working within W3C and directly with browser vendors for a while; he  
knows how to write specifications in the style that implementers  
prefer, and he also has their respect on technical matters.  You  
would do well to specify your proposal in a manner similar to his,  
with similar detail, and to cultivate credibility and relationships  
with browser vendors if you want to gain preference for your  
proposal.  I'm sorry this is not the most encouraging statement, but  
I believe it is factual, and it is intended as helpful advice.




No worries.

Re: Points of order on this WG

On Jun 24, 2009, at 7:34 PM, Michael(tm) Smith wrote:

"Nikunj R. Mehta" , 2009-06-24 17:13 -0700:

I want to raise two formal points of order about the manner in
which this WG

has operated, particularly in respect to Web Storage.

1. Charter
2. Process

Firstly, no one seriously responds to proposals about things that are
officially in the WG's charter.

That's not true.

If you believe that's been the case with a specific proposal, then
let's please talk about that specific proposal instead of turning
this into a process discussion.

Please look to the bottom of my original email to see why this became
a process discussion. If my interpretation is incorrect, please
explain in the context of the specific details there instead of simply
saying it is not true.

If there is inadequate interest, then we should get rid of the
responsibility from this WG's charter.

If there's inadequate interest in *a particular proposal* from
other members of the group -- particularly among the vendors who
would be expected to implement it -- then that would be a pretty
good indicator that an investment of the already-constrained
resources of the group into trying harder to move that the
proposal forward might not be an investment that's likely to pay
off for us well as a group (in terms of actually being successful
at getting it implemented in UAs).

1. Oracle provided use cases and requirements [1] around storage in
Web browsers.

2. Oracle submitted a proposal first in October [2].
3. The WG's charter was changed and deliverables added without an open
process [3].
4. Oracle revised its proposal in April [4] based on limited feedback
on public-webapps. Oracle has also implemented this proposal in a
browser plug-in for Safari and Firefox before submitting this proposal.
5. This WG provided two very brief sets of questions. I responded to
both sets of questions without any delay, and assumed that the
questions were answered. Of these one was simply asking me to go to
HTML5 without attempting to address any of the requirements identified
in [2] or the technical details of the revised proposal [4].
6. Then I asked for permission to add to the Web Storage draft, and
was told that my proposal did not belong there, without any explanation.

On the other hand, there were no documented requirements for Web
Storage (and most likely for other Web* FPWDs) until I asked for it.
Even then all I get is one requirement "at least be as useful as
SQL" [4a].

There has been reluctance to support for the current Web Storage
draft's SQL mission in popular browsers. It is evident from [5], [6],
and [7] and still we are consuming this WG's precious time for that
proposal.

The policy of this group is to interpret silence as assent, isn't it.
By that token, browser vendor members of this WG have have supported
Oracle's current proposal because no one has said that implementing
this spec is not in their and/or the Web's interest.

On the other hand, if Web Storage and related matters are in
this WG's charter based on this WG's agreement, there should be
feedback from its members,

As far as I can see, that's already happening.

Not true in the case of Oracle's proposals.

and at least substantive discussions by its appointed editors.

First off, Ian is not an appointed editor for the Web Storage
draft. He's the editor of that particular draft by virtue of the
fact that he's the one wrote it. But the fact that he wrote it
and contributed it to the group does not magically bless it nor
necessarily give it any position of special entitlement in the
group. If you or any other member wants to contribute a related or
alternative draft and check it into the group's document
repository, you are very much encouraged to do so. We can then
continue with discussion about it -- with a status of Editor's
Draft in the group -- up to the point where we decide if/when we
decide as a group that we want to transition it to a First Public
Working Draft.

Let's see how this process works in practice. W3C is setting up a CVS
account for me as we speak.

Nikunj
http://o-micron.blogspot.com

[1] http://lists.w3.org/Archives/Public/public-webapps/2008OctDec/0104.html
[2] http://lists.w3.org/Archives/Public/public-webapps/2008OctDec/0130.html
[3] http://lists.w3.org/Archives/Public/public-webapps/2009AprJun/0251.html
[4] http://lists.w3.org/Archives/Public/public-webapps/2009AprJun/0341.html
[4a] http://lists.w3.org/Archives/Public/public-webapps/2009AprJun/0303.html
[5]
http://groups.google.com/group/mozilla.community.web-standards/msg/d6a92db27bd52bcb
[6]
http://groups.google.com/group/mozilla.community.web-standards/browse_frm/thread/da7000dcc486c0fb

Re: Points of order on this WG