On 7/20/2010 4:47 PM, N. Zavaritsky wrote:
> Hi Brett,
>
> First of all let me express appreciation of your great work!
>
>
Thank you for your kind encouragement, mejedi!
> It sounds sort of fantastik (wow, Firefox! everyone use Firefox these
> days!). Unfortunately Sedna was never designed to be an embedable
> database.
Yeah. One other manifestation of this is how the data directory is not
configurable. Since I wanted to include Sedna along with the extension
(its small size making it ideal for inclusion with an extension,
avoiding a need for remote downloads after initial installation), if I
kept it inside the extension, the large databases you speak of would
have been included inside the extensions folder, meaning the user would
lose their data each time I updated the extension! My solution to this
was to copy all of Sedna into the Firefox profile folder and then use
that copy. This is less than ideal, but it did work.
> Sedna launches and manages it's own processes (one master +
> one process per db + one process per query session). Though I'd prefer
> for a database that Firefox is using internally to be nicely wrapped
> inside Firefox itself (no external processes, just a dll/so/dylib loaded
> into Firefox process, database code sharing threads with the browser) a
> bunch of external processes is just a minor inconvenience. However there
> is more to come.
>
Yes, that sounds like it would be nice. If you ever get it built in this
way, I can see about getting our extension's XPCOM code revised to deal
with a DLL/so/dylib file instead. That would be nice if it could be
possible. There might also be a bigger (if small?) chance of seeing
Firefox itself add this as a component for use by extensions if note
websites too.
> Embedable database must be lean. For the standalone database it is ok to
> consume 90% of avalible memory. Anyway the entire computer may be
> dedicated to the task of running that particular database. Obviously
> embedable database is not the main task running on the computer, but
> just the oposite!
>
>
I see... Sure...
> Sedna memory usage is configurable but if given less than 100Mb RAM
> Sedna slows down considerably.
On the plus side, I have at least made the code asynchronous, so while
it may be eating up RAM and get slowed down if not allowed enough RAM,
at least the browser shouldn't come to a complete stand-still while it's
waiting. :)
> Of course the amount of RAM neccessary to
> achieve adequite performance depends on the complexity of the queries
> used. It is quite reasonable to expect that queries executed against the
> XML database embeded inside Firefox won't be complex ones so probably we
> could use less memory.
>
I don't know how quickly complexity translates into RAM, but it sure
would be nice to be able to effectively allow complex queries, as one
advantage of being able to run XQuery statements in the browser is that
a server doesn't have to take the risk of getting bogged down (as far as
DoS or bandwidth costs) by such requests (and the user doesn't have to
wait for the usually biggest bottleneck of waiting for network delivery
of the results).
> Finally there are issues with the data file. Well, just one ISSUE. The
> data file size. It starts at 100Mb and grows in 100Mb increments. These
> numbers are configurable as well. But don't expect to tune it down to
> 1Mb or so :) The data representation used internally was not designed
> to be space-efficient. Basically Sedna manages storage in 64Kb blocks.
> By design distinct named XML elements must live in distinct blocks (it
> makes queries like doc("aaa")/foo/bar really fast). So even not so
> complex XML document can easily consume hundreds of blocks. There are
> certain techniques that helps to reduce the number of blocks used ([a]
> redesigning data to utilize fewer distinct named elements or [b] storing
> documents with similar structure together) but I doubt that end users
> will be willing to learn and apply these techniques.
>
And one of the big pluses of an XML database as I see it and understand
it is its ability to store hierarchical, non-predictably structured
documents...
> And lastly the concept of "distinctly named XML elements living in
> distinct blocks" was an oversimplification. It is not just the element
> name that counts but the complete path from the document root. For
> instance book/section/title and book/section/figure/title occupy
> distinct blocks. You may see Sedna is not great at storing XML markup
> (count the number of say<a>-s in average HTML page; how many<a>-s
> share the same path from the document root?).
>
>
I see...
> Over are the bad news.
>
> Now comes some good ones.
>
> Sedna is continually developed. The idea to refine data storage hovered
> for a long time. I guess we can address the abovementioned issues in
> some time.
>
That would indeed be quite wonderful...
> And I must say the opportunity to get Sedna into Firefox is pretty
> exciting! Achieving the really WIDE adoption of XML database is so much
> important for the XML database community.
>
Yes, that is my hope too. Thank you for doing all of the hard work!
Also, thank you very much for the very informative coverage of the
issues. Although I have been hopeful of producing something useful,
maybe an even larger intention was to demonstrate the potential or
"proof-of-concept" of using native XML databases in the browser and
hopefully get people to consider what a logical fit an XML database is
in the browser (where as I think I mentioned, anything from XHTML tables
to unpredictably hierarchical XML can be stored).
And feel free to try it out, as the main sample I refer to is not just
an example, but actually lets you store any number of documents and
perform XQueries on those documents (with syntax-coloring thanks to
EditArea and the XQuery plugin provided by DQ). I'd like to make another
sample file which allows iterating over the current databases and
viewing the original XML.
By the way, since Sedna's size is the main reason I chose it (not
knowing enough technically to compare) being that it is hard enough to
get people to install an extension (though restartless extensions are
also becoming possible), can I ask how or why (in laymen's terms) your
implementation is so much smaller, by even several orders of magnitude
from the other XML databases out there? Although it would be nice to see
XQUF and Full-text, etc. added too, it still seems quite impressive that
you can cover XQuery (and XUpdate) with such a small size!
best wishes
Brett
> wbr,
> mejedi
>
>
>
> On Mon, 2010-07-19 at 22:36 +0800, Brett Zamir wrote:
>
>> Hello all!
>>
>> I've just added an add-on for Firefox which uses Sedna; I'm calling it
>> "XDIB" or "XML Database In a Browser". See
>> https://addons.mozilla.org/en-US/firefox/addon/199900/ .
>>
>> Basically, this lets any web developer, who is given permission by the
>> user, to add XML content to Sedna (databases stored in Firefox's Profile
>> folder by default, so that the data will not be lost whenever the
>> extension may get updated) and then query it, update it, etc..
>>
>> Actually, we've basically wrapped every command there is in Sedna
>> (though removing "Sedna" from the methods in case this could become
>> abstracted--sorry, no XQJ at the moment), though I haven't really tested
>> anything besides loading XML and doing XQueries against data in a
>> collection.
>>
>> At the moment, the databases are accessible to any site which requests
>> permission, as I wanted this to be the default behavior, since I think
>> it should be up to users how they want their data shared, and not have
>> their own data be locked in by a particular site, even if that site
>> originated the storage of the data. The access is granted depending on
>> whether read permissions, insert permissions, etc., are desired, and I
>> really need to add the ability to specify which database one has
>> permission to access, since for now, it is any one! (though it does ask
>> permission at least).
>>
>> I also hope to make the API available as special protocol links, so a
>> link could trigger the view of certain content, and if my other add-on,
>> Open URIs (at https://addons.mozilla.org/en-US/firefox/addon/162154/ )
>> is used together with this, one could link to locally stored by default,
>> but fallback to an online site in case the user is visiting the site
>> without this add-on installed. I like the idea of bookmarks that work
>> offline, and which could do fancy queries, and trigger a download of the
>> data (and subsequent auto-updates) if the data had not yet been
>> downloaded. But that's not implemented at the moment...
>>
>> I paid a friend to implement the C++ code which as part of an XPCOM
>> component for Mozilla's Cross-Platform Component Object Model, among
>> other things, lets JavaScript communicate with C++ so that people like
>> me who do not know C++ can still write Firefox extensions or the like
>> which interact with cool tools or libraries like Sedna, so my apologies
>> if anything has been lost in translation as far as how the API is to be
>> used (though I know it is working for XQueries at least--though I do
>> need help on figuring out how Sedna can support UTF-8 if it uses "const
>> char" as I understand that to be ASCII only; the component may need to
>> be updated to support multi-byte strings).
>>
>> And since my friend is a Windows guy, we've only compiled so far in
>> Windows, but since this is supposed to be cross-platform, hopefully we
>> can get this working on other systems as well.
>>
>> As I mention on the add-on site, the API is of course very much new (not
>> to mention there may well be problems in our code), so while we are
>> whole-heartedly encouraging experimentation, please do not use this for
>> critical content, nor depend on the API remaining frozen.
>>
>> I have the off-hope that we could get XML databases to be part of HTML5;
>> e.g., see the debate at
>> http://hacks.mozilla.org/2010/06/beyond-html5-database-apis-and-the-road-to-indexeddb/
>> or
>> http://hacks.mozilla.org/2010/06/comparing-indexeddb-and-webdatabase/comment-page-1/#comment-95595
>> . My energy is fairly limited as far as what I can contribute, but I do
>> hope to make piecemeal progress on this, and fully welcome anyone
>> interested to offer their feedback, improvements, or whatever... :)
>> (I've cc'd myself if you wish to get in touch off-list to indicate you
>> wish to be kept informed.)
>>
>> best wishes,
>> Brett
>>
>>
>> ------------------------------------------------------------------------------
>> This SF.net email is sponsored by Sprint
>> What will you do first with EVO, the first 4G phone?
>> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
>> _______________________________________________
>> Sedna-discussion mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/sedna-discussion
>>
>
>
------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Sedna-discussion mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/sedna-discussion
