[issue3783] dbm.sqlite proof of concept

Change by Erlend E. Aasland : -- nosy: +erlendaasland ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://

[issue3783] dbm.sqlite proof of concept

Change by Josiah Carlson : -- nosy: -josiahcarlson ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://ma

[issue3783] dbm.sqlite proof of concept

Gerhard Häring added the comment: This wiki page is out of date. It appears that SQlite is now threadsafe by default: http://www.sqlite.org/threadsafe.html -- ___ Python tracker

[issue3783] dbm.sqlite proof of concept

Changes by Florent Xicluna : -- versions: +Python 3.4 -Python 3.3 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue3783] dbm.sqlite proof of concept

Changes by Ray.Allen : -- versions: +Python 3.3 -Python 3.2 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http

[issue3783] dbm.sqlite proof of concept

Changes by Éric Araujo : -- nosy: +merwok ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org

[issue3783] dbm.sqlite proof of concept

Changes by Skip Montanaro : -- nosy: -skip.montanaro ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mai

[issue3783] dbm.sqlite proof of concept

Changes by Gregory P. Smith : -- nosy: -gregory.p.smith ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://

[issue3783] dbm.sqlite proof of concept

Changes by Antoine Pitrou : -- type: behavior -> feature request ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue3783] dbm.sqlite proof of concept

Changes by Florent Xicluna : -- nosy: +flox ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.o

[issue3783] dbm.sqlite proof of concept

Changes by Jesús Cea Avión : ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/pytho

[issue3783] dbm.sqlite proof of concept

Runar Tenfjord added the comment: Multi threading: According to http://www.sqlite.org/cvstrac/wiki?p=MultiThreading we need to keep a connection for each thread to support multi threaded access to the database. I came across this when deploying an application in a multi threaded environment.

[issue3783] dbm.sqlite proof of concept

Antoine Pitrou added the comment: It would be nice to try to advance this at PyCon, or at another time. -- ___ Python tracker ___ ___

[issue3783] dbm.sqlite proof of concept

Changes by Antoine Pitrou : -- stage: -> needs patch versions: +Python 3.2 -Python 3.1 ___ Python tracker ___ ___ Python-bugs-list mai

[issue3783] dbm.sqlite proof of concept

Runar Tenfjord added the comment: By utilizing triggers on inserts and deletes it is possible to keep track of the size and speed up __len__ by 10 x. SQL: CREATE TABLE IF NOT EXISTS info (key TEXT UNIQUE NOT NULL, value INTEGER NOT NULL); INSERT OR IGNORE INTO info (key,value) VALUES (

[issue3783] dbm.sqlite proof of concept

Matthias Klose added the comment: is there any chance for inclusion in 3.1? -- nosy: +doko ___ Python tracker ___ ___ Python-bugs-list

[issue3783] dbm.sqlite proof of concept

Raymond Hettinger added the comment: Unassigning. The code works but no one seems to be pushing for or caring about inclusion in Py3.1. If commits are delayed, then you might as well adopt the dbdict.py approach instead (reading the file in once at the beginning, operating directly on a dict

[issue3783] dbm.sqlite proof of concept

Changes by Jean-Paul Calderone : -- nosy: -exarkun ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.

[issue3783] dbm.sqlite proof of concept

Raymond Hettinger added the comment: FWIW, I put an alternative in the sandbox /dbm_sqlite/alt/dbdict.py and am attaching a copy here. The idea is to emulate gdbm's fast mode and delay all writes until closing. That lets us subclass from dict and get high-speed lookups, sets, and deletions. F

[issue3783] dbm.sqlite proof of concept

Changes by Raymond Hettinger : Removed file: http://bugs.python.org/file12931/dbsqlite.py ___ Python tracker ___ ___ Python-bugs-list mailing l

[issue3783] dbm.sqlite proof of concept

Raymond Hettinger added the comment: That's a bummer. Changing this method to __bool__ and then setting __len__ back to "count(*)". Added file: http://bugs.python.org/file12933/dbsqlite.py ___ Python tracker _

[issue3783] dbm.sqlite proof of concept

Antoine Pitrou added the comment: I think issuing 'SELECT MAX(ROWID)' to compute the length of the table is not correct if some rows get deleted in the table. I've found a thread about it here: http://osdir.com/ml/db.sqlite.general/2004-03/msg00329.html In that thread someone suggested caching t

[issue3783] dbm.sqlite proof of concept

Raymond Hettinger added the comment: Here's an updated patch (it's also in the sandbox): * Added a sync() method to support shelves. * Removed commits on granular sets and gets. * Optimized __len__ and __contains__. Added file: http://bugs.python.org/file12931/dbsqlite.py

[issue3783] dbm.sqlite proof of concept

Changes by Raymond Hettinger : Removed file: http://bugs.python.org/file12896/dbsqlite.py ___ Python tracker ___ ___ Python-bugs-list mailing l

[issue3783] dbm.sqlite proof of concept

Changes by Raymond Hettinger : Removed file: http://bugs.python.org/file12891/tmp_dev_shelver.py ___ Python tracker ___ ___ Python-bugs-list ma

[issue3783] dbm.sqlite proof of concept

Changes by Raymond Hettinger : Added file: http://bugs.python.org/file12896/dbsqlite.py ___ Python tracker ___ ___ Python-bugs-list mailing lis

[issue3783] dbm.sqlite proof of concept

Changes by Raymond Hettinger : -- assignee: -> rhettinger ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http:

[issue3783] dbm.sqlite proof of concept

Skip Montanaro added the comment: Unassigning myself. I don't have time for this. I've taken my sandbox version about as far as I can, and the subject of this ticket has gone a bit far afield from just adding a sqlite module to the dbm pkg. -- assignee: skip.montanaro -> ___

[issue3783] dbm.sqlite proof of concept

Raymond Hettinger added the comment: Thanks for looking at this. I'll do an update in the next few days. ___ Python tracker ___ ___ Python-bug

[issue3783] dbm.sqlite proof of concept

Jean-Paul Calderone added the comment: Some comments on tmp_dev_shelver.py... Regarding SQLhash.__init__, it would be better to avoid relying on the "sqlite_master" table by using the CREATE TABLE IF NOT EXISTS form of table creation. Setting the isolation_level in __init__ will have essential

[issue3783] dbm.sqlite proof of concept

Raymond Hettinger added the comment: Am also working on a patch for this and would like to coordinate. My first draft is attached: Added file: http://bugs.python.org/file12891/tmp_dev_shelver.py ___ Python tracker ___

[issue3783] dbm.sqlite proof of concept

Skip Montanaro added the comment: Hopefully I'm not picking at a scab here. I updated the dbm.sqlite module in the sandbox. It now orders by rowid instead of by key. (I saw no performance penalty for the small table sizes I was using to ordering. I switched from ordering by key to ordering by

[issue3783] dbm.sqlite proof of concept

Josiah Carlson <[EMAIL PROTECTED]> added the comment: Thank you for the report (fixed in the newly attached version) :) . Added file: http://bugs.python.org/file11602/sq_dict.py ___ Python tracker <[EMAIL PROTECTED]> ___

[issue3783] dbm.sqlite proof of concept

Changes by Josiah Carlson <[EMAIL PROTECTED]>: Removed file: http://bugs.python.org/file11472/sq_dict.py ___ Python tracker <[EMAIL PROTECTED]> ___

[issue3783] dbm.sqlite proof of concept

Erno Kuusela <[EMAIL PROTECTED]> added the comment: I'm looking for a bsddb-shelve replacement (because of we bsddb corruption problems), and decided to give this a try. Don't overlook the free locking you get from sqlite when evaluating this for inclusion! A small bug: >>> from sq_dict import

[issue3783] dbm.sqlite proof of concept

Skip Montanaro <[EMAIL PROTECTED]> added the comment: Josiah> I know that no one hear likes my particular implementation Josiah> (though it offers more or less the full interface)... I think implementing as much of the bsddb interface as you can is fine. I agree people used first, next, las

[issue3783] dbm.sqlite proof of concept

Josiah Carlson <[EMAIL PROTECTED]> added the comment: Here's a version with views from Python 3.0 for keys, values, and items :) . I know that no one hear likes my particular implementation (though it offers more or less the full interface), but the Keys, Values, and Items classes in the foll

[issue3783] dbm.sqlite proof of concept

Changes by Josiah Carlson <[EMAIL PROTECTED]>: Removed file: http://bugs.python.org/file11470/sq_dict.py ___ Python tracker <[EMAIL PROTECTED]> ___

[issue3783] dbm.sqlite proof of concept

Changes by Josiah Carlson <[EMAIL PROTECTED]>: Removed file: http://bugs.python.org/file11467/sq_dict.py ___ Python tracker <[EMAIL PROTECTED]> ___

[issue3783] dbm.sqlite proof of concept

Josiah Carlson <[EMAIL PROTECTED]> added the comment: > If you like, but "ordering semantics" is something which is just as > easily done in Python, so I don't understand the point of integrating > it in the dbm layer... Actually, the db layer is *exactly* where it should be implemented, especi

[issue3783] dbm.sqlite proof of concept

Skip Montanaro <[EMAIL PROTECTED]> added the comment: >> Well, sure, but heaven only knows what an application programmer will >> do... Antoine> If the docs clearly explain that there is no guarantee, we Antoine> don't need heaven. I would find it strange to potentially ruin Ant

[issue3783] dbm.sqlite proof of concept

Antoine Pitrou <[EMAIL PROTECTED]> added the comment: > Benchmarks to prove or disprove performance changes? Agreed, benchmarks should be run. > Subclasses to > offer different order by semantics (see the version I uploaded for an > example)? If you like, but "ordering semantics" is something

[issue3783] dbm.sqlite proof of concept

Josiah Carlson <[EMAIL PROTECTED]> added the comment: > I would find it strange to potentially ruin performance just for a > guarantee which has no useful purpose. Benchmarks to prove or disprove performance changes? Subclasses to offer different order by semantics (see the version I uploaded

[issue3783] dbm.sqlite proof of concept

Antoine Pitrou <[EMAIL PROTECTED]> added the comment: > Well, sure, but heaven only knows what an application programmer will do... If the docs clearly explain that there is no guarantee, we don't need heaven. I would find it strange to potentially ruin performance just for a guarantee which has

[issue3783] dbm.sqlite proof of concept

Skip Montanaro <[EMAIL PROTECTED]> added the comment: Antoine> I might add that calling keys() then values() is suboptimal, Antoine> because it will issue two SQL queries while calling items() Antoine> will issue just one. Well, sure, but heaven only knows what an application programmer

[issue3783] dbm.sqlite proof of concept

Skip Montanaro <[EMAIL PROTECTED]> added the comment: >> As long as SQLite guarantees that the ordering is identical, then >> sure, dump the ORDER BY clause. Gerhard> It doesn't guarantee it, but the implementation behaves like Gerhard> this. If the behavior isn't guaranteed, I thin

[issue3783] dbm.sqlite proof of concept

Changes by Josiah Carlson <[EMAIL PROTECTED]>: Removed file: http://bugs.python.org/file11412/sq_dict.py ___ Python tracker <[EMAIL PROTECTED]> ___

[issue3783] dbm.sqlite proof of concept

Josiah Carlson <[EMAIL PROTECTED]> added the comment: > I like Skip's version better, because it's closer to the dbm > "specification" instead of trying to mimic bsddb (first, last, etc.). > I'd like to keep such things out. dbm.sqlite is meant as a potential replacement of dbm.bsddb. Since pe

[issue3783] dbm.sqlite proof of concept

Antoine Pitrou <[EMAIL PROTECTED]> added the comment: I might add that calling keys() then values() is suboptimal, because it will issue two SQL queries while calling items() will issue just one. ___ Python tracker <[EMAIL PROTECTED]>

[issue3783] dbm.sqlite proof of concept

Antoine Pitrou <[EMAIL PROTECTED]> added the comment: Le jeudi 11 septembre 2008 à 13:48 +, Skip Montanaro a écrit : > Actually, I think Python guarantees (for dicts at least - other mappings > should probably follow suit) that if you call keys() then call values() > without making any change

[issue3783] dbm.sqlite proof of concept

Gerhard Häring <[EMAIL PROTECTED]> added the comment: Skip Montanaro wrote: > Skip Montanaro <[EMAIL PROTECTED]> added the comment: > > Gerhard> FWIW that will also work without the ORDER BY, because you're > Gerhard> getting the rows back in the same ORDER. Something cheaper > Gerhard>

[issue3783] dbm.sqlite proof of concept

Skip Montanaro <[EMAIL PROTECTED]> added the comment: Gerhard> FWIW that will also work without the ORDER BY, because you're Gerhard> getting the rows back in the same ORDER. Something cheaper Gerhard> would also be "ORDER BY ROWID". I still propose to just do Gerhard> without the ORD

[issue3783] dbm.sqlite proof of concept

Skip Montanaro <[EMAIL PROTECTED]> added the comment: >> I'd like to guarantee that zip(db.keys(), db.values() == db.items(). Antoine> It doesn't sound very useful, and it may hurt performance on Antoine> big tables. Actually, I think Python guarantees (for dicts at least - other mappin

[issue3783] dbm.sqlite proof of concept

Gerhard Häring <[EMAIL PROTECTED]> added the comment: > I'd like to guarantee that zip(db.keys(), db.values() == db.items(). Ok. If that isn't guaranteed elsewhere just drop it here? FWIW that will also work without the ORDER BY, because you're getting the rows back in the same ORDER. Something

[issue3783] dbm.sqlite proof of concept

Antoine Pitrou <[EMAIL PROTECTED]> added the comment: Le jeudi 11 septembre 2008 à 12:46 +, Skip Montanaro a écrit : > Gerhard> What's all this ORDER BY in both your implementations about? > Gerhard> The dbm "spec" says nothing about keys being ordered AFAIC. Can > Gerhard> we get rid

[issue3783] dbm.sqlite proof of concept

Skip Montanaro <[EMAIL PROTECTED]> added the comment: Gerhard> What's all this ORDER BY in both your implementations about? Gerhard> The dbm "spec" says nothing about keys being ordered AFAIC. Can Gerhard> we get rid of these? I'd like to guarantee that zip(db.keys(), db.values() == db.i

[issue3783] dbm.sqlite proof of concept

Gerhard Häring <[EMAIL PROTECTED]> added the comment: One question about Josiah's _check_value(). SQLite is less crippled than other simplistic databases and also supports integers, reals and blobs in addition to strings. Shouldn't we make this accessible to users? Or is compatibility with other

[issue3783] dbm.sqlite proof of concept

Gerhard Häring <[EMAIL PROTECTED]> added the comment: I like Skip's version better, because it's closer to the dbm "specification" instead of trying to mimic bsddb (first, last, etc.). I'd like to keep such things out. I've made a few changes to the sandbox project which I will check in later to

[issue3783] dbm.sqlite proof of concept

Josiah Carlson <[EMAIL PROTECTED]> added the comment: I tried passing the db name as a parameter with '?', it doesn't always work. Also, there shouldn't be any SQL injection issues here unless someone designed their system wrong (if a third party is allowed to pass the name of a db table into

[issue3783] dbm.sqlite proof of concept

Gregory P. Smith <[EMAIL PROTECTED]> added the comment: sq_dict review: have sqlite quote/escape self._mtn before using it with a python %s substitution. or pass it into the sql query function as a positional ? parameter like you do for keys and values. (avoid sql injection) raise a TypeError

[issue3783] dbm.sqlite proof of concept

Josiah Carlson <[EMAIL PROTECTED]> added the comment: Here's an alternate version with most of bsddb's interface intact. -- nosy: +josiahcarlson Added file: http://bugs.python.org/file11412/sq_dict.py ___ Python tracker <[EMAIL PROTECTED]>

[issue3783] dbm.sqlite proof of concept

Raymond Hettinger <[EMAIL PROTECTED]> added the comment: I would like to see something like this go into 3.0 so that shelves don't become useless for Windows users. -- nosy: +rhettinger ___ Python tracker <[EMAIL PROTECTED]>

[issue3783] dbm.sqlite proof of concept

Changes by Gregory P. Smith <[EMAIL PROTECTED]>: -- nosy: +gregory.p.smith ___ Python tracker <[EMAIL PROTECTED]> ___ ___ Python-bugs-li

[issue3783] dbm.sqlite proof of concept

Changes by Skip Montanaro <[EMAIL PROTECTED]>: Removed file: http://bugs.python.org/file11388/test_dbm_sqlite.py ___ Python tracker <[EMAIL PROTECTED]> ___

[issue3783] dbm.sqlite proof of concept

Changes by Skip Montanaro <[EMAIL PROTECTED]>: Removed file: http://bugs.python.org/file11387/sqlite.py ___ Python tracker <[EMAIL PROTECTED]> ___ _

[issue3783] dbm.sqlite proof of concept

Changes by Skip Montanaro <[EMAIL PROTECTED]>: Removed file: http://bugs.python.org/file11386/dbm.diff ___ Python tracker <[EMAIL PROTECTED]> ___ __

[issue3783] dbm.sqlite proof of concept

Skip Montanaro <[EMAIL PROTECTED]> added the comment: OK, I made a sandbox project out of it: svn+ssh://[EMAIL PROTECTED]/sandbox/trunk/dbm_sqlite Hack away! -- assignee: -> skip.montanaro ___ Python tracker <[EMAIL PROTECTED]>

[issue3783] dbm.sqlite proof of concept

Changes by Gregory Burd <[EMAIL PROTECTED]>: -- nosy: +gregburd ___ Python tracker <[EMAIL PROTECTED]> ___ ___ Python-bugs-list mailing

[issue3783] dbm.sqlite proof of concept

Changes by Jesús Cea Avión <[EMAIL PROTECTED]>: -- nosy: +jcea ___ Python tracker <[EMAIL PROTECTED]> ___ ___ Python-bugs-list mailing l

[issue3783] dbm.sqlite proof of concept

Skip Montanaro <[EMAIL PROTECTED]> added the comment: Antoine> It would be more efficient to base keys() on iterkeys() than the Antoine> reverse, IMO. True. I was just modifying the dumbdbm implementation. Antoine> Other than that, why not open a branch or at least upload Antoine>

[issue3783] dbm.sqlite proof of concept

Antoine Pitrou <[EMAIL PROTECTED]> added the comment: It would be more efficient to base keys() on iterkeys() than the reverse, IMO. Other than that, why not open a branch or at least upload full-fledged patch files? :) -- nosy: +pitrou ___ Python tra

[issue3783] dbm.sqlite proof of concept

Changes by Skip Montanaro <[EMAIL PROTECTED]>: Removed file: http://bugs.python.org/file11385/sqlite.py ___ Python tracker <[EMAIL PROTECTED]> ___ _

[issue3783] dbm.sqlite proof of concept

Changes by Skip Montanaro <[EMAIL PROTECTED]>: Removed file: http://bugs.python.org/file11384/test_dbm_sqlite.py ___ Python tracker <[EMAIL PROTECTED]> ___

[issue3783] dbm.sqlite proof of concept

Skip Montanaro <[EMAIL PROTECTED]> added the comment: Updated test cases Added file: http://bugs.python.org/file11388/test_dbm_sqlite.py ___ Python tracker <[EMAIL PROTECTED]> ___

[issue3783] dbm.sqlite proof of concept

Skip Montanaro <[EMAIL PROTECTED]> added the comment: Another tweak - add values() Added file: http://bugs.python.org/file11387/sqlite.py ___ Python tracker <[EMAIL PROTECTED]> ___ ___

[issue3783] dbm.sqlite proof of concept

Skip Montanaro <[EMAIL PROTECTED]> added the comment: Trivial doc diffs against 3.0b3 doc. Added file: http://bugs.python.org/file11386/dbm.diff ___ Python tracker <[EMAIL PROTECTED]> ___

[issue3783] dbm.sqlite proof of concept

Changes by Skip Montanaro <[EMAIL PROTECTED]>: Removed file: http://bugs.python.org/file11383/sqlite.py ___ Python tracker <[EMAIL PROTECTED]> ___ _

[issue3783] dbm.sqlite proof of concept

Skip Montanaro <[EMAIL PROTECTED]> added the comment: Another slight revision to the module. Added file: http://bugs.python.org/file11385/sqlite.py ___ Python tracker <[EMAIL PROTECTED]> _

[issue3783] dbm.sqlite proof of concept

Skip Montanaro <[EMAIL PROTECTED]> added the comment: Attaching test cases based on dumbdbm tests. Added file: http://bugs.python.org/file11384/test_dbm_sqlite.py ___ Python tracker <[EMAIL PROTECTED]> __

[issue3783] dbm.sqlite proof of concept

Changes by Skip Montanaro <[EMAIL PROTECTED]>: Removed file: http://bugs.python.org/file11380/sqlite.py ___ Python tracker <[EMAIL PROTECTED]> ___ _

[issue3783] dbm.sqlite proof of concept

Skip Montanaro <[EMAIL PROTECTED]> added the comment: Attaching corrected module. Added file: http://bugs.python.org/file11383/sqlite.py ___ Python tracker <[EMAIL PROTECTED]> ___

[issue3783] dbm.sqlite proof of concept

New submission from Skip Montanaro <[EMAIL PROTECTED]>: Based on recent discussions about ridding Python of bsddb I decided to see how hard it would be to implement a barebones dbm.sqlite module. Turns out, not very hard. No docs. No test cases. Caveat emptor. But I think it can serve as at l