Re: [sqlite] LevelDB benchmark

On 07/30/2011 12:19 PM, Dan Kennedy wrote: > On 07/30/2011 04:45 AM, Gabor Cselle wrote: >> Just a heads-up that we just posted an update to the benchmarks: >> - No more superfluous index on the primary key >> - WAL turned on with auto-checkpointing every 4096 pages >> >>

Re: [sqlite] LevelDB benchmark

...@sqlite.org] on behalf of Alexey Pechnikov [pechni...@mobigroup.ru] Sent: Saturday, July 30, 2011 2:18 AM To: sqlite_us...@googlegroups.com; General Discussion of SQLite Database Subject: EXT :Re: [sqlite] LevelDB benchmark 2011/7/30 Gabor Cselle <ga...@google.com>: > Just a heads-up that we ju

Re: [sqlite] LevelDB benchmark

2011/7/30 Gabor Cselle : > Just a heads-up that we just posted an update to the benchmarks: > - No more superfluous index on the primary key > - WAL turned on with auto-checkpointing every 4096 pages You may use "PRAGMA synchronous = NORMAL" instead of "PRAGMA synchronous =

Re: [sqlite] LevelDB benchmark

On 07/30/2011 04:45 AM, Gabor Cselle wrote: > Just a heads-up that we just posted an update to the benchmarks: > - No more superfluous index on the primary key > - WAL turned on with auto-checkpointing every 4096 pages > > http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html > > A diff of

Re: [sqlite] LevelDB benchmark

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/29/2011 02:45 PM, Gabor Cselle wrote: > (My understanding is that SQLite stores 64-bit, not 32-bit integers). SQLite actually stores integers using a variable length encoding and is documented here:

Re: [sqlite] LevelDB benchmark

Just a heads-up that we just posted an update to the benchmarks: - No more superfluous index on the primary key - WAL turned on with auto-checkpointing every 4096 pages http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html A diff of the results is here:

Re: [sqlite] LevelDB benchmark

Dear Dr. Hipp, I'm a Product Manager here at Google and one of the authors of the benchmarks discussed on this thread. Our intention with the benchmarks was to compare LevelDB with popular storage engines for the specific use case for which it was built, mapping string keys to string values.

Re: [sqlite] LevelDB benchmark

2011/7/29 Black, Michael (IS) : > 2X-20X is hardly "small...overhead" in my world. > > Even 2X is the difference between 30 days and 15 days.  One 16-computer blade > rack vs two racks ($200,000 vs $400,000). > > That's why google did this.  Works for what they need and is

Re: [sqlite] LevelDB benchmark

At 14:19 29/07/2011, Black, Michael (IS) wrote: >What they don't say explicitly is that if all you need is key/value >capability then an SQL database is overkill and only slows you down >(bit of a duh factor there though not obvious to >neophytes). Generally speaking that's one thing they

Re: [sqlite] LevelDB benchmark

bject: EXT :Re: [sqlite] LevelDB benchmark 2011/7/29 Black, Michael (IS) <michael.bla...@ngc.com>: > What they don't say explicitly is that if all you need is key/value > capability then an SQL database is overkill and only slows you down (bit of a > duh factor there though

Re: [sqlite] LevelDB benchmark

On Fri, Jul 29, 2011 at 12:30 AM, Alexey Pechnikov wrote: > With integer->blob mapping patch I get these results: > > Good information, Alexey. Thanks for running this. It would seem that the SQLite team has some serious work ahead of us. > > $ ./db_bench_sqlite3 >

Re: [sqlite] LevelDB benchmark

2011/7/29 Black, Michael (IS) : > What they don't say explicitly is that if all you need is key/value > capability then an SQL database is overkill and only slows you down (bit of a > duh factor there though not obvious to neophytes). The overhead by SQL layer is small.

Re: [sqlite] LevelDB benchmark

From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on behalf of Eduardo Morras [nec...@retena.com] Sent: Friday, July 29, 2011 2:50 AM To: General Discussion of SQLite Database Subject: EXT :Re: [sqlite] LevelDB benchmark At 02:53 29/07/2011, Richard Hipp wrote: >On

Re: [sqlite] LevelDB benchmark

At 02:53 29/07/2011, Richard Hipp wrote: >On Thu, Jul 28, 2011 at 12:27 AM, wrote: > >I suspect that I will come up with other suggestions once I have a chance to >dig a little deeper into the benchmarks. If you have suggestions, please >publish them here. They are

Re: [sqlite] LevelDB benchmark

With integer->blob mapping patch I get these results: $ ./db_bench_sqlite3 SQLite: version 3.7.7.1 Date: Fri Jul 29 05:32:05 2011 CPU:2 * Intel(R) Atom(TM) CPU N450 @ 1.66GHz CPUCache: 512 KB Keys: 16 bytes each Values: 100 bytes each Entries:100 RawSize:

Re: [sqlite] LevelDB benchmark

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/28/2011 07:39 PM, Simon Slavin wrote: > Actually I don't see how BLOBs can be used in an index anyway, since > technically blobs have no ordering. memcmp provides an ordering just as it can for strings without a collation. (That is what

Re: [sqlite] LevelDB benchmark

On 29 Jul 2011, at 3:34am, Roger Binns wrote: > On 07/28/2011 06:57 PM, Simon Slavin wrote: >> Would it improve the SQLite time if it was changed to strings instead of >> BLOBs ? > > Note that internally SQLite treats strings and blobs virtually identically. > Usually the same data structure

Re: [sqlite] LevelDB benchmark

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/28/2011 06:57 PM, Simon Slavin wrote: > Would it improve the SQLite time if it was changed to strings instead of > BLOBs ? Note that internally SQLite treats strings and blobs virtually identically. Usually the same data structure and

Re: [sqlite] LevelDB benchmark

On Fri, Jul 29, 2011 at 8:53 AM, Richard Hipp wrote: > On Thu, Jul 28, 2011 at 12:27 AM, wrote: > > > they used > > > > CREATE TABLE test (key blob, value blob, PRIMARY KEY(key)) > > CREATE INDEX keyindex ON test (key) > > > > Notice the inefficiencies

Re: [sqlite] LevelDB benchmark

There are the LevelDB sources and tests svn checkout http://leveldb.googlecode.com/svn/trunk/ leveldb-read-only Build SQLite test as make db_bench_sqlite3 And LevelDB test as make db_bench My patch for leveldb-read-only/doc/bench/db_bench_sqlite3.cc to disable redudant index and enable WAL is

Re: [sqlite] LevelDB benchmark

On Thu, Jul 28, 2011 at 12:27 AM, wrote: > they used > > CREATE TABLE test (key blob, value blob, PRIMARY KEY(key)) > CREATE INDEX keyindex ON test (key) > Notice the inefficiencies inherent in this schema. (1) A primary key on a BLOB? Really? (2) They create an

Re: [sqlite] LevelDB benchmark

Hm, I test I find index on PK field: CREATE TABLE test (key blob, value blob, PRIMARY KEY(key)) CREATE INDEX keyindex ON test (key) Epic fail, I think :D Default test on Intel(R) Atom(TM) CPU N450 @ 1.66GHz fillseq : 442.937 micros/op;0.2 MB/s fillseqsync :1678.168

Re: [sqlite] LevelDB benchmark

LevelDB use append log but SQLite is tested without WAL :) I check and some tests 2.5x faster with WAL. -- Best regards, Alexey Pechnikov. http://pechnikov.tel/ ___ sqlite-users mailing list sqlite-users@sqlite.org

Re: [sqlite] LevelDB benchmark

they used CREATE TABLE test (key blob, value blob, PRIMARY KEY(key)) CREATE INDEX keyindex ON test (key) on random replaces it doubles the write operations. J Decker schrieb: > On Wed, Jul 27, 2011 at 6:22 PM, Stephan Wehner > wrote: >> There are some benchmark's

Re: [sqlite] LevelDB benchmark

On Wed, Jul 27, 2011 at 7:00 PM, Simon Slavin wrote: > > On 28 Jul 2011, at 2:53am, Stephan Wehner wrote: > >> On Wed, Jul 27, 2011 at 6:44 PM, Simon Slavin wrote: >>> >>> On 28 Jul 2011, at 2:22am, Stephan Wehner wrote: >>> There are some

Re: [sqlite] LevelDB benchmark

On Wed, Jul 27, 2011 at 9:22 PM, Stephan Wehner wrote: > There are some benchmark's at > http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html > > I don't have anything to point to, but I thought sqlite3 does better > than stated there. > > In particular, 26,900

Re: [sqlite] LevelDB benchmark

On 28 Jul 2011, at 3:01am, David Garfield wrote: > They used REPLACE. See > http://code.google.com/p/leveldb/source/browse/trunk/doc/bench/db_bench_sqlite3.cc#492 > > They used explicit transactions, and tested with both single REPLACE > transactions and 1000 REPLACE transactions. Section 1A

Re: [sqlite] LevelDB benchmark

They used REPLACE. See http://code.google.com/p/leveldb/source/browse/trunk/doc/bench/db_bench_sqlite3.cc#492 They used explicit transactions, and tested with both single REPLACE transactions and 1000 REPLACE transactions. Section 1A would be the single REPLACE transactions, while 2B is the

Re: [sqlite] LevelDB benchmark

On 28 Jul 2011, at 2:53am, Stephan Wehner wrote: > On Wed, Jul 27, 2011 at 6:44 PM, Simon Slavin wrote: >> >> On 28 Jul 2011, at 2:22am, Stephan Wehner wrote: >> >>> There are some benchmark's at >>> http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html >>> >>> I

Re: [sqlite] LevelDB benchmark

On Wed, Jul 27, 2011 at 6:44 PM, Simon Slavin wrote: > > On 28 Jul 2011, at 2:22am, Stephan Wehner wrote: > >> There are some benchmark's at >> http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html >> >> I don't have anything to point to, but I thought sqlite3 does

Re: [sqlite] LevelDB benchmark

On Wed, Jul 27, 2011 at 6:22 PM, Stephan Wehner wrote: > There are some benchmark's at > http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html > > I don't have anything to point to, but I thought sqlite3 does better > than stated there. > > In particular, 26,900

Re: [sqlite] LevelDB benchmark

On 28 Jul 2011, at 2:22am, Stephan Wehner wrote: > There are some benchmark's at > http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html > > I don't have anything to point to, but I thought sqlite3 does better > than stated there. i looked through their source code, trying to see if

[sqlite] LevelDB benchmark

There are some benchmark's at http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html I don't have anything to point to, but I thought sqlite3 does better than stated there. In particular, 26,900 sequential writes per second and 420 random writes per second from section "1. Baseline