Re: Please report your indexing speed
My first tests for Centos were against erlang R12B, I'm seeing yet
another speed improvement using R14B04 on Centos 6.
{"couchdb":"Welcome","version":"1.2.0"}
real4m45.824s
user0m0.003s
sys 0m0.003s
Huge improvement over my current production scenario R12B, Spidermonkey
1.7, Couchdb 1.0.2, index rebuilding takes 13m16.480s
Great job on the speed improvments!
Wendall
On 03/06/2012 01:19 PM, Wendall Cada wrote:
Here is another set of results using a db with ~300k docs and ~10
views in the design doc.
Fedora Core2Duo 2GHz, 7200RPM disk:
{"couchdb":"Welcome","version":"1.0.3"} (spidermonkey 1.8.5) (stock
package)
real9m9.086s
user0m0.007s
sys 0m0.016s
{"couchdb":"Welcome","version":"1.1.1"} (spidermonkey 1.8.5) (git-1.1.1)
real7m59.437s
user0m0.007s
sys 0m0.016s
{"couchdb":"Welcome","version":"1.2.0"} (spidermonkey 1.8.5)
(git-db8335bf2a)
real6m25.469s
user0m0.009s
sys 0m0.027s
Centos Xeon 2.27GHz (2 cores) 15k SCSI/RAID 10:
{"couchdb":"Welcome","version":"1.0.1"} (spidermonkey 1.7.0) (stock
package)
real13m16.480s
user0m0.001s
sys 0m0.004s
{"couchdb":"Welcome","version":"1.1.1"} (spidermonkey 1.8.5) (git-1.1.1)
real6m14.999s
user0m0.001s
sys 0m0.003s
{"couchdb":"Welcome","version":"1.2.0"} (spidermonkey 1.8.5)
(git-fb72251bc7)
real5m24.925s
user0m0.002s
sys 0m0.004s
While there are certainly speed improvements over 1.1.1, the biggest
change is moving to a newer spidermonkey.
Wendall
Re: Please report your indexing speed
Here is another set of results using a db with ~300k docs and ~10 views
in the design doc.
Fedora Core2Duo 2GHz, 7200RPM disk:
{"couchdb":"Welcome","version":"1.0.3"} (spidermonkey 1.8.5) (stock
package)
real9m9.086s
user0m0.007s
sys 0m0.016s
{"couchdb":"Welcome","version":"1.1.1"} (spidermonkey 1.8.5) (git-1.1.1)
real7m59.437s
user0m0.007s
sys 0m0.016s
{"couchdb":"Welcome","version":"1.2.0"} (spidermonkey 1.8.5)
(git-db8335bf2a)
real6m25.469s
user0m0.009s
sys 0m0.027s
Centos Xeon 2.27GHz (2 cores) 15k SCSI/RAID 10:
{"couchdb":"Welcome","version":"1.0.1"} (spidermonkey 1.7.0) (stock package)
real13m16.480s
user0m0.001s
sys 0m0.004s
{"couchdb":"Welcome","version":"1.1.1"} (spidermonkey 1.8.5) (git-1.1.1)
real6m14.999s
user0m0.001s
sys 0m0.003s
{"couchdb":"Welcome","version":"1.2.0"} (spidermonkey 1.8.5)
(git-fb72251bc7)
real5m24.925s
user0m0.002s
sys 0m0.004s
While there are certainly speed improvements over 1.1.1, the biggest
change is moving to a newer spidermonkey.
Wendall
Re: Please report your indexing speed
Awesome Filipe, so these two were related, I didn't get that subtlety in your
original post. This is great, thanks for the patch -- Bob
On Mar 5, 2012, at 2:41 AM, Filipe David Manana wrote:
> On Sun, Mar 4, 2012 at 9:45 AM, Bob Dionne
> wrote:
>> yes, I was surprised by the 30% claim as my numbers showed it only getting
>> back to where we were with 1.1.x
>>
>> I think Bob's suggestion to get to the root code change that caused this
>> regression is important as it will help us assess all the other cases this
>> testing hasn't even touched yet
>
> The explanation I gave in the 1.2.0 second round vote identifies the
> reason, which is that the updater is (depending on timings) collecting
> smaller batches of map results, which makes the btree updates less
> efficient (besides higher number of btree updates). The patch
> addresses this by queuing a batch of map results instead of queuing
> map results one by one. Jan's tests and mine are evidence that this is
> valid in practice and not just theory.
>
> The original main goal of COUCHDB-1186 was to make the indexing of
> views that emit reasonably large (or complex in structure) map values
> more efficient.
> Here's an example using Jason's slow_couchdb script with wow.tpl and
> map function of "function(doc) {emit([doc.type, doc.category],
> doc);}":
>
> 1.1.x:
>
> fdmanana 07:04:12 ~/git/hub/slow_couchdb (master)> docs=20
> batch=5000 ./bench.sh wow.tpl
> Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03)
> {"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
>
> [INFO] Created DB named `db1'
> [INFO] Uploaded 5000 documents via _bulk_docs
> ()
> [INFO] Uploaded 5000 documents via _bulk_docs
> Building view.
> {"total_rows":20,"offset":0,"rows":[
> {"id":"00144af5-9f07-448e-af88-026674e3e3d0","key":["dwarf","assassin"],"value":{"_id":"00144af5-9f07-448e-af88-026674e3e3d0","_rev":"1-785fbf5e641f3d10fa083501ad82a9fe","data3":"Vl6BftQEWY6imvNs0FasOj2CrPCptP70z5d","ratio":1.6,"integers":[48028,3170,54066,95547,70643,23763,25804,33180,89061,35274,48244,91792,37936,11855],"category":"assassin","nested":{"dict":{"3XGVdTTF":31490,"SDxKa54e":40,"XIzUloRH":7,"5Mj9F4bp":192,"1sXfjgYf":1203,"XP5YSqhX":25461,"QJr0Xhxn":9941},"string1":"3Q4tvmhHwKvedKiRnoL6xUz","string2":"dWI1mrrAypRh","values":[33712,57371,88567,88361,70873,6327,17326,91004,41840,86257],"string3":"i7OGysnXvynz41VMQJ","coords":[{"x":65350.46,"y":103881.18},{"x":24180.14,"y":8474.9},{"x":88326.66,"y":43151.76},{"x":120199.77,"y":102270.29},{"x":191924.18,"y":74479.75}]},"level":21,"type":"dwarf","data1":"Vpkplo80LshlcjBE0ySJNNpfgDy2bu8byWrmZ44B","data2":"GnyNbos75Wxm1C5MLdOeXNniHamBjld70vHqoJnEtnlfekuPXJ"}}
> ]}
>
> real 2m49.227s
> user 0m0.006s
> sys 0m0.005s
>
>
> 1.2.x:
>
> fdmanana 07:13:30 ~/git/hub/slow_couchdb (master)> docs=20
> batch=5000 ./bench.sh wow.tpl
> Server: CouchDB/1.2.0 (Erlang OTP/R14B03)
> {"couchdb":"Welcome","version":"1.2.0"}
>
> [INFO] Created DB named `db1'
> [INFO] Uploaded 5000 documents via _bulk_docs
> ()
> [INFO] Uploaded 5000 documents via _bulk_docs
> Building view.
> {"total_rows":20,"offset":0,"rows":[
> {"id":"0005cd07-49f4-4a99-b506-acef948f2acc","key":["dwarf","assassin"],"value":{"_id":"0005cd07-49f4-4a99-b506-acef948f2acc","_rev":"1-4b418e69618bf11124a03e1a3845f071","data3":"T0W2JBUET9yzRXHfUqcUBwFhYGKh14YFVxk","ratio":1.6999556,"integers":[25658,7573,47779,43217,49586,57992,13549,90984,45253,49560,1643,64085,38381,62544],"category":"assassin","nested":{"dict":{"oWhW4jJ6":199,"EPSVtKtS":5638,"8WpzvD5x":73714,"stD9Ynfh":8924,"0qh5Nc1g":5994,"pBa5vJyy":18,"s25oAkRc":165270},"string1":"fNNHb8lxtcy7GpwSU3yRyaC","string2":"rilbiZM7yAaK","values":[49632,93665,73258,75675,4229,15742,16965,76825,22049,79829],"string3":"IwX09SiOLMSSyxffMB","coords":[{"x":179620.451164,"y":11539.9899782},{"x":68483.8206985,"y":110559.199709},{"x":67197.9402328,"y":96702.2106403},{"x":25469.8698981,"y":79049.4905239},{"x":157059.899418,"y":34963.4103492}]},"level":6,"type":"dwarf","data1":"njpz38JSfz00p2Lc2Jv0dON7UfTljRgz0J2B7w7K","data2":"4hpsT2szDrssbUCHEirTzHOIhSxTd83i1FO5aNXgoGAfO2srH1"}}
> ]}
>
> real 1m51.989s
> user 0m0.006s
> sys 0m0.004s
>
>
> 1.2.x + patch:
>
> fdmanana 07:29:11 ~/git/hub/slow_couchdb (master)> docs=20
> batch=5000 ./bench.sh wow.tpl
> Server: CouchDB/1.2.0 (Erlang OTP/R14B03)
> {"couchdb":"Welcome","version":"1.2.0"}
>
> [INFO] Created DB named `db1'
> [INFO] Uploaded 5000 documents via _bulk_docs
> ()
> [INFO] Uploaded 5000 documents via _bulk_docs
> Building view.
> {"total_rows":20,"offset":0,"rows":[
> {"id":"0005cd07-49f4-4a99-b506-acef948f2acc","key":["dwarf","assassin"],"value":{"_id":"0005cd07-49f4-4a99-b506-acef948f2acc","_rev":"1-4b418e69618bf11124a03e1a3845f071","data3":"T0W2JBUET9yzRXHfUqcUBwFhYGKh14YFVxk","ratio":1.6999556,"integers":[25658,7573,47779,43217,49586,57992,13549,9
Re: Please report your indexing speed
go for it! Please mention "COUCHDB-1186" in the commit msg so we can trace it.
On 5 March 2012 10:17, Filipe David Manana wrote:
> On Mon, Mar 5, 2012 at 2:15 AM, Robert Newson wrote:
>> Filipe,
>>
>> Thanks for the explanation. I agree on the way forward and will apply
>> your patch to the relevant branches with attribution.
>
> Thanks. I'll do that myself soon unless someone in the meanwhile shows
> worse results with it.
>
>
>>
>> B.
>>
>> On 5 March 2012 07:41, Filipe David Manana wrote:
>>> On Sun, Mar 4, 2012 at 9:45 AM, Bob Dionne
>>> wrote:
yes, I was surprised by the 30% claim as my numbers showed it only getting
back to where we were with 1.1.x
I think Bob's suggestion to get to the root code change that caused this
regression is important as it will help us assess all the other cases this
testing hasn't even touched yet
>>>
>>> The explanation I gave in the 1.2.0 second round vote identifies the
>>> reason, which is that the updater is (depending on timings) collecting
>>> smaller batches of map results, which makes the btree updates less
>>> efficient (besides higher number of btree updates). The patch
>>> addresses this by queuing a batch of map results instead of queuing
>>> map results one by one. Jan's tests and mine are evidence that this is
>>> valid in practice and not just theory.
>>>
>>> The original main goal of COUCHDB-1186 was to make the indexing of
>>> views that emit reasonably large (or complex in structure) map values
>>> more efficient.
>>> Here's an example using Jason's slow_couchdb script with wow.tpl and
>>> map function of "function(doc) {emit([doc.type, doc.category],
>>> doc);}":
>>>
>>> 1.1.x:
>>>
>>> fdmanana 07:04:12 ~/git/hub/slow_couchdb (master)> docs=20
>>> batch=5000 ./bench.sh wow.tpl
>>> Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03)
>>> {"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
>>>
>>> [INFO] Created DB named `db1'
>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>> ()
>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>> Building view.
>>> {"total_rows":20,"offset":0,"rows":[
>>> {"id":"00144af5-9f07-448e-af88-026674e3e3d0","key":["dwarf","assassin"],"value":{"_id":"00144af5-9f07-448e-af88-026674e3e3d0","_rev":"1-785fbf5e641f3d10fa083501ad82a9fe","data3":"Vl6BftQEWY6imvNs0FasOj2CrPCptP70z5d","ratio":1.6,"integers":[48028,3170,54066,95547,70643,23763,25804,33180,89061,35274,48244,91792,37936,11855],"category":"assassin","nested":{"dict":{"3XGVdTTF":31490,"SDxKa54e":40,"XIzUloRH":7,"5Mj9F4bp":192,"1sXfjgYf":1203,"XP5YSqhX":25461,"QJr0Xhxn":9941},"string1":"3Q4tvmhHwKvedKiRnoL6xUz","string2":"dWI1mrrAypRh","values":[33712,57371,88567,88361,70873,6327,17326,91004,41840,86257],"string3":"i7OGysnXvynz41VMQJ","coords":[{"x":65350.46,"y":103881.18},{"x":24180.14,"y":8474.9},{"x":88326.66,"y":43151.76},{"x":120199.77,"y":102270.29},{"x":191924.18,"y":74479.75}]},"level":21,"type":"dwarf","data1":"Vpkplo80LshlcjBE0ySJNNpfgDy2bu8byWrmZ44B","data2":"GnyNbos75Wxm1C5MLdOeXNniHamBjld70vHqoJnEtnlfekuPXJ"}}
>>> ]}
>>>
>>> real 2m49.227s
>>> user 0m0.006s
>>> sys 0m0.005s
>>>
>>>
>>> 1.2.x:
>>>
>>> fdmanana 07:13:30 ~/git/hub/slow_couchdb (master)> docs=20
>>> batch=5000 ./bench.sh wow.tpl
>>> Server: CouchDB/1.2.0 (Erlang OTP/R14B03)
>>> {"couchdb":"Welcome","version":"1.2.0"}
>>>
>>> [INFO] Created DB named `db1'
>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>> ()
>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>> Building view.
>>> {"total_rows":20,"offset":0,"rows":[
>>> {"id":"0005cd07-49f4-4a99-b506-acef948f2acc","key":["dwarf","assassin"],"value":{"_id":"0005cd07-49f4-4a99-b506-acef948f2acc","_rev":"1-4b418e69618bf11124a03e1a3845f071","data3":"T0W2JBUET9yzRXHfUqcUBwFhYGKh14YFVxk","ratio":1.6999556,"integers":[25658,7573,47779,43217,49586,57992,13549,90984,45253,49560,1643,64085,38381,62544],"category":"assassin","nested":{"dict":{"oWhW4jJ6":199,"EPSVtKtS":5638,"8WpzvD5x":73714,"stD9Ynfh":8924,"0qh5Nc1g":5994,"pBa5vJyy":18,"s25oAkRc":165270},"string1":"fNNHb8lxtcy7GpwSU3yRyaC","string2":"rilbiZM7yAaK","values":[49632,93665,73258,75675,4229,15742,16965,76825,22049,79829],"string3":"IwX09SiOLMSSyxffMB","coords":[{"x":179620.451164,"y":11539.9899782},{"x":68483.8206985,"y":110559.199709},{"x":67197.9402328,"y":96702.2106403},{"x":25469.8698981,"y":79049.4905239},{"x":157059.899418,"y":34963.4103492}]},"level":6,"type":"dwarf","data1":"njpz38JSfz00p2Lc2Jv0dON7UfTljRgz0J2B7w7K","data2":"4hpsT2szDrssbUCHEirTzHOIhSxTd83i1FO5aNXgoGAfO2srH1"}}
>>> ]}
>>>
>>> real 1m51.989s
>>> user 0m0.006s
>>> sys 0m0.004s
>>>
>>>
>>> 1.2.x + patch:
>>>
>>> fdmanana 07:29:11 ~/git/hub/slow_couchdb (master)> docs=20
>>> batch=5000 ./bench.sh wow.tpl
>>> Server: CouchDB/1.2.0 (Erlang OTP/R14B03)
>>> {"couchdb":"Welcome","version":"1.2.0"}
>>>
>>> [INFO] Created DB named `db1'
>>> [INFO] Upload
Re: Please report your indexing speed
On Mon, Mar 5, 2012 at 2:15 AM, Robert Newson wrote:
> Filipe,
>
> Thanks for the explanation. I agree on the way forward and will apply
> your patch to the relevant branches with attribution.
Thanks. I'll do that myself soon unless someone in the meanwhile shows
worse results with it.
>
> B.
>
> On 5 March 2012 07:41, Filipe David Manana wrote:
>> On Sun, Mar 4, 2012 at 9:45 AM, Bob Dionne
>> wrote:
>>> yes, I was surprised by the 30% claim as my numbers showed it only getting
>>> back to where we were with 1.1.x
>>>
>>> I think Bob's suggestion to get to the root code change that caused this
>>> regression is important as it will help us assess all the other cases this
>>> testing hasn't even touched yet
>>
>> The explanation I gave in the 1.2.0 second round vote identifies the
>> reason, which is that the updater is (depending on timings) collecting
>> smaller batches of map results, which makes the btree updates less
>> efficient (besides higher number of btree updates). The patch
>> addresses this by queuing a batch of map results instead of queuing
>> map results one by one. Jan's tests and mine are evidence that this is
>> valid in practice and not just theory.
>>
>> The original main goal of COUCHDB-1186 was to make the indexing of
>> views that emit reasonably large (or complex in structure) map values
>> more efficient.
>> Here's an example using Jason's slow_couchdb script with wow.tpl and
>> map function of "function(doc) {emit([doc.type, doc.category],
>> doc);}":
>>
>> 1.1.x:
>>
>> fdmanana 07:04:12 ~/git/hub/slow_couchdb (master)> docs=20
>> batch=5000 ./bench.sh wow.tpl
>> Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03)
>> {"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
>>
>> [INFO] Created DB named `db1'
>> [INFO] Uploaded 5000 documents via _bulk_docs
>> ()
>> [INFO] Uploaded 5000 documents via _bulk_docs
>> Building view.
>> {"total_rows":20,"offset":0,"rows":[
>> {"id":"00144af5-9f07-448e-af88-026674e3e3d0","key":["dwarf","assassin"],"value":{"_id":"00144af5-9f07-448e-af88-026674e3e3d0","_rev":"1-785fbf5e641f3d10fa083501ad82a9fe","data3":"Vl6BftQEWY6imvNs0FasOj2CrPCptP70z5d","ratio":1.6,"integers":[48028,3170,54066,95547,70643,23763,25804,33180,89061,35274,48244,91792,37936,11855],"category":"assassin","nested":{"dict":{"3XGVdTTF":31490,"SDxKa54e":40,"XIzUloRH":7,"5Mj9F4bp":192,"1sXfjgYf":1203,"XP5YSqhX":25461,"QJr0Xhxn":9941},"string1":"3Q4tvmhHwKvedKiRnoL6xUz","string2":"dWI1mrrAypRh","values":[33712,57371,88567,88361,70873,6327,17326,91004,41840,86257],"string3":"i7OGysnXvynz41VMQJ","coords":[{"x":65350.46,"y":103881.18},{"x":24180.14,"y":8474.9},{"x":88326.66,"y":43151.76},{"x":120199.77,"y":102270.29},{"x":191924.18,"y":74479.75}]},"level":21,"type":"dwarf","data1":"Vpkplo80LshlcjBE0ySJNNpfgDy2bu8byWrmZ44B","data2":"GnyNbos75Wxm1C5MLdOeXNniHamBjld70vHqoJnEtnlfekuPXJ"}}
>> ]}
>>
>> real 2m49.227s
>> user 0m0.006s
>> sys 0m0.005s
>>
>>
>> 1.2.x:
>>
>> fdmanana 07:13:30 ~/git/hub/slow_couchdb (master)> docs=20
>> batch=5000 ./bench.sh wow.tpl
>> Server: CouchDB/1.2.0 (Erlang OTP/R14B03)
>> {"couchdb":"Welcome","version":"1.2.0"}
>>
>> [INFO] Created DB named `db1'
>> [INFO] Uploaded 5000 documents via _bulk_docs
>> ()
>> [INFO] Uploaded 5000 documents via _bulk_docs
>> Building view.
>> {"total_rows":20,"offset":0,"rows":[
>> {"id":"0005cd07-49f4-4a99-b506-acef948f2acc","key":["dwarf","assassin"],"value":{"_id":"0005cd07-49f4-4a99-b506-acef948f2acc","_rev":"1-4b418e69618bf11124a03e1a3845f071","data3":"T0W2JBUET9yzRXHfUqcUBwFhYGKh14YFVxk","ratio":1.6999556,"integers":[25658,7573,47779,43217,49586,57992,13549,90984,45253,49560,1643,64085,38381,62544],"category":"assassin","nested":{"dict":{"oWhW4jJ6":199,"EPSVtKtS":5638,"8WpzvD5x":73714,"stD9Ynfh":8924,"0qh5Nc1g":5994,"pBa5vJyy":18,"s25oAkRc":165270},"string1":"fNNHb8lxtcy7GpwSU3yRyaC","string2":"rilbiZM7yAaK","values":[49632,93665,73258,75675,4229,15742,16965,76825,22049,79829],"string3":"IwX09SiOLMSSyxffMB","coords":[{"x":179620.451164,"y":11539.9899782},{"x":68483.8206985,"y":110559.199709},{"x":67197.9402328,"y":96702.2106403},{"x":25469.8698981,"y":79049.4905239},{"x":157059.899418,"y":34963.4103492}]},"level":6,"type":"dwarf","data1":"njpz38JSfz00p2Lc2Jv0dON7UfTljRgz0J2B7w7K","data2":"4hpsT2szDrssbUCHEirTzHOIhSxTd83i1FO5aNXgoGAfO2srH1"}}
>> ]}
>>
>> real 1m51.989s
>> user 0m0.006s
>> sys 0m0.004s
>>
>>
>> 1.2.x + patch:
>>
>> fdmanana 07:29:11 ~/git/hub/slow_couchdb (master)> docs=20
>> batch=5000 ./bench.sh wow.tpl
>> Server: CouchDB/1.2.0 (Erlang OTP/R14B03)
>> {"couchdb":"Welcome","version":"1.2.0"}
>>
>> [INFO] Created DB named `db1'
>> [INFO] Uploaded 5000 documents via _bulk_docs
>> ()
>> [INFO] Uploaded 5000 documents via _bulk_docs
>> Building view.
>> {"total_rows":20,"offset":0,"rows":[
>> {"id":"0005cd07-49f4-4a99-b506-acef948f2acc","key":["dwarf","assas
Re: Please report your indexing speed
Filipe,
Thanks for the explanation. I agree on the way forward and will apply
your patch to the relevant branches with attribution.
B.
On 5 March 2012 07:41, Filipe David Manana wrote:
> On Sun, Mar 4, 2012 at 9:45 AM, Bob Dionne
> wrote:
>> yes, I was surprised by the 30% claim as my numbers showed it only getting
>> back to where we were with 1.1.x
>>
>> I think Bob's suggestion to get to the root code change that caused this
>> regression is important as it will help us assess all the other cases this
>> testing hasn't even touched yet
>
> The explanation I gave in the 1.2.0 second round vote identifies the
> reason, which is that the updater is (depending on timings) collecting
> smaller batches of map results, which makes the btree updates less
> efficient (besides higher number of btree updates). The patch
> addresses this by queuing a batch of map results instead of queuing
> map results one by one. Jan's tests and mine are evidence that this is
> valid in practice and not just theory.
>
> The original main goal of COUCHDB-1186 was to make the indexing of
> views that emit reasonably large (or complex in structure) map values
> more efficient.
> Here's an example using Jason's slow_couchdb script with wow.tpl and
> map function of "function(doc) {emit([doc.type, doc.category],
> doc);}":
>
> 1.1.x:
>
> fdmanana 07:04:12 ~/git/hub/slow_couchdb (master)> docs=20
> batch=5000 ./bench.sh wow.tpl
> Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03)
> {"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
>
> [INFO] Created DB named `db1'
> [INFO] Uploaded 5000 documents via _bulk_docs
> ()
> [INFO] Uploaded 5000 documents via _bulk_docs
> Building view.
> {"total_rows":20,"offset":0,"rows":[
> {"id":"00144af5-9f07-448e-af88-026674e3e3d0","key":["dwarf","assassin"],"value":{"_id":"00144af5-9f07-448e-af88-026674e3e3d0","_rev":"1-785fbf5e641f3d10fa083501ad82a9fe","data3":"Vl6BftQEWY6imvNs0FasOj2CrPCptP70z5d","ratio":1.6,"integers":[48028,3170,54066,95547,70643,23763,25804,33180,89061,35274,48244,91792,37936,11855],"category":"assassin","nested":{"dict":{"3XGVdTTF":31490,"SDxKa54e":40,"XIzUloRH":7,"5Mj9F4bp":192,"1sXfjgYf":1203,"XP5YSqhX":25461,"QJr0Xhxn":9941},"string1":"3Q4tvmhHwKvedKiRnoL6xUz","string2":"dWI1mrrAypRh","values":[33712,57371,88567,88361,70873,6327,17326,91004,41840,86257],"string3":"i7OGysnXvynz41VMQJ","coords":[{"x":65350.46,"y":103881.18},{"x":24180.14,"y":8474.9},{"x":88326.66,"y":43151.76},{"x":120199.77,"y":102270.29},{"x":191924.18,"y":74479.75}]},"level":21,"type":"dwarf","data1":"Vpkplo80LshlcjBE0ySJNNpfgDy2bu8byWrmZ44B","data2":"GnyNbos75Wxm1C5MLdOeXNniHamBjld70vHqoJnEtnlfekuPXJ"}}
> ]}
>
> real 2m49.227s
> user 0m0.006s
> sys 0m0.005s
>
>
> 1.2.x:
>
> fdmanana 07:13:30 ~/git/hub/slow_couchdb (master)> docs=20
> batch=5000 ./bench.sh wow.tpl
> Server: CouchDB/1.2.0 (Erlang OTP/R14B03)
> {"couchdb":"Welcome","version":"1.2.0"}
>
> [INFO] Created DB named `db1'
> [INFO] Uploaded 5000 documents via _bulk_docs
> ()
> [INFO] Uploaded 5000 documents via _bulk_docs
> Building view.
> {"total_rows":20,"offset":0,"rows":[
> {"id":"0005cd07-49f4-4a99-b506-acef948f2acc","key":["dwarf","assassin"],"value":{"_id":"0005cd07-49f4-4a99-b506-acef948f2acc","_rev":"1-4b418e69618bf11124a03e1a3845f071","data3":"T0W2JBUET9yzRXHfUqcUBwFhYGKh14YFVxk","ratio":1.6999556,"integers":[25658,7573,47779,43217,49586,57992,13549,90984,45253,49560,1643,64085,38381,62544],"category":"assassin","nested":{"dict":{"oWhW4jJ6":199,"EPSVtKtS":5638,"8WpzvD5x":73714,"stD9Ynfh":8924,"0qh5Nc1g":5994,"pBa5vJyy":18,"s25oAkRc":165270},"string1":"fNNHb8lxtcy7GpwSU3yRyaC","string2":"rilbiZM7yAaK","values":[49632,93665,73258,75675,4229,15742,16965,76825,22049,79829],"string3":"IwX09SiOLMSSyxffMB","coords":[{"x":179620.451164,"y":11539.9899782},{"x":68483.8206985,"y":110559.199709},{"x":67197.9402328,"y":96702.2106403},{"x":25469.8698981,"y":79049.4905239},{"x":157059.899418,"y":34963.4103492}]},"level":6,"type":"dwarf","data1":"njpz38JSfz00p2Lc2Jv0dON7UfTljRgz0J2B7w7K","data2":"4hpsT2szDrssbUCHEirTzHOIhSxTd83i1FO5aNXgoGAfO2srH1"}}
> ]}
>
> real 1m51.989s
> user 0m0.006s
> sys 0m0.004s
>
>
> 1.2.x + patch:
>
> fdmanana 07:29:11 ~/git/hub/slow_couchdb (master)> docs=20
> batch=5000 ./bench.sh wow.tpl
> Server: CouchDB/1.2.0 (Erlang OTP/R14B03)
> {"couchdb":"Welcome","version":"1.2.0"}
>
> [INFO] Created DB named `db1'
> [INFO] Uploaded 5000 documents via _bulk_docs
> ()
> [INFO] Uploaded 5000 documents via _bulk_docs
> Building view.
> {"total_rows":20,"offset":0,"rows":[
> {"id":"0005cd07-49f4-4a99-b506-acef948f2acc","key":["dwarf","assassin"],"value":{"_id":"0005cd07-49f4-4a99-b506-acef948f2acc","_rev":"1-4b418e69618bf11124a03e1a3845f071","data3":"T0W2JBUET9yzRXHfUqcUBwFhYGKh14YFVxk","ratio":1.6999556,"integers":[25658,7573,47779,43217,49586,57992,13549,90984,45253
Re: Please report your indexing speed
On Sun, Mar 4, 2012 at 9:45 AM, Bob Dionne wrote:
> yes, I was surprised by the 30% claim as my numbers showed it only getting
> back to where we were with 1.1.x
>
> I think Bob's suggestion to get to the root code change that caused this
> regression is important as it will help us assess all the other cases this
> testing hasn't even touched yet
The explanation I gave in the 1.2.0 second round vote identifies the
reason, which is that the updater is (depending on timings) collecting
smaller batches of map results, which makes the btree updates less
efficient (besides higher number of btree updates). The patch
addresses this by queuing a batch of map results instead of queuing
map results one by one. Jan's tests and mine are evidence that this is
valid in practice and not just theory.
The original main goal of COUCHDB-1186 was to make the indexing of
views that emit reasonably large (or complex in structure) map values
more efficient.
Here's an example using Jason's slow_couchdb script with wow.tpl and
map function of "function(doc) {emit([doc.type, doc.category],
doc);}":
1.1.x:
fdmanana 07:04:12 ~/git/hub/slow_couchdb (master)> docs=20
batch=5000 ./bench.sh wow.tpl
Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03)
{"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
[INFO] Created DB named `db1'
[INFO] Uploaded 5000 documents via _bulk_docs
()
[INFO] Uploaded 5000 documents via _bulk_docs
Building view.
{"total_rows":20,"offset":0,"rows":[
{"id":"00144af5-9f07-448e-af88-026674e3e3d0","key":["dwarf","assassin"],"value":{"_id":"00144af5-9f07-448e-af88-026674e3e3d0","_rev":"1-785fbf5e641f3d10fa083501ad82a9fe","data3":"Vl6BftQEWY6imvNs0FasOj2CrPCptP70z5d","ratio":1.6,"integers":[48028,3170,54066,95547,70643,23763,25804,33180,89061,35274,48244,91792,37936,11855],"category":"assassin","nested":{"dict":{"3XGVdTTF":31490,"SDxKa54e":40,"XIzUloRH":7,"5Mj9F4bp":192,"1sXfjgYf":1203,"XP5YSqhX":25461,"QJr0Xhxn":9941},"string1":"3Q4tvmhHwKvedKiRnoL6xUz","string2":"dWI1mrrAypRh","values":[33712,57371,88567,88361,70873,6327,17326,91004,41840,86257],"string3":"i7OGysnXvynz41VMQJ","coords":[{"x":65350.46,"y":103881.18},{"x":24180.14,"y":8474.9},{"x":88326.66,"y":43151.76},{"x":120199.77,"y":102270.29},{"x":191924.18,"y":74479.75}]},"level":21,"type":"dwarf","data1":"Vpkplo80LshlcjBE0ySJNNpfgDy2bu8byWrmZ44B","data2":"GnyNbos75Wxm1C5MLdOeXNniHamBjld70vHqoJnEtnlfekuPXJ"}}
]}
real2m49.227s
user0m0.006s
sys 0m0.005s
1.2.x:
fdmanana 07:13:30 ~/git/hub/slow_couchdb (master)> docs=20
batch=5000 ./bench.sh wow.tpl
Server: CouchDB/1.2.0 (Erlang OTP/R14B03)
{"couchdb":"Welcome","version":"1.2.0"}
[INFO] Created DB named `db1'
[INFO] Uploaded 5000 documents via _bulk_docs
()
[INFO] Uploaded 5000 documents via _bulk_docs
Building view.
{"total_rows":20,"offset":0,"rows":[
{"id":"0005cd07-49f4-4a99-b506-acef948f2acc","key":["dwarf","assassin"],"value":{"_id":"0005cd07-49f4-4a99-b506-acef948f2acc","_rev":"1-4b418e69618bf11124a03e1a3845f071","data3":"T0W2JBUET9yzRXHfUqcUBwFhYGKh14YFVxk","ratio":1.6999556,"integers":[25658,7573,47779,43217,49586,57992,13549,90984,45253,49560,1643,64085,38381,62544],"category":"assassin","nested":{"dict":{"oWhW4jJ6":199,"EPSVtKtS":5638,"8WpzvD5x":73714,"stD9Ynfh":8924,"0qh5Nc1g":5994,"pBa5vJyy":18,"s25oAkRc":165270},"string1":"fNNHb8lxtcy7GpwSU3yRyaC","string2":"rilbiZM7yAaK","values":[49632,93665,73258,75675,4229,15742,16965,76825,22049,79829],"string3":"IwX09SiOLMSSyxffMB","coords":[{"x":179620.451164,"y":11539.9899782},{"x":68483.8206985,"y":110559.199709},{"x":67197.9402328,"y":96702.2106403},{"x":25469.8698981,"y":79049.4905239},{"x":157059.899418,"y":34963.4103492}]},"level":6,"type":"dwarf","data1":"njpz38JSfz00p2Lc2Jv0dON7UfTljRgz0J2B7w7K","data2":"4hpsT2szDrssbUCHEirTzHOIhSxTd83i1FO5aNXgoGAfO2srH1"}}
]}
real1m51.989s
user0m0.006s
sys 0m0.004s
1.2.x + patch:
fdmanana 07:29:11 ~/git/hub/slow_couchdb (master)> docs=20
batch=5000 ./bench.sh wow.tpl
Server: CouchDB/1.2.0 (Erlang OTP/R14B03)
{"couchdb":"Welcome","version":"1.2.0"}
[INFO] Created DB named `db1'
[INFO] Uploaded 5000 documents via _bulk_docs
()
[INFO] Uploaded 5000 documents via _bulk_docs
Building view.
{"total_rows":20,"offset":0,"rows":[
{"id":"0005cd07-49f4-4a99-b506-acef948f2acc","key":["dwarf","assassin"],"value":{"_id":"0005cd07-49f4-4a99-b506-acef948f2acc","_rev":"1-4b418e69618bf11124a03e1a3845f071","data3":"T0W2JBUET9yzRXHfUqcUBwFhYGKh14YFVxk","ratio":1.6999556,"integers":[25658,7573,47779,43217,49586,57992,13549,90984,45253,49560,1643,64085,38381,62544],"category":"assassin","nested":{"dict":{"oWhW4jJ6":199,"EPSVtKtS":5638,"8WpzvD5x":73714,"stD9Ynfh":8924,"0qh5Nc1g":5994,"pBa5vJyy":18,"s25oAkRc":165270},"string1":"fNNHb8lxtcy7GpwSU3yRyaC","string2":"rilbiZM7yAaK","values":[49632,93665,73258,75675,4229,15742,16965,76825,22049,79829],"string3":"IwX
Re: Please report your indexing speed
On 3 March 2012 00:56, Dave Cottlehuber wrote: > # localhost, mac with vmware fusion internal network to windows 7 vm > Disk: /dev/disk0 > Device / Media Name: APPLE SSD TS128C Media > Solid State: Yes > dave@akai slow_couchdb % ./bench.sh small_doc.tpl With real disk and a low iMac. Some anomalies and differing builds muddy the results: http://friendpaste.com/6npWxkdhjqd0p3rHKBa3Kx Have found Filipe's patch so will redo these tests tomorrow permitting. A+ Dave
Re: Please report your indexing speed
On Mar 4, 2012, at 18:48 , Jan Lehnardt wrote: > > On Mar 4, 2012, at 18:45 , Bob Dionne wrote: > >> yes, I was surprised by the 30% claim as my numbers showed it only getting >> back to where we were with 1.1.x > > I see ~10% faster than where 1.1.1 was for small docs and ~30% for large docs. I got curious whether there was a drop-off (or up, I guess) point for the performance improvement between doc size and %-improved. I re-ran the tests for 300, 500 & 700 byte docs respectively and the numbers (updated in the spreadsheet) suggest a ~5% improvement on each iteration. So, looks linear, no sweet spot. Moving on. Cheers Jan -- > > >> I think Bob's suggestion to get to the root code change that caused this >> regression is important as it will help us assess all the other cases this >> testing hasn't even touched yet > > +1 > > Cheers > Jan > -- > > > >> >> On Mar 3, 2012, at 5:25 PM, Bob Dionne wrote: >> >>> I ran some tests, using Bob's latest script. The first versus the second >>> clearly show the regression. The third is the 1.2.x with the patch >>> to couch_os_process reverted and it seems to have no impact. The last has >>> Filipe's latest patch to couch_view_updater discussed in the >>> other thread and it brings the performance back to the 1.1.x level. >>> >>> I'd say that patch is a +1 >>> >>> 1.2.x >>> real3m3.093s >>> user0m0.028s >>> sys 0m0.008s >>> == >>> 1.1.x >>> real2m16.609s >>> user0m0.026s >>> sys 0m0.007s >>> = >>> 1.2.x with patch to couch_os_process reverted >>> real3m7.012s >>> user0m0.029s >>> sys 0m0.008s >>> = >>> 1.2.x with Filipe's katest patch to couch_view_updater >>> real2m11.038s >>> user0m0.028s >>> sys 0m0.007s >>> On Feb 28, 2012, at 8:17 AM, Jason Smith wrote: >>> Forgive the clean new thread. Hopefully it will not remain so. If you can, would you please clone https://github.com/jhs/slow_couchdb And build whatever Erlangs and CouchDB checkouts you see fit, and run the test. For example: docs=50 ./bench.sh small_doc.tpl That should run the test and, God willing, upload the results to a couch in the cloud. We should be able to use that information to identify who you are, whether you are on SSD, what Erlang and Couch build, and how fast it ran. Modulo bugs. >>> >> >
Re: Please report your indexing speed
On Mar 4, 2012, at 18:45 , Bob Dionne wrote: > yes, I was surprised by the 30% claim as my numbers showed it only getting > back to where we were with 1.1.x I see ~10% faster than where 1.1.1 was for small docs and ~30% for large docs. > I think Bob's suggestion to get to the root code change that caused this > regression is important as it will help us assess all the other cases this > testing hasn't even touched yet +1 Cheers Jan -- > > On Mar 3, 2012, at 5:25 PM, Bob Dionne wrote: > >> I ran some tests, using Bob's latest script. The first versus the second >> clearly show the regression. The third is the 1.2.x with the patch >> to couch_os_process reverted and it seems to have no impact. The last has >> Filipe's latest patch to couch_view_updater discussed in the >> other thread and it brings the performance back to the 1.1.x level. >> >> I'd say that patch is a +1 >> >> 1.2.x >> real 3m3.093s >> user 0m0.028s >> sys 0m0.008s >> == >> 1.1.x >> real 2m16.609s >> user 0m0.026s >> sys 0m0.007s >> = >> 1.2.x with patch to couch_os_process reverted >> real 3m7.012s >> user 0m0.029s >> sys 0m0.008s >> = >> 1.2.x with Filipe's katest patch to couch_view_updater >> real 2m11.038s >> user 0m0.028s >> sys 0m0.007s >> On Feb 28, 2012, at 8:17 AM, Jason Smith wrote: >> >>> Forgive the clean new thread. Hopefully it will not remain so. >>> >>> If you can, would you please clone https://github.com/jhs/slow_couchdb >>> >>> And build whatever Erlangs and CouchDB checkouts you see fit, and run >>> the test. For example: >>> >>> docs=50 ./bench.sh small_doc.tpl >>> >>> That should run the test and, God willing, upload the results to a >>> couch in the cloud. We should be able to use that information to >>> identify who you are, whether you are on SSD, what Erlang and Couch >>> build, and how fast it ran. Modulo bugs. >> >
Re: Please report your indexing speed
On Mar 4, 2012, at 18:40 , Jan Lehnardt wrote: > > On Mar 4, 2012, at 18:24 , Jan Lehnardt wrote: > >> I updated the google doc with results from an EC2 cc1.4xlarge instance >> (details are in the spreadsheet) >> >> This on EBS and Ubuntu 11.04/64. >> >> The results are bit different from the previous machine, but that isn't at >> all unexpected. >> >> tl;dr: for small docs (10bytes, 100bytes) 1.2.x-filipe beats 1.2.x and 1.1.1 >> , for large docs (1000bytes), 1.2.x beats 1.2.x-filipe (6% difference). > > Hah, I re-read through the results to make sure this is correct and I found a > mistake. A copy and paste formula error accounted for bigger improvements of > 1.2.x-filipe. This includes all my previous results. > > The good thing is 1.2.x-filipe is still faster, across the board than 1.1.1 > and 1.2.x. Still significantly, but not *as* much as about 30% in all but one > case. > > The tl;dr for the EC2 run can now be changed to that 1.2.x-filipe beats 1.1.1 > and 1.2.x for all docs, it's just that for large docs (1000bytes), 1.2.x is > faster than 1.1.1. But 1.2.x-filipe is even faster. > > >> So far, across the board, 1.2.x-filipe is ~16% faster (stdev 9%) than 1.1.1 >> for view builds. Sorry for misquoting this line, it is new and the most significant of this email, I'll just repeat it :) So far, across the board, 1.2.x-filipe is ~16% faster (stdev 9%) than 1.1.1 for view builds. The bigger the docs, the better the results, on both SSD and spinning disk. Cheers Jan -- > > > If you have any more hardware I could run this on, I'm happy to help with the > setup, it isn't hard :) > > Cheers > Jan > -- > > >> >> This still makes me want to include Filipe's patch into 1.2.x. >> >> Cheers >> Jan >> -- >> >> On Mar 4, 2012, at 10:24 , Jan Lehnardt wrote: >> >>> Hey all, >>> >>> I made another run with a bit of a different scenario. >>> >>> >>> # The Scenario >>> >>> I used a modified benchbulk.sh for inserting data (because it is an order >>> of magnitude faster than the other methods we had). I added a command line >>> parameter to specify the size of a single document in bytes (this was >>> previously hardcoded in the script). Note that this script creates docs in >>> a btree-friendly incrementing ID way. >>> >>> I added a new script benchview.sh which is basically the lower part of >>> Robert Newson's script. It creates a single view and queries it, measuring >>> execution time of curl. >>> >>> And a third matrix.sh (yay) that would run, on my system, different >>> configurations. >>> >>> See https://gist.github.com/1971611 for the scripts. >>> >>> I ran ./benchbulk $size && ./benchview.sh for the following combinations, >>> all on Mac OS X 10.7.3, Erlang R15B, Spidermonkey 1.8.5: >>> >>> - Doc sizes 10, 100, 1000 bytes >>> - CouchDB 1.1.1, 1.2.x (as of last night), 1.2.x-filipe (as of last night + >>> Filipe's patch from earlier in the thread) >>> - On an SSD and on a 5400rpm internal drive. >>> >>> I ran each individual test three times and took the average to compare >>> numbers. The full report (see below) includes each individual run's numbers) >>> >>> (The gist includes the raw output data from matrix.sh for the 5400rpm run, >>> for the SSDs, I don't have the original numbers anymore. I'm happy to >>> re-run this, if you want that data as well.) >>> >>> # The Numbers >>> >>> See >>> https://docs.google.com/spreadsheet/ccc?key=0AhESVUYnc_sQdDJ1Ry1KMTQ5enBDY0s1dHk2UVEzMHc >>> for the full data set. It'd be great to get a second pair of eyes to make >>> sure I didn't make any mistakes. >>> >>> See the "Grouped Data" sheet for comparisons. >>> >>> tl;dr: 1.2.x is about 30% slower and 1.2.x-filipe is about 30% faster than >>> 1.1.1 in the scenario above. >>> >>> >>> # Conclusion >>> >>> +1 to include Filipe's patch into 1.2.x. >>> >>> >>> >>> I'd love any feedback on methods, calculations and whatnot :) >>> >>> Also, I can run more variations, if you like, other Erlang or SpiderMokney >>> versions e.g., just let me know. >>> >>> >>> Cheers >>> Jan >>> -- >>> >>> On Feb 28, 2012, at 14:17 , Jason Smith wrote: >>> Forgive the clean new thread. Hopefully it will not remain so. If you can, would you please clone https://github.com/jhs/slow_couchdb And build whatever Erlangs and CouchDB checkouts you see fit, and run the test. For example: docs=50 ./bench.sh small_doc.tpl That should run the test and, God willing, upload the results to a couch in the cloud. We should be able to use that information to identify who you are, whether you are on SSD, what Erlang and Couch build, and how fast it ran. Modulo bugs. >>> >> >
Re: Please report your indexing speed
yes, I was surprised by the 30% claim as my numbers showed it only getting back to where we were with 1.1.x I think Bob's suggestion to get to the root code change that caused this regression is important as it will help us assess all the other cases this testing hasn't even touched yet On Mar 3, 2012, at 5:25 PM, Bob Dionne wrote: > I ran some tests, using Bob's latest script. The first versus the second > clearly show the regression. The third is the 1.2.x with the patch > to couch_os_process reverted and it seems to have no impact. The last has > Filipe's latest patch to couch_view_updater discussed in the > other thread and it brings the performance back to the 1.1.x level. > > I'd say that patch is a +1 > > 1.2.x > real 3m3.093s > user 0m0.028s > sys 0m0.008s > == > 1.1.x > real 2m16.609s > user 0m0.026s > sys 0m0.007s > = > 1.2.x with patch to couch_os_process reverted > real 3m7.012s > user 0m0.029s > sys 0m0.008s > = > 1.2.x with Filipe's katest patch to couch_view_updater > real 2m11.038s > user 0m0.028s > sys 0m0.007s > On Feb 28, 2012, at 8:17 AM, Jason Smith wrote: > >> Forgive the clean new thread. Hopefully it will not remain so. >> >> If you can, would you please clone https://github.com/jhs/slow_couchdb >> >> And build whatever Erlangs and CouchDB checkouts you see fit, and run >> the test. For example: >> >> docs=50 ./bench.sh small_doc.tpl >> >> That should run the test and, God willing, upload the results to a >> couch in the cloud. We should be able to use that information to >> identify who you are, whether you are on SSD, what Erlang and Couch >> build, and how fast it ran. Modulo bugs. >
Re: Please report your indexing speed
On Mar 4, 2012, at 18:24 , Jan Lehnardt wrote: > I updated the google doc with results from an EC2 cc1.4xlarge instance > (details are in the spreadsheet) > > This on EBS and Ubuntu 11.04/64. > > The results are bit different from the previous machine, but that isn't at > all unexpected. > > tl;dr: for small docs (10bytes, 100bytes) 1.2.x-filipe beats 1.2.x and 1.1.1 > , for large docs (1000bytes), 1.2.x beats 1.2.x-filipe (6% difference). Hah, I re-read through the results to make sure this is correct and I found a mistake. A copy and paste formula error accounted for bigger improvements of 1.2.x-filipe. This includes all my previous results. The good thing is 1.2.x-filipe is still faster, across the board than 1.1.1 and 1.2.x. Still significantly, but not *as* much as about 30% in all but one case. The tl;dr for the EC2 run can now be changed to that 1.2.x-filipe beats 1.1.1 and 1.2.x for all docs, it's just that for large docs (1000bytes), 1.2.x is faster than 1.1.1. But 1.2.x-filipe is even faster. > So far, across the board, 1.2.x-filipe is ~16% faster (stdev 9%) than 1.1.1 for view builds. If you have any more hardware I could run this on, I'm happy to help with the setup, it isn't hard :) Cheers Jan -- > > This still makes me want to include Filipe's patch into 1.2.x. > > Cheers > Jan > -- > > On Mar 4, 2012, at 10:24 , Jan Lehnardt wrote: > >> Hey all, >> >> I made another run with a bit of a different scenario. >> >> >> # The Scenario >> >> I used a modified benchbulk.sh for inserting data (because it is an order of >> magnitude faster than the other methods we had). I added a command line >> parameter to specify the size of a single document in bytes (this was >> previously hardcoded in the script). Note that this script creates docs in a >> btree-friendly incrementing ID way. >> >> I added a new script benchview.sh which is basically the lower part of >> Robert Newson's script. It creates a single view and queries it, measuring >> execution time of curl. >> >> And a third matrix.sh (yay) that would run, on my system, different >> configurations. >> >> See https://gist.github.com/1971611 for the scripts. >> >> I ran ./benchbulk $size && ./benchview.sh for the following combinations, >> all on Mac OS X 10.7.3, Erlang R15B, Spidermonkey 1.8.5: >> >> - Doc sizes 10, 100, 1000 bytes >> - CouchDB 1.1.1, 1.2.x (as of last night), 1.2.x-filipe (as of last night + >> Filipe's patch from earlier in the thread) >> - On an SSD and on a 5400rpm internal drive. >> >> I ran each individual test three times and took the average to compare >> numbers. The full report (see below) includes each individual run's numbers) >> >> (The gist includes the raw output data from matrix.sh for the 5400rpm run, >> for the SSDs, I don't have the original numbers anymore. I'm happy to re-run >> this, if you want that data as well.) >> >> # The Numbers >> >> See >> https://docs.google.com/spreadsheet/ccc?key=0AhESVUYnc_sQdDJ1Ry1KMTQ5enBDY0s1dHk2UVEzMHc >> for the full data set. It'd be great to get a second pair of eyes to make >> sure I didn't make any mistakes. >> >> See the "Grouped Data" sheet for comparisons. >> >> tl;dr: 1.2.x is about 30% slower and 1.2.x-filipe is about 30% faster than >> 1.1.1 in the scenario above. >> >> >> # Conclusion >> >> +1 to include Filipe's patch into 1.2.x. >> >> >> >> I'd love any feedback on methods, calculations and whatnot :) >> >> Also, I can run more variations, if you like, other Erlang or SpiderMokney >> versions e.g., just let me know. >> >> >> Cheers >> Jan >> -- >> >> On Feb 28, 2012, at 14:17 , Jason Smith wrote: >> >>> Forgive the clean new thread. Hopefully it will not remain so. >>> >>> If you can, would you please clone https://github.com/jhs/slow_couchdb >>> >>> And build whatever Erlangs and CouchDB checkouts you see fit, and run >>> the test. For example: >>> >>> docs=50 ./bench.sh small_doc.tpl >>> >>> That should run the test and, God willing, upload the results to a >>> couch in the cloud. We should be able to use that information to >>> identify who you are, whether you are on SSD, what Erlang and Couch >>> build, and how fast it ran. Modulo bugs. >> >
Re: Please report your indexing speed
On Mar 4, 2012, at 13:03 , Bob Dionne wrote: > Great Jan, so this confirms my back of the envelope test using Bob's script > and Filipe's results. The patch is definitely helpful. > > I was wondering why no one had looked at test/bench, perhaps this more > rigorous approach could provide the basis for a comprehensive performance tool Good call! I'd really like that our current efforts morph into a situation where we can `make perf` and get a bunch of good results to compare to other builds' `make perf`. Down the road, though, I think we need to write Erlang tools to do that, so Windows users can run them without too much trouble. (we could also bundle whatever scripting environment or C-based binaries with the builds, but since we already ship Erlang, we might as well use it :) Cheers Jan -- > > On Mar 4, 2012, at 4:24 AM, Jan Lehnardt wrote: > >> Hey all, >> >> I made another run with a bit of a different scenario. >> >> >> # The Scenario >> >> I used a modified benchbulk.sh for inserting data (because it is an order of >> magnitude faster than the other methods we had). I added a command line >> parameter to specify the size of a single document in bytes (this was >> previously hardcoded in the script). Note that this script creates docs in a >> btree-friendly incrementing ID way. >> >> I added a new script benchview.sh which is basically the lower part of >> Robert Newson's script. It creates a single view and queries it, measuring >> execution time of curl. >> >> And a third matrix.sh (yay) that would run, on my system, different >> configurations. >> >> See https://gist.github.com/1971611 for the scripts. >> >> I ran ./benchbulk $size && ./benchview.sh for the following combinations, >> all on Mac OS X 10.7.3, Erlang R15B, Spidermonkey 1.8.5: >> >> - Doc sizes 10, 100, 1000 bytes >> - CouchDB 1.1.1, 1.2.x (as of last night), 1.2.x-filipe (as of last night + >> Filipe's patch from earlier in the thread) >> - On an SSD and on a 5400rpm internal drive. >> >> I ran each individual test three times and took the average to compare >> numbers. The full report (see below) includes each individual run's numbers) >> >> (The gist includes the raw output data from matrix.sh for the 5400rpm run, >> for the SSDs, I don't have the original numbers anymore. I'm happy to re-run >> this, if you want that data as well.) >> >> # The Numbers >> >> See >> https://docs.google.com/spreadsheet/ccc?key=0AhESVUYnc_sQdDJ1Ry1KMTQ5enBDY0s1dHk2UVEzMHc >> for the full data set. It'd be great to get a second pair of eyes to make >> sure I didn't make any mistakes. >> >> See the "Grouped Data" sheet for comparisons. >> >> tl;dr: 1.2.x is about 30% slower and 1.2.x-filipe is about 30% faster than >> 1.1.1 in the scenario above. >> >> >> # Conclusion >> >> +1 to include Filipe's patch into 1.2.x. >> >> >> >> I'd love any feedback on methods, calculations and whatnot :) >> >> Also, I can run more variations, if you like, other Erlang or SpiderMokney >> versions e.g., just let me know. >> >> >> Cheers >> Jan >> -- >> >> On Feb 28, 2012, at 14:17 , Jason Smith wrote: >> >>> Forgive the clean new thread. Hopefully it will not remain so. >>> >>> If you can, would you please clone https://github.com/jhs/slow_couchdb >>> >>> And build whatever Erlangs and CouchDB checkouts you see fit, and run >>> the test. For example: >>> >>> docs=50 ./bench.sh small_doc.tpl >>> >>> That should run the test and, God willing, upload the results to a >>> couch in the cloud. We should be able to use that information to >>> identify who you are, whether you are on SSD, what Erlang and Couch >>> build, and how fast it ran. Modulo bugs. >> >
Re: Please report your indexing speed
I updated the google doc with results from an EC2 cc1.4xlarge instance (details are in the spreadsheet) This on EBS and Ubuntu 11.04/64. The results are bit different from the previous machine, but that isn't at all unexpected. tl;dr: for small docs (10bytes, 100bytes) 1.2.x-filipe beats 1.2.x and 1.1.1 , for large docs (1000bytes), 1.2.x beats 1.2.x-filipe (6% difference). This still makes me want to include Filipe's patch into 1.2.x. Cheers Jan -- On Mar 4, 2012, at 10:24 , Jan Lehnardt wrote: > Hey all, > > I made another run with a bit of a different scenario. > > > # The Scenario > > I used a modified benchbulk.sh for inserting data (because it is an order of > magnitude faster than the other methods we had). I added a command line > parameter to specify the size of a single document in bytes (this was > previously hardcoded in the script). Note that this script creates docs in a > btree-friendly incrementing ID way. > > I added a new script benchview.sh which is basically the lower part of Robert > Newson's script. It creates a single view and queries it, measuring execution > time of curl. > > And a third matrix.sh (yay) that would run, on my system, different > configurations. > > See https://gist.github.com/1971611 for the scripts. > > I ran ./benchbulk $size && ./benchview.sh for the following combinations, all > on Mac OS X 10.7.3, Erlang R15B, Spidermonkey 1.8.5: > > - Doc sizes 10, 100, 1000 bytes > - CouchDB 1.1.1, 1.2.x (as of last night), 1.2.x-filipe (as of last night + > Filipe's patch from earlier in the thread) > - On an SSD and on a 5400rpm internal drive. > > I ran each individual test three times and took the average to compare > numbers. The full report (see below) includes each individual run's numbers) > > (The gist includes the raw output data from matrix.sh for the 5400rpm run, > for the SSDs, I don't have the original numbers anymore. I'm happy to re-run > this, if you want that data as well.) > > # The Numbers > > See > https://docs.google.com/spreadsheet/ccc?key=0AhESVUYnc_sQdDJ1Ry1KMTQ5enBDY0s1dHk2UVEzMHc > for the full data set. It'd be great to get a second pair of eyes to make > sure I didn't make any mistakes. > > See the "Grouped Data" sheet for comparisons. > > tl;dr: 1.2.x is about 30% slower and 1.2.x-filipe is about 30% faster than > 1.1.1 in the scenario above. > > > # Conclusion > > +1 to include Filipe's patch into 1.2.x. > > > > I'd love any feedback on methods, calculations and whatnot :) > > Also, I can run more variations, if you like, other Erlang or SpiderMokney > versions e.g., just let me know. > > > Cheers > Jan > -- > > On Feb 28, 2012, at 14:17 , Jason Smith wrote: > >> Forgive the clean new thread. Hopefully it will not remain so. >> >> If you can, would you please clone https://github.com/jhs/slow_couchdb >> >> And build whatever Erlangs and CouchDB checkouts you see fit, and run >> the test. For example: >> >> docs=50 ./bench.sh small_doc.tpl >> >> That should run the test and, God willing, upload the results to a >> couch in the cloud. We should be able to use that information to >> identify who you are, whether you are on SSD, what Erlang and Couch >> build, and how fast it ran. Modulo bugs. >
Re: Please report your indexing speed
Excellent stuff, thanks Jan. I think it would be prudent to attempt to identify which patch, or, more likely, which part of the patch, caused the 30% regression and why. I will attempt that tomorrow or possibly later today. B. On 4 March 2012 12:03, Bob Dionne wrote: > Great Jan, so this confirms my back of the envelope test using Bob's script > and Filipe's results. The patch is definitely helpful. > > I was wondering why no one had looked at test/bench, perhaps this more > rigorous approach could provide the basis for a comprehensive performance tool > > On Mar 4, 2012, at 4:24 AM, Jan Lehnardt wrote: > >> Hey all, >> >> I made another run with a bit of a different scenario. >> >> >> # The Scenario >> >> I used a modified benchbulk.sh for inserting data (because it is an order of >> magnitude faster than the other methods we had). I added a command line >> parameter to specify the size of a single document in bytes (this was >> previously hardcoded in the script). Note that this script creates docs in a >> btree-friendly incrementing ID way. >> >> I added a new script benchview.sh which is basically the lower part of >> Robert Newson's script. It creates a single view and queries it, measuring >> execution time of curl. >> >> And a third matrix.sh (yay) that would run, on my system, different >> configurations. >> >> See https://gist.github.com/1971611 for the scripts. >> >> I ran ./benchbulk $size && ./benchview.sh for the following combinations, >> all on Mac OS X 10.7.3, Erlang R15B, Spidermonkey 1.8.5: >> >> - Doc sizes 10, 100, 1000 bytes >> - CouchDB 1.1.1, 1.2.x (as of last night), 1.2.x-filipe (as of last night + >> Filipe's patch from earlier in the thread) >> - On an SSD and on a 5400rpm internal drive. >> >> I ran each individual test three times and took the average to compare >> numbers. The full report (see below) includes each individual run's numbers) >> >> (The gist includes the raw output data from matrix.sh for the 5400rpm run, >> for the SSDs, I don't have the original numbers anymore. I'm happy to re-run >> this, if you want that data as well.) >> >> # The Numbers >> >> See >> https://docs.google.com/spreadsheet/ccc?key=0AhESVUYnc_sQdDJ1Ry1KMTQ5enBDY0s1dHk2UVEzMHc >> for the full data set. It'd be great to get a second pair of eyes to make >> sure I didn't make any mistakes. >> >> See the "Grouped Data" sheet for comparisons. >> >> tl;dr: 1.2.x is about 30% slower and 1.2.x-filipe is about 30% faster than >> 1.1.1 in the scenario above. >> >> >> # Conclusion >> >> +1 to include Filipe's patch into 1.2.x. >> >> >> >> I'd love any feedback on methods, calculations and whatnot :) >> >> Also, I can run more variations, if you like, other Erlang or SpiderMokney >> versions e.g., just let me know. >> >> >> Cheers >> Jan >> -- >> >> On Feb 28, 2012, at 14:17 , Jason Smith wrote: >> >>> Forgive the clean new thread. Hopefully it will not remain so. >>> >>> If you can, would you please clone https://github.com/jhs/slow_couchdb >>> >>> And build whatever Erlangs and CouchDB checkouts you see fit, and run >>> the test. For example: >>> >>> docs=50 ./bench.sh small_doc.tpl >>> >>> That should run the test and, God willing, upload the results to a >>> couch in the cloud. We should be able to use that information to >>> identify who you are, whether you are on SSD, what Erlang and Couch >>> build, and how fast it ran. Modulo bugs. >> >
Re: Please report your indexing speed
Great Jan, so this confirms my back of the envelope test using Bob's script and Filipe's results. The patch is definitely helpful. I was wondering why no one had looked at test/bench, perhaps this more rigorous approach could provide the basis for a comprehensive performance tool On Mar 4, 2012, at 4:24 AM, Jan Lehnardt wrote: > Hey all, > > I made another run with a bit of a different scenario. > > > # The Scenario > > I used a modified benchbulk.sh for inserting data (because it is an order of > magnitude faster than the other methods we had). I added a command line > parameter to specify the size of a single document in bytes (this was > previously hardcoded in the script). Note that this script creates docs in a > btree-friendly incrementing ID way. > > I added a new script benchview.sh which is basically the lower part of Robert > Newson's script. It creates a single view and queries it, measuring execution > time of curl. > > And a third matrix.sh (yay) that would run, on my system, different > configurations. > > See https://gist.github.com/1971611 for the scripts. > > I ran ./benchbulk $size && ./benchview.sh for the following combinations, all > on Mac OS X 10.7.3, Erlang R15B, Spidermonkey 1.8.5: > > - Doc sizes 10, 100, 1000 bytes > - CouchDB 1.1.1, 1.2.x (as of last night), 1.2.x-filipe (as of last night + > Filipe's patch from earlier in the thread) > - On an SSD and on a 5400rpm internal drive. > > I ran each individual test three times and took the average to compare > numbers. The full report (see below) includes each individual run's numbers) > > (The gist includes the raw output data from matrix.sh for the 5400rpm run, > for the SSDs, I don't have the original numbers anymore. I'm happy to re-run > this, if you want that data as well.) > > # The Numbers > > See > https://docs.google.com/spreadsheet/ccc?key=0AhESVUYnc_sQdDJ1Ry1KMTQ5enBDY0s1dHk2UVEzMHc > for the full data set. It'd be great to get a second pair of eyes to make > sure I didn't make any mistakes. > > See the "Grouped Data" sheet for comparisons. > > tl;dr: 1.2.x is about 30% slower and 1.2.x-filipe is about 30% faster than > 1.1.1 in the scenario above. > > > # Conclusion > > +1 to include Filipe's patch into 1.2.x. > > > > I'd love any feedback on methods, calculations and whatnot :) > > Also, I can run more variations, if you like, other Erlang or SpiderMokney > versions e.g., just let me know. > > > Cheers > Jan > -- > > On Feb 28, 2012, at 14:17 , Jason Smith wrote: > >> Forgive the clean new thread. Hopefully it will not remain so. >> >> If you can, would you please clone https://github.com/jhs/slow_couchdb >> >> And build whatever Erlangs and CouchDB checkouts you see fit, and run >> the test. For example: >> >> docs=50 ./bench.sh small_doc.tpl >> >> That should run the test and, God willing, upload the results to a >> couch in the cloud. We should be able to use that information to >> identify who you are, whether you are on SSD, what Erlang and Couch >> build, and how fast it ran. Modulo bugs. >
Re: Please report your indexing speed
Hey all, I made another run with a bit of a different scenario. # The Scenario I used a modified benchbulk.sh for inserting data (because it is an order of magnitude faster than the other methods we had). I added a command line parameter to specify the size of a single document in bytes (this was previously hardcoded in the script). Note that this script creates docs in a btree-friendly incrementing ID way. I added a new script benchview.sh which is basically the lower part of Robert Newson's script. It creates a single view and queries it, measuring execution time of curl. And a third matrix.sh (yay) that would run, on my system, different configurations. See https://gist.github.com/1971611 for the scripts. I ran ./benchbulk $size && ./benchview.sh for the following combinations, all on Mac OS X 10.7.3, Erlang R15B, Spidermonkey 1.8.5: - Doc sizes 10, 100, 1000 bytes - CouchDB 1.1.1, 1.2.x (as of last night), 1.2.x-filipe (as of last night + Filipe's patch from earlier in the thread) - On an SSD and on a 5400rpm internal drive. I ran each individual test three times and took the average to compare numbers. The full report (see below) includes each individual run's numbers) (The gist includes the raw output data from matrix.sh for the 5400rpm run, for the SSDs, I don't have the original numbers anymore. I'm happy to re-run this, if you want that data as well.) # The Numbers See https://docs.google.com/spreadsheet/ccc?key=0AhESVUYnc_sQdDJ1Ry1KMTQ5enBDY0s1dHk2UVEzMHc for the full data set. It'd be great to get a second pair of eyes to make sure I didn't make any mistakes. See the "Grouped Data" sheet for comparisons. tl;dr: 1.2.x is about 30% slower and 1.2.x-filipe is about 30% faster than 1.1.1 in the scenario above. # Conclusion +1 to include Filipe's patch into 1.2.x. I'd love any feedback on methods, calculations and whatnot :) Also, I can run more variations, if you like, other Erlang or SpiderMokney versions e.g., just let me know. Cheers Jan -- On Feb 28, 2012, at 14:17 , Jason Smith wrote: > Forgive the clean new thread. Hopefully it will not remain so. > > If you can, would you please clone https://github.com/jhs/slow_couchdb > > And build whatever Erlangs and CouchDB checkouts you see fit, and run > the test. For example: > >docs=50 ./bench.sh small_doc.tpl > > That should run the test and, God willing, upload the results to a > couch in the cloud. We should be able to use that information to > identify who you are, whether you are on SSD, what Erlang and Couch > build, and how fast it ran. Modulo bugs.
Re: Please report your indexing speed
I ran some tests, using Bob's latest script. The first versus the second clearly show the regression. The third is the 1.2.x with the patch to couch_os_process reverted and it seems to have no impact. The last has Filipe's latest patch to couch_view_updater discussed in the other thread and it brings the performance back to the 1.1.x level. I'd say that patch is a +1 1.2.x real3m3.093s user0m0.028s sys 0m0.008s == 1.1.x real2m16.609s user0m0.026s sys 0m0.007s = 1.2.x with patch to couch_os_process reverted real3m7.012s user0m0.029s sys 0m0.008s = 1.2.x with Filipe's katest patch to couch_view_updater real2m11.038s user0m0.028s sys 0m0.007s On Feb 28, 2012, at 8:17 AM, Jason Smith wrote: > Forgive the clean new thread. Hopefully it will not remain so. > > If you can, would you please clone https://github.com/jhs/slow_couchdb > > And build whatever Erlangs and CouchDB checkouts you see fit, and run > the test. For example: > >docs=50 ./bench.sh small_doc.tpl > > That should run the test and, God willing, upload the results to a > couch in the cloud. We should be able to use that information to > identify who you are, whether you are on SSD, what Erlang and Couch > build, and how fast it ran. Modulo bugs.
Re: Please report your indexing speed
# localhost, mac with vmware fusion internal network to windows 7 vm Disk: /dev/disk0 Device / Media Name: APPLE SSD TS128C Media Solid State: Yes dave@akai slow_couchdb % ./bench.sh small_doc.tpl Testing 5 of small_doc.tpl in batches of 1 all with js 1.8.5, I can compare 1.8.0 if interested. Server: CouchDB/1.1.1 (Erlang OTP/R14B03) real0m13.800s user0m0.006s sys 0m0.003s Server: CouchDB/1.2.0 (Erlang OTP/R14B04) real0m19.185s user0m0.006s sys 0m0.003s Server: CouchDB/1.2.0 (Erlang OTP/R15B) Date: Thu, 01 Mar 2012 12:50:47 GMT real0m18.428s user0m0.006s sys 0m0.003s Should get spinning disk (on a 2006 iMac) over the weekend with luck. Is there a jira for this investigation? A+ Dave
Re: Please report your indexing speed
Here are my 1.1.1 results on the same hardware/kernel/spidermonkey:
{"couchdb":"Welcome","version":"1.1.1"}
real0m32.029s
user0m0.000s
sys 0m0.006s
Wendall
On 02/29/2012 01:30 PM, Robert Newson wrote:
Benoit, could you run the 1.7.0 test for longer? Say, double the size
of BULK_COUNT?
Wendall, that's a useful new data point. Any chance you could compare
1.1.1 with 1.2 so we can line it up with the other results?
B.
On 29 February 2012 19:40, Wendall Cada wrote:
I'm using the bash script provided by Robert Newson.
Linux version 3.2.7-1.fc16.i686.PAE
(mockbu...@x86-16.phx2.fedoraproject.org) (gcc version 4.6.2 20111027 (Red
Hat 4.6.2-1) (GCC) )
spidermonkey 1.8.5
i686
{"couchdb":"Welcome","version":"1.0.3"} (from stock fedora rpm Release:
2.fc16)
real0m35.652s
user0m0.001s
sys 0m0.006s
{"couchdb":"Welcome","version":"1.2.0"} git 4cd60f3d1683
real0m20.134s
user0m0.002s
sys 0m0.004s
Wendall
On 02/28/2012 05:17 AM, Jason Smith wrote:
Forgive the clean new thread. Hopefully it will not remain so.
If you can, would you please clone https://github.com/jhs/slow_couchdb
And build whatever Erlangs and CouchDB checkouts you see fit, and run
the test. For example:
docs=50 ./bench.sh small_doc.tpl
That should run the test and, God willing, upload the results to a
couch in the cloud. We should be able to use that information to
identify who you are, whether you are on SSD, what Erlang and Couch
build, and how fast it ran. Modulo bugs.
Re: Please report your indexing speed
Benoit, could you run the 1.7.0 test for longer? Say, double the size
of BULK_COUNT?
Wendall, that's a useful new data point. Any chance you could compare
1.1.1 with 1.2 so we can line it up with the other results?
B.
On 29 February 2012 19:40, Wendall Cada wrote:
> I'm using the bash script provided by Robert Newson.
>
> Linux version 3.2.7-1.fc16.i686.PAE
> (mockbu...@x86-16.phx2.fedoraproject.org) (gcc version 4.6.2 20111027 (Red
> Hat 4.6.2-1) (GCC) )
> spidermonkey 1.8.5
> i686
>
> {"couchdb":"Welcome","version":"1.0.3"} (from stock fedora rpm Release:
> 2.fc16)
> real 0m35.652s
> user 0m0.001s
> sys 0m0.006s
>
> {"couchdb":"Welcome","version":"1.2.0"} git 4cd60f3d1683
> real 0m20.134s
> user 0m0.002s
> sys 0m0.004s
>
> Wendall
>
>
> On 02/28/2012 05:17 AM, Jason Smith wrote:
>>
>> Forgive the clean new thread. Hopefully it will not remain so.
>>
>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>
>> And build whatever Erlangs and CouchDB checkouts you see fit, and run
>> the test. For example:
>>
>> docs=50 ./bench.sh small_doc.tpl
>>
>> That should run the test and, God willing, upload the results to a
>> couch in the cloud. We should be able to use that information to
>> identify who you are, whether you are on SSD, what Erlang and Couch
>> build, and how fast it ran. Modulo bugs.
>
>
Re: Please report your indexing speed
I'm using the bash script provided by Robert Newson.
Linux version 3.2.7-1.fc16.i686.PAE
(mockbu...@x86-16.phx2.fedoraproject.org) (gcc version 4.6.2 20111027
(Red Hat 4.6.2-1) (GCC) )
spidermonkey 1.8.5
i686
{"couchdb":"Welcome","version":"1.0.3"} (from stock fedora rpm Release:
2.fc16)
real0m35.652s
user0m0.001s
sys 0m0.006s
{"couchdb":"Welcome","version":"1.2.0"} git 4cd60f3d1683
real0m20.134s
user0m0.002s
sys 0m0.004s
Wendall
On 02/28/2012 05:17 AM, Jason Smith wrote:
Forgive the clean new thread. Hopefully it will not remain so.
If you can, would you please clone https://github.com/jhs/slow_couchdb
And build whatever Erlangs and CouchDB checkouts you see fit, and run
the test. For example:
docs=50 ./bench.sh small_doc.tpl
That should run the test and, God willing, upload the results to a
couch in the cloud. We should be able to use that information to
identify who you are, whether you are on SSD, what Erlang and Couch
build, and how fast it ran. Modulo bugs.
Re: Please report your indexing speed
On Wed, Feb 29, 2012 at 6:25 PM, Benoit Chesneau wrote: > On Wed, Feb 29, 2012 at 6:07 PM, Benoit Chesneau wrote: >> On Wed, Feb 29, 2012 at 5:53 PM, Benoit Chesneau wrote: >>> On Wed, Feb 29, 2012 at 4:03 PM, Robert Newson wrote: I've produced a new script that reproduces the view regression. I apologize in advance for exposing my awful Bash scripting abilities (also my inability to write "pure" shell. Your "Bashism" is my "it works"). I get 0m56.521s for 1.1.x and 1m17.108s for 1.2.x. That is, 1.1.x complete the same task in only 72% of the time that 1.2.x takes. http://friendpaste.com/UI1OcECLEzR6i4D75LqQy >>> >>> hrm with the same script: >>> >>> 1.2.0 : >>> real 0m23.842s >>> user 0m0.007s >>> sys 0m0.004s >>> >>> 1.1.1: >>> real 0m28.625s >>> user 0m0.008s >>> sys 0m0.005s >>> >>> hw specs : >>> $ uname -a >>> Darwin enki.local 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12 >>> 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64 >>> mba 2011: i7 4 GB SSD 256 GB >>> >>> >>> - benoît >> >> Just for my curiosity I tested it in the couchdb based distribution I >> maintain (head, last mochiweb + jiffy) : >> >> >> real 0m19.539s >> user 0m0.007s >> sys 0m0.005s >> done > > I launched the tests a second time and same results: > > 1.1.1 : > real 0m29.155s > user 0m0.009s > sys 0m0.006s > > 1.2.0 : > > real 0m25.466s > user 0m0.008s > sys 0m0.005s tested with 10 docs : 1.1.1: real1m2.352s user0m0.007s sys 0m0.013s 1.2.0: real0m52.005s user0m0.009s sys 0m0.008s All previous tests were with 1.8.5 . With 1.7.0 on 5 docs: 1.1.1: real2m3.828s user0m0.009s sys 0m0.011s 1.2.0: real2m49.243s user0m0.009s sys 0m0.011s - benoît
Re: Please report your indexing speed
On Feb 29, 2012, at 18:44 , Jan Lehnardt wrote:
>
> On Feb 29, 2012, at 18:42 , Robert Newson wrote:
>
>> My new test again;
>>
>> Filling db.
>> done
>> server is;
>> {"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
>> Building views.
>>
>> real 0m24.973s
>> user 0m0.006s
>> sys 0m0.004s
>> done
>> ~/Source/sandbox $ ./bench.sh
>> Filling db.
>> done
>> server is;
>> {"couchdb":"Welcome","version":"1.2.0"}
>> Building views.
>>
>> real 0m21.925s
>> user 0m0.006s
>> sys 0m0.003s
>> done
>>
>> Important Note: The builds that show *regression* were all using
>> SpiderMonkey 1.7.0. The tests above were 1.8.5. I think that's
>> significant. Can other people who saw regressions report their
>> spidermonkey version?
>
>
> Good observation!
>
> Except for the linux VPS one that had 1.8.0, all previous reports have been
> on 1.8.5.
On the Mac from earlier, using 1.7.0 via homebrew:
1.1.1: 0m54.203s
1.2.x: 0m55.639s
Cheers
Jan
--
>
> Cheers
> Jan
> --
>
>>
>> B.
>>
>> On 29 February 2012 17:25, Benoit Chesneau wrote:
>>> On Wed, Feb 29, 2012 at 6:07 PM, Benoit Chesneau
>>> wrote:
On Wed, Feb 29, 2012 at 5:53 PM, Benoit Chesneau
wrote:
> On Wed, Feb 29, 2012 at 4:03 PM, Robert Newson wrote:
>> I've produced a new script that reproduces the view regression. I
>> apologize in advance for exposing my awful Bash scripting abilities
>> (also my inability to write "pure" shell. Your "Bashism" is my "it
>> works").
>>
>> I get 0m56.521s for 1.1.x and 1m17.108s for 1.2.x. That is, 1.1.x
>> complete the same task in only 72% of the time that 1.2.x takes.
>>
>> http://friendpaste.com/UI1OcECLEzR6i4D75LqQy
>>
>
> hrm with the same script:
>
> 1.2.0 :
> real0m23.842s
> user0m0.007s
> sys 0m0.004s
>
> 1.1.1:
> real0m28.625s
> user0m0.008s
> sys 0m0.005s
>
> hw specs :
> $ uname -a
> Darwin enki.local 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12
> 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64
> mba 2011: i7 4 GB SSD 256 GB
>
>
> - benoît
Just for my curiosity I tested it in the couchdb based distribution I
maintain (head, last mochiweb + jiffy) :
real0m19.539s
user0m0.007s
sys 0m0.005s
done
>>>
>>> I launched the tests a second time and same results:
>>>
>>> 1.1.1 :
>>> real0m29.155s
>>> user0m0.009s
>>> sys 0m0.006s
>>>
>>> 1.2.0 :
>>>
>>> real0m25.466s
>>> user0m0.008s
>>> sys 0m0.005s
>
Re: Please report your indexing speed
On Feb 29, 2012, at 18:42 , Robert Newson wrote:
> My new test again;
>
> Filling db.
> done
> server is;
> {"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
> Building views.
>
> real 0m24.973s
> user 0m0.006s
> sys 0m0.004s
> done
> ~/Source/sandbox $ ./bench.sh
> Filling db.
> done
> server is;
> {"couchdb":"Welcome","version":"1.2.0"}
> Building views.
>
> real 0m21.925s
> user 0m0.006s
> sys 0m0.003s
> done
>
> Important Note: The builds that show *regression* were all using
> SpiderMonkey 1.7.0. The tests above were 1.8.5. I think that's
> significant. Can other people who saw regressions report their
> spidermonkey version?
Good observation!
Except for the linux VPS one that had 1.8.0, all previous reports have been
on 1.8.5.
Cheers
Jan
--
>
> B.
>
> On 29 February 2012 17:25, Benoit Chesneau wrote:
>> On Wed, Feb 29, 2012 at 6:07 PM, Benoit Chesneau wrote:
>>> On Wed, Feb 29, 2012 at 5:53 PM, Benoit Chesneau
>>> wrote:
On Wed, Feb 29, 2012 at 4:03 PM, Robert Newson wrote:
> I've produced a new script that reproduces the view regression. I
> apologize in advance for exposing my awful Bash scripting abilities
> (also my inability to write "pure" shell. Your "Bashism" is my "it
> works").
>
> I get 0m56.521s for 1.1.x and 1m17.108s for 1.2.x. That is, 1.1.x
> complete the same task in only 72% of the time that 1.2.x takes.
>
> http://friendpaste.com/UI1OcECLEzR6i4D75LqQy
>
hrm with the same script:
1.2.0 :
real0m23.842s
user0m0.007s
sys 0m0.004s
1.1.1:
real0m28.625s
user0m0.008s
sys 0m0.005s
hw specs :
$ uname -a
Darwin enki.local 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12
18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64
mba 2011: i7 4 GB SSD 256 GB
- benoît
>>>
>>> Just for my curiosity I tested it in the couchdb based distribution I
>>> maintain (head, last mochiweb + jiffy) :
>>>
>>>
>>> real0m19.539s
>>> user0m0.007s
>>> sys 0m0.005s
>>> done
>>
>> I launched the tests a second time and same results:
>>
>> 1.1.1 :
>> real0m29.155s
>> user0m0.009s
>> sys 0m0.006s
>>
>> 1.2.0 :
>>
>> real0m25.466s
>> user0m0.008s
>> sys 0m0.005s
Re: Please report your indexing speed
And one more on a Linux Ubuntu VPS, spinning disk dual core cpu: 1.1.1: 0m54.721s 1.2.x: 0m40.356s (SpiderMonkey 1.8.0) On Feb 29, 2012, at 18:25 , Benoit Chesneau wrote: > On Wed, Feb 29, 2012 at 6:07 PM, Benoit Chesneau wrote: >> On Wed, Feb 29, 2012 at 5:53 PM, Benoit Chesneau wrote: >>> On Wed, Feb 29, 2012 at 4:03 PM, Robert Newson wrote: I've produced a new script that reproduces the view regression. I apologize in advance for exposing my awful Bash scripting abilities (also my inability to write "pure" shell. Your "Bashism" is my "it works"). I get 0m56.521s for 1.1.x and 1m17.108s for 1.2.x. That is, 1.1.x complete the same task in only 72% of the time that 1.2.x takes. http://friendpaste.com/UI1OcECLEzR6i4D75LqQy >>> >>> hrm with the same script: >>> >>> 1.2.0 : >>> real0m23.842s >>> user0m0.007s >>> sys 0m0.004s >>> >>> 1.1.1: >>> real0m28.625s >>> user0m0.008s >>> sys 0m0.005s >>> >>> hw specs : >>> $ uname -a >>> Darwin enki.local 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12 >>> 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64 >>> mba 2011: i7 4 GB SSD 256 GB >>> >>> >>> - benoît >> >> Just for my curiosity I tested it in the couchdb based distribution I >> maintain (head, last mochiweb + jiffy) : >> >> >> real0m19.539s >> user0m0.007s >> sys 0m0.005s >> done > > I launched the tests a second time and same results: > > 1.1.1 : > real 0m29.155s > user 0m0.009s > sys 0m0.006s > > 1.2.0 : > > real 0m25.466s > user 0m0.008s > sys 0m0.005s
Re: Please report your indexing speed
My new test again;
Filling db.
done
server is;
{"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
Building views.
real0m24.973s
user0m0.006s
sys 0m0.004s
done
~/Source/sandbox $ ./bench.sh
Filling db.
done
server is;
{"couchdb":"Welcome","version":"1.2.0"}
Building views.
real0m21.925s
user0m0.006s
sys 0m0.003s
done
Important Note: The builds that show *regression* were all using
SpiderMonkey 1.7.0. The tests above were 1.8.5. I think that's
significant. Can other people who saw regressions report their
spidermonkey version?
B.
On 29 February 2012 17:25, Benoit Chesneau wrote:
> On Wed, Feb 29, 2012 at 6:07 PM, Benoit Chesneau wrote:
>> On Wed, Feb 29, 2012 at 5:53 PM, Benoit Chesneau wrote:
>>> On Wed, Feb 29, 2012 at 4:03 PM, Robert Newson wrote:
I've produced a new script that reproduces the view regression. I
apologize in advance for exposing my awful Bash scripting abilities
(also my inability to write "pure" shell. Your "Bashism" is my "it
works").
I get 0m56.521s for 1.1.x and 1m17.108s for 1.2.x. That is, 1.1.x
complete the same task in only 72% of the time that 1.2.x takes.
http://friendpaste.com/UI1OcECLEzR6i4D75LqQy
>>>
>>> hrm with the same script:
>>>
>>> 1.2.0 :
>>> real 0m23.842s
>>> user 0m0.007s
>>> sys 0m0.004s
>>>
>>> 1.1.1:
>>> real 0m28.625s
>>> user 0m0.008s
>>> sys 0m0.005s
>>>
>>> hw specs :
>>> $ uname -a
>>> Darwin enki.local 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12
>>> 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64
>>> mba 2011: i7 4 GB SSD 256 GB
>>>
>>>
>>> - benoît
>>
>> Just for my curiosity I tested it in the couchdb based distribution I
>> maintain (head, last mochiweb + jiffy) :
>>
>>
>> real 0m19.539s
>> user 0m0.007s
>> sys 0m0.005s
>> done
>
> I launched the tests a second time and same results:
>
> 1.1.1 :
> real 0m29.155s
> user 0m0.009s
> sys 0m0.006s
>
> 1.2.0 :
>
> real 0m25.466s
> user 0m0.008s
> sys 0m0.005s
Re: Please report your indexing speed
On Wed, Feb 29, 2012 at 6:07 PM, Benoit Chesneau wrote: > On Wed, Feb 29, 2012 at 5:53 PM, Benoit Chesneau wrote: >> On Wed, Feb 29, 2012 at 4:03 PM, Robert Newson wrote: >>> I've produced a new script that reproduces the view regression. I >>> apologize in advance for exposing my awful Bash scripting abilities >>> (also my inability to write "pure" shell. Your "Bashism" is my "it >>> works"). >>> >>> I get 0m56.521s for 1.1.x and 1m17.108s for 1.2.x. That is, 1.1.x >>> complete the same task in only 72% of the time that 1.2.x takes. >>> >>> http://friendpaste.com/UI1OcECLEzR6i4D75LqQy >>> >> >> hrm with the same script: >> >> 1.2.0 : >> real 0m23.842s >> user 0m0.007s >> sys 0m0.004s >> >> 1.1.1: >> real 0m28.625s >> user 0m0.008s >> sys 0m0.005s >> >> hw specs : >> $ uname -a >> Darwin enki.local 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12 >> 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64 >> mba 2011: i7 4 GB SSD 256 GB >> >> >> - benoît > > Just for my curiosity I tested it in the couchdb based distribution I > maintain (head, last mochiweb + jiffy) : > > > real 0m19.539s > user 0m0.007s > sys 0m0.005s > done I launched the tests a second time and same results: 1.1.1 : real0m29.155s user0m0.009s sys 0m0.006s 1.2.0 : real0m25.466s user0m0.008s sys 0m0.005s
Re: Please report your indexing speed
Mac OS X 10.7.3, 2.6Ghz i7 (2x2 cores), R14B04 SSD: 1.1.1: 0m31.337s 1.2.x: 0m21.229s Spinning disk, 5400rpm: 1.1.1: 0m32.807s 1.2.x: 0m21.172s Cheers Jan -- On Feb 29, 2012, at 16:03 , Robert Newson wrote: > I've produced a new script that reproduces the view regression. I > apologize in advance for exposing my awful Bash scripting abilities > (also my inability to write "pure" shell. Your "Bashism" is my "it > works"). > > I get 0m56.521s for 1.1.x and 1m17.108s for 1.2.x. That is, 1.1.x > complete the same task in only 72% of the time that 1.2.x takes. > > http://friendpaste.com/UI1OcECLEzR6i4D75LqQy > > It would be very useful if others could reproduce (or contradict!) this > finding. > > B. > > On 29 February 2012 00:56, Jan Lehnardt wrote: >> One more report. >> >> I got suspicious of the rather short runtimes, so I picked the >> default_doc and ran it at 500k: >> >> bench_R14B04_1.1.1_default_doc.tpl.log 2m19.139s >> bench_R14B04_1.2.x_default_doc.tpl.log 2m18.875s >> >> It seems to me that we need more variation in what we test, >> more OSs, larger ddocs, like the one Stefan linked to. Can >> anyone help providing this? >> >> Cheers >> Jan >> -- >> >> >> On Feb 29, 2012, at 00:23 , Jan Lehnardt wrote: >> >>> For Robert Newson, avoiding bulk inserts to populate the dbs: >>> >>> bench_R14B04_1.1.1_default_doc.tpl.log 0m19.692s >>> bench_R14B04_1.2.x_default_doc.tpl.log 0m17.033s >>> >>> bench_R14B04_1.1.1_nested_6k.tpl.log 1m31.393s >>> bench_R14B04_1.2.x_nested_6k.tpl.log 0m42.010s >>> >>> bench_R14B04_1.1.1_small_doc.tpl.log 0m8.103s >>> bench_R14B04_1.2.x_small_doc.tpl.log 0m10.597s >>> >>> bench_R14B04_1.1.1_wow.tpl.log 0m33.944s >>> bench_R14B04_1.2.x_wow.tpl.log 0m27.087s >>> >>> (Just R14B04, full logs available on demand) >>> >>> Cheers >>> Jan >>> -- >>> >>> >>> On Feb 28, 2012, at 23:09 , Jan Lehnardt wrote: >>> Same story, but spinning disk, 5400rpm: bench_R14B04_1.1.1_default_doc.tpl.log 0m19.175s bench_R14B04_1.2.x_default_doc.tpl.log 0m16.821s bench_R15B_1.1.1_default_doc.tpl.log 0m13.050s bench_R15B_1.2.x_default_doc.tpl.log 0m13.292s bench_R14B04_1.1.1_nested_6k.tpl.log 1m26.941s bench_R14B04_1.2.x_nested_6k.tpl.log 0m39.178s bench_R15B_1.1.1_nested_6k.tpl.log 0m47.766s bench_R15B_1.2.x_nested_6k.tpl.log 0m31.697s bench_R14B04_1.1.1_small_doc.tpl.log 1m19.851s bench_R14B04_1.2.x_small_doc.tpl.log 1m43.057s bench_R15B_1.1.1_small_doc.tpl.log 0m52.249s bench_R15B_1.2.x_small_doc.tpl.log 1m8.195s bench_R14B04_1.1.1_wow.tpl.log 0m29.589s bench_R14B04_1.2.x_wow.tpl.log 0m24.867s bench_R15B_1.1.1_wow.tpl.log 0m20.171s bench_R15B_1.2.x_wow.tpl.log 0m18.800s Full logs at http://jan.prima.de/slow_couch/rust/ Cheers Jan -- On Feb 28, 2012, at 21:22 , Jan Lehnardt wrote: > > # tl;dr: > > bench_R14B04_1.1.1_default_doc.tpl.log 0m18.749s > bench_R14B04_1.2.x_default_doc.tpl.log 0m16.304s > bench_R15B_1.1.1_default_doc.tpl.log 0m12.946s > bench_R15B_1.2.x_default_doc.tpl.log 0m13.616s > > bench_R14B04_1.1.1_nested_6k.tpl.log 1m27.267s > bench_R14B04_1.2.x_nested_6k.tpl.log 0m37.910s > bench_R15B_1.1.1_nested_6k.tpl.log 0m46.963s > bench_R15B_1.2.x_nested_6k.tpl.log 0m33.011s > > bench_R14B04_1.1.1_small_doc.tpl.log 1m17.212s > bench_R14B04_1.2.x_small_doc.tpl.log 1m41.383s > bench_R15B_1.1.1_small_doc.tpl.log 0m52.858s > bench_R15B_1.2.x_small_doc.tpl.log 1m9.043s > > bench_R14B04_1.1.1_wow.tpl.log 0m29.842s > bench_R14B04_1.2.x_wow.tpl.log 0m24.178s > bench_R15B_1.1.1_wow.tpl.log 0m20.493s > bench_R15B_1.2.x_wow.tpl.log 0m19.584s > > (Full logs at [5]) > > > # Description > > All of these are on Mac OS X 10.7.3 on an SSD. > > I'll be running the same set on spinning disk and then Robert N asked > me to populate the DBs not using builk docs. Since that's gonna take > a while, I'll probably run this overnight. > > All of the results are generated by my fork of Jason's slow_couchdb[1] > and Filipe's seatoncouch[2]. > > The changes I've made is have the small_doc test run with 500k instead > of 50k docs, added .view files to match the tpl files in > seatoncouch/templates/* so we can have similar views use the different > doc structures. > > I also added two scripts to orchestrate the above testing in a more > automated fashion. It also allows you to run the full matrix yourself. > All you need is set up ho
Re: Please report your indexing speed
On Wed, Feb 29, 2012 at 5:53 PM, Benoit Chesneau wrote: > On Wed, Feb 29, 2012 at 4:03 PM, Robert Newson wrote: >> I've produced a new script that reproduces the view regression. I >> apologize in advance for exposing my awful Bash scripting abilities >> (also my inability to write "pure" shell. Your "Bashism" is my "it >> works"). >> >> I get 0m56.521s for 1.1.x and 1m17.108s for 1.2.x. That is, 1.1.x >> complete the same task in only 72% of the time that 1.2.x takes. >> >> http://friendpaste.com/UI1OcECLEzR6i4D75LqQy >> > > hrm with the same script: > > 1.2.0 : > real 0m23.842s > user 0m0.007s > sys 0m0.004s > > 1.1.1: > real 0m28.625s > user 0m0.008s > sys 0m0.005s > > hw specs : > $ uname -a > Darwin enki.local 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12 > 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64 > mba 2011: i7 4 GB SSD 256 GB > > > - benoît Just for my curiosity I tested it in the couchdb based distribution I maintain (head, last mochiweb + jiffy) : real0m19.539s user0m0.007s sys 0m0.005s done
Re: Please report your indexing speed
That same time but for 10x doc count; 1.1.x: 10m3.734s 1.2.x: 14m17.877s My machine is a Macbook Air 1.7 GHz Intel Core i5 with 4 GB 1333 MHz DDR3 running OS X 10.7.3. On 29 February 2012 15:03, Robert Newson wrote: > I've produced a new script that reproduces the view regression. I > apologize in advance for exposing my awful Bash scripting abilities > (also my inability to write "pure" shell. Your "Bashism" is my "it > works"). > > I get 0m56.521s for 1.1.x and 1m17.108s for 1.2.x. That is, 1.1.x > complete the same task in only 72% of the time that 1.2.x takes. > > http://friendpaste.com/UI1OcECLEzR6i4D75LqQy > > It would be very useful if others could reproduce (or contradict!) this > finding. > > B. > > On 29 February 2012 00:56, Jan Lehnardt wrote: >> One more report. >> >> I got suspicious of the rather short runtimes, so I picked the >> default_doc and ran it at 500k: >> >> bench_R14B04_1.1.1_default_doc.tpl.log 2m19.139s >> bench_R14B04_1.2.x_default_doc.tpl.log 2m18.875s >> >> It seems to me that we need more variation in what we test, >> more OSs, larger ddocs, like the one Stefan linked to. Can >> anyone help providing this? >> >> Cheers >> Jan >> -- >> >> >> On Feb 29, 2012, at 00:23 , Jan Lehnardt wrote: >> >>> For Robert Newson, avoiding bulk inserts to populate the dbs: >>> >>> bench_R14B04_1.1.1_default_doc.tpl.log 0m19.692s >>> bench_R14B04_1.2.x_default_doc.tpl.log 0m17.033s >>> >>> bench_R14B04_1.1.1_nested_6k.tpl.log 1m31.393s >>> bench_R14B04_1.2.x_nested_6k.tpl.log 0m42.010s >>> >>> bench_R14B04_1.1.1_small_doc.tpl.log 0m8.103s >>> bench_R14B04_1.2.x_small_doc.tpl.log 0m10.597s >>> >>> bench_R14B04_1.1.1_wow.tpl.log 0m33.944s >>> bench_R14B04_1.2.x_wow.tpl.log 0m27.087s >>> >>> (Just R14B04, full logs available on demand) >>> >>> Cheers >>> Jan >>> -- >>> >>> >>> On Feb 28, 2012, at 23:09 , Jan Lehnardt wrote: >>> Same story, but spinning disk, 5400rpm: bench_R14B04_1.1.1_default_doc.tpl.log 0m19.175s bench_R14B04_1.2.x_default_doc.tpl.log 0m16.821s bench_R15B_1.1.1_default_doc.tpl.log 0m13.050s bench_R15B_1.2.x_default_doc.tpl.log 0m13.292s bench_R14B04_1.1.1_nested_6k.tpl.log 1m26.941s bench_R14B04_1.2.x_nested_6k.tpl.log 0m39.178s bench_R15B_1.1.1_nested_6k.tpl.log 0m47.766s bench_R15B_1.2.x_nested_6k.tpl.log 0m31.697s bench_R14B04_1.1.1_small_doc.tpl.log 1m19.851s bench_R14B04_1.2.x_small_doc.tpl.log 1m43.057s bench_R15B_1.1.1_small_doc.tpl.log 0m52.249s bench_R15B_1.2.x_small_doc.tpl.log 1m8.195s bench_R14B04_1.1.1_wow.tpl.log 0m29.589s bench_R14B04_1.2.x_wow.tpl.log 0m24.867s bench_R15B_1.1.1_wow.tpl.log 0m20.171s bench_R15B_1.2.x_wow.tpl.log 0m18.800s Full logs at http://jan.prima.de/slow_couch/rust/ Cheers Jan -- On Feb 28, 2012, at 21:22 , Jan Lehnardt wrote: > > # tl;dr: > > bench_R14B04_1.1.1_default_doc.tpl.log 0m18.749s > bench_R14B04_1.2.x_default_doc.tpl.log 0m16.304s > bench_R15B_1.1.1_default_doc.tpl.log 0m12.946s > bench_R15B_1.2.x_default_doc.tpl.log 0m13.616s > > bench_R14B04_1.1.1_nested_6k.tpl.log 1m27.267s > bench_R14B04_1.2.x_nested_6k.tpl.log 0m37.910s > bench_R15B_1.1.1_nested_6k.tpl.log 0m46.963s > bench_R15B_1.2.x_nested_6k.tpl.log 0m33.011s > > bench_R14B04_1.1.1_small_doc.tpl.log 1m17.212s > bench_R14B04_1.2.x_small_doc.tpl.log 1m41.383s > bench_R15B_1.1.1_small_doc.tpl.log 0m52.858s > bench_R15B_1.2.x_small_doc.tpl.log 1m9.043s > > bench_R14B04_1.1.1_wow.tpl.log 0m29.842s > bench_R14B04_1.2.x_wow.tpl.log 0m24.178s > bench_R15B_1.1.1_wow.tpl.log 0m20.493s > bench_R15B_1.2.x_wow.tpl.log 0m19.584s > > (Full logs at [5]) > > > # Description > > All of these are on Mac OS X 10.7.3 on an SSD. > > I'll be running the same set on spinning disk and then Robert N asked > me to populate the DBs not using builk docs. Since that's gonna take > a while, I'll probably run this overnight. > > All of the results are generated by my fork of Jason's slow_couchdb[1] > and Filipe's seatoncouch[2]. > > The changes I've made is have the small_doc test run with 500k instead > of 50k docs, added .view files to match the tpl files in > seatoncouch/templates/* so we can have similar views use the different > doc structures. > > I also added two scripts to orchestrate the above testing in a more > automated fashion. It also allows you to run the full matrix yourself. > All you need is set up homebrew allow `brew switch erlang R14B04
Re: Please report your indexing speed
On Wed, Feb 29, 2012 at 4:03 PM, Robert Newson wrote: > I've produced a new script that reproduces the view regression. I > apologize in advance for exposing my awful Bash scripting abilities > (also my inability to write "pure" shell. Your "Bashism" is my "it > works"). > > I get 0m56.521s for 1.1.x and 1m17.108s for 1.2.x. That is, 1.1.x > complete the same task in only 72% of the time that 1.2.x takes. > > http://friendpaste.com/UI1OcECLEzR6i4D75LqQy > hrm with the same script: 1.2.0 : real0m23.842s user0m0.007s sys 0m0.004s 1.1.1: real0m28.625s user0m0.008s sys 0m0.005s hw specs : $ uname -a Darwin enki.local 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64 mba 2011: i7 4 GB SSD 256 GB - benoît
Re: Please report your indexing speed
I've produced a new script that reproduces the view regression. I apologize in advance for exposing my awful Bash scripting abilities (also my inability to write "pure" shell. Your "Bashism" is my "it works"). I get 0m56.521s for 1.1.x and 1m17.108s for 1.2.x. That is, 1.1.x complete the same task in only 72% of the time that 1.2.x takes. http://friendpaste.com/UI1OcECLEzR6i4D75LqQy It would be very useful if others could reproduce (or contradict!) this finding. B. On 29 February 2012 00:56, Jan Lehnardt wrote: > One more report. > > I got suspicious of the rather short runtimes, so I picked the > default_doc and ran it at 500k: > > bench_R14B04_1.1.1_default_doc.tpl.log 2m19.139s > bench_R14B04_1.2.x_default_doc.tpl.log 2m18.875s > > It seems to me that we need more variation in what we test, > more OSs, larger ddocs, like the one Stefan linked to. Can > anyone help providing this? > > Cheers > Jan > -- > > > On Feb 29, 2012, at 00:23 , Jan Lehnardt wrote: > >> For Robert Newson, avoiding bulk inserts to populate the dbs: >> >> bench_R14B04_1.1.1_default_doc.tpl.log 0m19.692s >> bench_R14B04_1.2.x_default_doc.tpl.log 0m17.033s >> >> bench_R14B04_1.1.1_nested_6k.tpl.log 1m31.393s >> bench_R14B04_1.2.x_nested_6k.tpl.log 0m42.010s >> >> bench_R14B04_1.1.1_small_doc.tpl.log 0m8.103s >> bench_R14B04_1.2.x_small_doc.tpl.log 0m10.597s >> >> bench_R14B04_1.1.1_wow.tpl.log 0m33.944s >> bench_R14B04_1.2.x_wow.tpl.log 0m27.087s >> >> (Just R14B04, full logs available on demand) >> >> Cheers >> Jan >> -- >> >> >> On Feb 28, 2012, at 23:09 , Jan Lehnardt wrote: >> >>> Same story, but spinning disk, 5400rpm: >>> >>> bench_R14B04_1.1.1_default_doc.tpl.log 0m19.175s >>> bench_R14B04_1.2.x_default_doc.tpl.log 0m16.821s >>> bench_R15B_1.1.1_default_doc.tpl.log 0m13.050s >>> bench_R15B_1.2.x_default_doc.tpl.log 0m13.292s >>> >>> bench_R14B04_1.1.1_nested_6k.tpl.log 1m26.941s >>> bench_R14B04_1.2.x_nested_6k.tpl.log 0m39.178s >>> bench_R15B_1.1.1_nested_6k.tpl.log 0m47.766s >>> bench_R15B_1.2.x_nested_6k.tpl.log 0m31.697s >>> >>> bench_R14B04_1.1.1_small_doc.tpl.log 1m19.851s >>> bench_R14B04_1.2.x_small_doc.tpl.log 1m43.057s >>> bench_R15B_1.1.1_small_doc.tpl.log 0m52.249s >>> bench_R15B_1.2.x_small_doc.tpl.log 1m8.195s >>> >>> bench_R14B04_1.1.1_wow.tpl.log 0m29.589s >>> bench_R14B04_1.2.x_wow.tpl.log 0m24.867s >>> bench_R15B_1.1.1_wow.tpl.log 0m20.171s >>> bench_R15B_1.2.x_wow.tpl.log 0m18.800s >>> >>> Full logs at http://jan.prima.de/slow_couch/rust/ >>> >>> Cheers >>> Jan >>> -- >>> >>> >>> On Feb 28, 2012, at 21:22 , Jan Lehnardt wrote: >>> # tl;dr: bench_R14B04_1.1.1_default_doc.tpl.log 0m18.749s bench_R14B04_1.2.x_default_doc.tpl.log 0m16.304s bench_R15B_1.1.1_default_doc.tpl.log 0m12.946s bench_R15B_1.2.x_default_doc.tpl.log 0m13.616s bench_R14B04_1.1.1_nested_6k.tpl.log 1m27.267s bench_R14B04_1.2.x_nested_6k.tpl.log 0m37.910s bench_R15B_1.1.1_nested_6k.tpl.log 0m46.963s bench_R15B_1.2.x_nested_6k.tpl.log 0m33.011s bench_R14B04_1.1.1_small_doc.tpl.log 1m17.212s bench_R14B04_1.2.x_small_doc.tpl.log 1m41.383s bench_R15B_1.1.1_small_doc.tpl.log 0m52.858s bench_R15B_1.2.x_small_doc.tpl.log 1m9.043s bench_R14B04_1.1.1_wow.tpl.log 0m29.842s bench_R14B04_1.2.x_wow.tpl.log 0m24.178s bench_R15B_1.1.1_wow.tpl.log 0m20.493s bench_R15B_1.2.x_wow.tpl.log 0m19.584s (Full logs at [5]) # Description All of these are on Mac OS X 10.7.3 on an SSD. I'll be running the same set on spinning disk and then Robert N asked me to populate the DBs not using builk docs. Since that's gonna take a while, I'll probably run this overnight. All of the results are generated by my fork of Jason's slow_couchdb[1] and Filipe's seatoncouch[2]. The changes I've made is have the small_doc test run with 500k instead of 50k docs, added .view files to match the tpl files in seatoncouch/templates/* so we can have similar views use the different doc structures. I also added two scripts to orchestrate the above testing in a more automated fashion. It also allows you to run the full matrix yourself. All you need is set up homebrew allow `brew switch erlang R14B04` and R15B (which is controlled in matrix.sh[3]) and have a git checkout of the CouchDB sources that allow you to do `git checkout 1.1.1` or `1.2.x` (which is controlled in runner.sh[4], adjust the path to the git checkout there as well). matrix.sh also allows you to specify which docs to run. Please shout if you need any mo
Re: Please report your indexing speed
One more report. I got suspicious of the rather short runtimes, so I picked the default_doc and ran it at 500k: bench_R14B04_1.1.1_default_doc.tpl.log 2m19.139s bench_R14B04_1.2.x_default_doc.tpl.log 2m18.875s It seems to me that we need more variation in what we test, more OSs, larger ddocs, like the one Stefan linked to. Can anyone help providing this? Cheers Jan -- On Feb 29, 2012, at 00:23 , Jan Lehnardt wrote: > For Robert Newson, avoiding bulk inserts to populate the dbs: > > bench_R14B04_1.1.1_default_doc.tpl.log 0m19.692s > bench_R14B04_1.2.x_default_doc.tpl.log 0m17.033s > > bench_R14B04_1.1.1_nested_6k.tpl.log 1m31.393s > bench_R14B04_1.2.x_nested_6k.tpl.log 0m42.010s > > bench_R14B04_1.1.1_small_doc.tpl.log 0m8.103s > bench_R14B04_1.2.x_small_doc.tpl.log 0m10.597s > > bench_R14B04_1.1.1_wow.tpl.log 0m33.944s > bench_R14B04_1.2.x_wow.tpl.log 0m27.087s > > (Just R14B04, full logs available on demand) > > Cheers > Jan > -- > > > On Feb 28, 2012, at 23:09 , Jan Lehnardt wrote: > >> Same story, but spinning disk, 5400rpm: >> >> bench_R14B04_1.1.1_default_doc.tpl.log 0m19.175s >> bench_R14B04_1.2.x_default_doc.tpl.log 0m16.821s >> bench_R15B_1.1.1_default_doc.tpl.log 0m13.050s >> bench_R15B_1.2.x_default_doc.tpl.log 0m13.292s >> >> bench_R14B04_1.1.1_nested_6k.tpl.log 1m26.941s >> bench_R14B04_1.2.x_nested_6k.tpl.log 0m39.178s >> bench_R15B_1.1.1_nested_6k.tpl.log 0m47.766s >> bench_R15B_1.2.x_nested_6k.tpl.log 0m31.697s >> >> bench_R14B04_1.1.1_small_doc.tpl.log 1m19.851s >> bench_R14B04_1.2.x_small_doc.tpl.log 1m43.057s >> bench_R15B_1.1.1_small_doc.tpl.log 0m52.249s >> bench_R15B_1.2.x_small_doc.tpl.log 1m8.195s >> >> bench_R14B04_1.1.1_wow.tpl.log 0m29.589s >> bench_R14B04_1.2.x_wow.tpl.log 0m24.867s >> bench_R15B_1.1.1_wow.tpl.log 0m20.171s >> bench_R15B_1.2.x_wow.tpl.log 0m18.800s >> >> Full logs at http://jan.prima.de/slow_couch/rust/ >> >> Cheers >> Jan >> -- >> >> >> On Feb 28, 2012, at 21:22 , Jan Lehnardt wrote: >> >>> >>> # tl;dr: >>> >>> bench_R14B04_1.1.1_default_doc.tpl.log 0m18.749s >>> bench_R14B04_1.2.x_default_doc.tpl.log 0m16.304s >>> bench_R15B_1.1.1_default_doc.tpl.log 0m12.946s >>> bench_R15B_1.2.x_default_doc.tpl.log 0m13.616s >>> >>> bench_R14B04_1.1.1_nested_6k.tpl.log 1m27.267s >>> bench_R14B04_1.2.x_nested_6k.tpl.log 0m37.910s >>> bench_R15B_1.1.1_nested_6k.tpl.log 0m46.963s >>> bench_R15B_1.2.x_nested_6k.tpl.log 0m33.011s >>> >>> bench_R14B04_1.1.1_small_doc.tpl.log 1m17.212s >>> bench_R14B04_1.2.x_small_doc.tpl.log 1m41.383s >>> bench_R15B_1.1.1_small_doc.tpl.log 0m52.858s >>> bench_R15B_1.2.x_small_doc.tpl.log 1m9.043s >>> >>> bench_R14B04_1.1.1_wow.tpl.log 0m29.842s >>> bench_R14B04_1.2.x_wow.tpl.log 0m24.178s >>> bench_R15B_1.1.1_wow.tpl.log 0m20.493s >>> bench_R15B_1.2.x_wow.tpl.log 0m19.584s >>> >>> (Full logs at [5]) >>> >>> >>> # Description >>> >>> All of these are on Mac OS X 10.7.3 on an SSD. >>> >>> I'll be running the same set on spinning disk and then Robert N asked >>> me to populate the DBs not using builk docs. Since that's gonna take >>> a while, I'll probably run this overnight. >>> >>> All of the results are generated by my fork of Jason's slow_couchdb[1] >>> and Filipe's seatoncouch[2]. >>> >>> The changes I've made is have the small_doc test run with 500k instead >>> of 50k docs, added .view files to match the tpl files in >>> seatoncouch/templates/* so we can have similar views use the different >>> doc structures. >>> >>> I also added two scripts to orchestrate the above testing in a more >>> automated fashion. It also allows you to run the full matrix yourself. >>> All you need is set up homebrew allow `brew switch erlang R14B04` and >>> R15B (which is controlled in matrix.sh[3]) and have a git checkout of the >>> CouchDB sources that allow you to do `git checkout 1.1.1` or `1.2.x` >>> (which is controlled in runner.sh[4], adjust the path to the git checkout >>> there as well). >>> >>> matrix.sh also allows you to specify which docs to run. >>> >>> Please shout if you need any more info about this test run or how to >>> run this yourself. >>> >>> >>> # Analysis >>> >>> Inconclusive, I'l like to run this on larger dbs in general to see if >>> there are more differences that shake out and I've yet have to run this >>> on a spinning disk let alone another OS* or more complex view functions >>> or larger design docs (like the one Stefan had). >>> >>> * It shouldn't be too much work to port slow_couchdb to other OSs, I'll >>> definitely be looking into that, but we can do with every bit of help :) >>> >>> So far, I'm happy to conclude that while there are definitely provable >>> differences,
Re: Please report your indexing speed
For Robert Newson, avoiding bulk inserts to populate the dbs: bench_R14B04_1.1.1_default_doc.tpl.log 0m19.692s bench_R14B04_1.2.x_default_doc.tpl.log 0m17.033s bench_R14B04_1.1.1_nested_6k.tpl.log 1m31.393s bench_R14B04_1.2.x_nested_6k.tpl.log 0m42.010s bench_R14B04_1.1.1_small_doc.tpl.log 0m8.103s bench_R14B04_1.2.x_small_doc.tpl.log 0m10.597s bench_R14B04_1.1.1_wow.tpl.log 0m33.944s bench_R14B04_1.2.x_wow.tpl.log 0m27.087s (Just R14B04, full logs available on demand) Cheers Jan -- On Feb 28, 2012, at 23:09 , Jan Lehnardt wrote: > Same story, but spinning disk, 5400rpm: > > bench_R14B04_1.1.1_default_doc.tpl.log 0m19.175s > bench_R14B04_1.2.x_default_doc.tpl.log 0m16.821s > bench_R15B_1.1.1_default_doc.tpl.log 0m13.050s > bench_R15B_1.2.x_default_doc.tpl.log 0m13.292s > > bench_R14B04_1.1.1_nested_6k.tpl.log 1m26.941s > bench_R14B04_1.2.x_nested_6k.tpl.log 0m39.178s > bench_R15B_1.1.1_nested_6k.tpl.log 0m47.766s > bench_R15B_1.2.x_nested_6k.tpl.log 0m31.697s > > bench_R14B04_1.1.1_small_doc.tpl.log 1m19.851s > bench_R14B04_1.2.x_small_doc.tpl.log 1m43.057s > bench_R15B_1.1.1_small_doc.tpl.log 0m52.249s > bench_R15B_1.2.x_small_doc.tpl.log 1m8.195s > > bench_R14B04_1.1.1_wow.tpl.log 0m29.589s > bench_R14B04_1.2.x_wow.tpl.log 0m24.867s > bench_R15B_1.1.1_wow.tpl.log 0m20.171s > bench_R15B_1.2.x_wow.tpl.log 0m18.800s > > Full logs at http://jan.prima.de/slow_couch/rust/ > > Cheers > Jan > -- > > > On Feb 28, 2012, at 21:22 , Jan Lehnardt wrote: > >> >> # tl;dr: >> >> bench_R14B04_1.1.1_default_doc.tpl.log 0m18.749s >> bench_R14B04_1.2.x_default_doc.tpl.log 0m16.304s >> bench_R15B_1.1.1_default_doc.tpl.log 0m12.946s >> bench_R15B_1.2.x_default_doc.tpl.log 0m13.616s >> >> bench_R14B04_1.1.1_nested_6k.tpl.log 1m27.267s >> bench_R14B04_1.2.x_nested_6k.tpl.log 0m37.910s >> bench_R15B_1.1.1_nested_6k.tpl.log 0m46.963s >> bench_R15B_1.2.x_nested_6k.tpl.log 0m33.011s >> >> bench_R14B04_1.1.1_small_doc.tpl.log 1m17.212s >> bench_R14B04_1.2.x_small_doc.tpl.log 1m41.383s >> bench_R15B_1.1.1_small_doc.tpl.log 0m52.858s >> bench_R15B_1.2.x_small_doc.tpl.log 1m9.043s >> >> bench_R14B04_1.1.1_wow.tpl.log 0m29.842s >> bench_R14B04_1.2.x_wow.tpl.log 0m24.178s >> bench_R15B_1.1.1_wow.tpl.log 0m20.493s >> bench_R15B_1.2.x_wow.tpl.log 0m19.584s >> >> (Full logs at [5]) >> >> >> # Description >> >> All of these are on Mac OS X 10.7.3 on an SSD. >> >> I'll be running the same set on spinning disk and then Robert N asked >> me to populate the DBs not using builk docs. Since that's gonna take >> a while, I'll probably run this overnight. >> >> All of the results are generated by my fork of Jason's slow_couchdb[1] >> and Filipe's seatoncouch[2]. >> >> The changes I've made is have the small_doc test run with 500k instead >> of 50k docs, added .view files to match the tpl files in >> seatoncouch/templates/* so we can have similar views use the different >> doc structures. >> >> I also added two scripts to orchestrate the above testing in a more >> automated fashion. It also allows you to run the full matrix yourself. >> All you need is set up homebrew allow `brew switch erlang R14B04` and >> R15B (which is controlled in matrix.sh[3]) and have a git checkout of the >> CouchDB sources that allow you to do `git checkout 1.1.1` or `1.2.x` >> (which is controlled in runner.sh[4], adjust the path to the git checkout >> there as well). >> >> matrix.sh also allows you to specify which docs to run. >> >> Please shout if you need any more info about this test run or how to >> run this yourself. >> >> >> # Analysis >> >> Inconclusive, I'l like to run this on larger dbs in general to see if >> there are more differences that shake out and I've yet have to run this >> on a spinning disk let alone another OS* or more complex view functions >> or larger design docs (like the one Stefan had). >> >> * It shouldn't be too much work to port slow_couchdb to other OSs, I'll >> definitely be looking into that, but we can do with every bit of help :) >> >> So far, I'm happy to conclude that while there are definitely provable >> differences, that we can live with them. >> >> Cheers >> Jan >> -- >> >> >> [1]: https://github.com/janl/slow_couchdb >> [2]: https://github.com/janl/seatoncouch >> [3]: https://github.com/janl/slow_couchdb/blob/master/matrix.sh >> [4]: https://github.com/janl/slow_couchdb/blob/master/runner.sh >> [5]: http://jan.prima.de/slow_couch/ssd/ >> >> >> On Feb 28, 2012, at 18:53 , Filipe David Manana wrote: >> >>> Jason, repeated my last test with the 1Kb docs ( >>> https://gist.github.com/1930804, map function >>> http://friendpaste.com/5C99aqXocN6N6H1BAYIigs ) to cover branch 1.1.x >>
Re: Please report your indexing speed
Same story, but spinning disk, 5400rpm:
bench_R14B04_1.1.1_default_doc.tpl.log 0m19.175s
bench_R14B04_1.2.x_default_doc.tpl.log 0m16.821s
bench_R15B_1.1.1_default_doc.tpl.log 0m13.050s
bench_R15B_1.2.x_default_doc.tpl.log 0m13.292s
bench_R14B04_1.1.1_nested_6k.tpl.log 1m26.941s
bench_R14B04_1.2.x_nested_6k.tpl.log 0m39.178s
bench_R15B_1.1.1_nested_6k.tpl.log 0m47.766s
bench_R15B_1.2.x_nested_6k.tpl.log 0m31.697s
bench_R14B04_1.1.1_small_doc.tpl.log 1m19.851s
bench_R14B04_1.2.x_small_doc.tpl.log 1m43.057s
bench_R15B_1.1.1_small_doc.tpl.log 0m52.249s
bench_R15B_1.2.x_small_doc.tpl.log 1m8.195s
bench_R14B04_1.1.1_wow.tpl.log 0m29.589s
bench_R14B04_1.2.x_wow.tpl.log 0m24.867s
bench_R15B_1.1.1_wow.tpl.log 0m20.171s
bench_R15B_1.2.x_wow.tpl.log 0m18.800s
Full logs at http://jan.prima.de/slow_couch/rust/
Cheers
Jan
--
On Feb 28, 2012, at 21:22 , Jan Lehnardt wrote:
>
> # tl;dr:
>
> bench_R14B04_1.1.1_default_doc.tpl.log 0m18.749s
> bench_R14B04_1.2.x_default_doc.tpl.log 0m16.304s
> bench_R15B_1.1.1_default_doc.tpl.log 0m12.946s
> bench_R15B_1.2.x_default_doc.tpl.log 0m13.616s
>
> bench_R14B04_1.1.1_nested_6k.tpl.log 1m27.267s
> bench_R14B04_1.2.x_nested_6k.tpl.log 0m37.910s
> bench_R15B_1.1.1_nested_6k.tpl.log 0m46.963s
> bench_R15B_1.2.x_nested_6k.tpl.log 0m33.011s
>
> bench_R14B04_1.1.1_small_doc.tpl.log 1m17.212s
> bench_R14B04_1.2.x_small_doc.tpl.log 1m41.383s
> bench_R15B_1.1.1_small_doc.tpl.log 0m52.858s
> bench_R15B_1.2.x_small_doc.tpl.log 1m9.043s
>
> bench_R14B04_1.1.1_wow.tpl.log 0m29.842s
> bench_R14B04_1.2.x_wow.tpl.log 0m24.178s
> bench_R15B_1.1.1_wow.tpl.log 0m20.493s
> bench_R15B_1.2.x_wow.tpl.log 0m19.584s
>
> (Full logs at [5])
>
>
> # Description
>
> All of these are on Mac OS X 10.7.3 on an SSD.
>
> I'll be running the same set on spinning disk and then Robert N asked
> me to populate the DBs not using builk docs. Since that's gonna take
> a while, I'll probably run this overnight.
>
> All of the results are generated by my fork of Jason's slow_couchdb[1]
> and Filipe's seatoncouch[2].
>
> The changes I've made is have the small_doc test run with 500k instead
> of 50k docs, added .view files to match the tpl files in
> seatoncouch/templates/* so we can have similar views use the different
> doc structures.
>
> I also added two scripts to orchestrate the above testing in a more
> automated fashion. It also allows you to run the full matrix yourself.
> All you need is set up homebrew allow `brew switch erlang R14B04` and
> R15B (which is controlled in matrix.sh[3]) and have a git checkout of the
> CouchDB sources that allow you to do `git checkout 1.1.1` or `1.2.x`
> (which is controlled in runner.sh[4], adjust the path to the git checkout
> there as well).
>
> matrix.sh also allows you to specify which docs to run.
>
> Please shout if you need any more info about this test run or how to
> run this yourself.
>
>
> # Analysis
>
> Inconclusive, I'l like to run this on larger dbs in general to see if
> there are more differences that shake out and I've yet have to run this
> on a spinning disk let alone another OS* or more complex view functions
> or larger design docs (like the one Stefan had).
>
> * It shouldn't be too much work to port slow_couchdb to other OSs, I'll
> definitely be looking into that, but we can do with every bit of help :)
>
> So far, I'm happy to conclude that while there are definitely provable
> differences, that we can live with them.
>
> Cheers
> Jan
> --
>
>
> [1]: https://github.com/janl/slow_couchdb
> [2]: https://github.com/janl/seatoncouch
> [3]: https://github.com/janl/slow_couchdb/blob/master/matrix.sh
> [4]: https://github.com/janl/slow_couchdb/blob/master/runner.sh
> [5]: http://jan.prima.de/slow_couch/ssd/
>
>
> On Feb 28, 2012, at 18:53 , Filipe David Manana wrote:
>
>> Jason, repeated my last test with the 1Kb docs (
>> https://gist.github.com/1930804, map function
>> http://friendpaste.com/5C99aqXocN6N6H1BAYIigs ) to cover branch 1.1.x
>> as well. Here are the full results (also in
>> https://gist.github.com/1930807):
>>
>>
>> Before COUCHDB-1186
>>
>> fdmanana 23:21:05 ~/git/hub/slow_couchdb (master)> docs=50
>> batch=5000 ./bench.sh wow.tpl
>> Server: CouchDB/1.2.0a-a68a792-git (Erlang OTP/R14B03)
>> {"couchdb":"Welcome","version":"1.2.0a-a68a792-git"}
>>
>> [INFO] Created DB named `db1'
>> [INFO] Uploaded 5000 documents via _bulk_docs
>> ()
>> [INFO] Uploaded 5000 documents via _bulk_docs
>> Building view.
>> {"total_rows":50,"offset":0,"rows":[
>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":
Re: Please report your indexing speed
# tl;dr:
bench_R14B04_1.1.1_default_doc.tpl.log 0m18.749s
bench_R14B04_1.2.x_default_doc.tpl.log 0m16.304s
bench_R15B_1.1.1_default_doc.tpl.log 0m12.946s
bench_R15B_1.2.x_default_doc.tpl.log 0m13.616s
bench_R14B04_1.1.1_nested_6k.tpl.log 1m27.267s
bench_R14B04_1.2.x_nested_6k.tpl.log 0m37.910s
bench_R15B_1.1.1_nested_6k.tpl.log 0m46.963s
bench_R15B_1.2.x_nested_6k.tpl.log 0m33.011s
bench_R14B04_1.1.1_small_doc.tpl.log 1m17.212s
bench_R14B04_1.2.x_small_doc.tpl.log 1m41.383s
bench_R15B_1.1.1_small_doc.tpl.log 0m52.858s
bench_R15B_1.2.x_small_doc.tpl.log 1m9.043s
bench_R14B04_1.1.1_wow.tpl.log 0m29.842s
bench_R14B04_1.2.x_wow.tpl.log 0m24.178s
bench_R15B_1.1.1_wow.tpl.log 0m20.493s
bench_R15B_1.2.x_wow.tpl.log 0m19.584s
(Full logs at [5])
# Description
All of these are on Mac OS X 10.7.3 on an SSD.
I'll be running the same set on spinning disk and then Robert N asked
me to populate the DBs not using builk docs. Since that's gonna take
a while, I'll probably run this overnight.
All of the results are generated by my fork of Jason's slow_couchdb[1]
and Filipe's seatoncouch[2].
The changes I've made is have the small_doc test run with 500k instead
of 50k docs, added .view files to match the tpl files in
seatoncouch/templates/* so we can have similar views use the different
doc structures.
I also added two scripts to orchestrate the above testing in a more
automated fashion. It also allows you to run the full matrix yourself.
All you need is set up homebrew allow `brew switch erlang R14B04` and
R15B (which is controlled in matrix.sh[3]) and have a git checkout of the
CouchDB sources that allow you to do `git checkout 1.1.1` or `1.2.x`
(which is controlled in runner.sh[4], adjust the path to the git checkout
there as well).
matrix.sh also allows you to specify which docs to run.
Please shout if you need any more info about this test run or how to
run this yourself.
# Analysis
Inconclusive, I'l like to run this on larger dbs in general to see if
there are more differences that shake out and I've yet have to run this
on a spinning disk let alone another OS* or more complex view functions
or larger design docs (like the one Stefan had).
* It shouldn't be too much work to port slow_couchdb to other OSs, I'll
definitely be looking into that, but we can do with every bit of help :)
So far, I'm happy to conclude that while there are definitely provable
differences, that we can live with them.
Cheers
Jan
--
[1]: https://github.com/janl/slow_couchdb
[2]: https://github.com/janl/seatoncouch
[3]: https://github.com/janl/slow_couchdb/blob/master/matrix.sh
[4]: https://github.com/janl/slow_couchdb/blob/master/runner.sh
[5]: http://jan.prima.de/slow_couch/ssd/
On Feb 28, 2012, at 18:53 , Filipe David Manana wrote:
> Jason, repeated my last test with the 1Kb docs (
> https://gist.github.com/1930804, map function
> http://friendpaste.com/5C99aqXocN6N6H1BAYIigs ) to cover branch 1.1.x
> as well. Here are the full results (also in
> https://gist.github.com/1930807):
>
>
> Before COUCHDB-1186
>
> fdmanana 23:21:05 ~/git/hub/slow_couchdb (master)> docs=50
> batch=5000 ./bench.sh wow.tpl
> Server: CouchDB/1.2.0a-a68a792-git (Erlang OTP/R14B03)
> {"couchdb":"Welcome","version":"1.2.0a-a68a792-git"}
>
> [INFO] Created DB named `db1'
> [INFO] Uploaded 5000 documents via _bulk_docs
> ()
> [INFO] Uploaded 5000 documents via _bulk_docs
> Building view.
> {"total_rows":50,"offset":0,"rows":[
> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
> ]}
>
> real 5m6.676s
> user 0m0.009s
> sys 0m0.010s
>
>
> After COUCHDB-1186
>
> fdmanana 23:50:07 ~/git/hub/slow_couchdb (master)> docs=50
> batch=5000 ./bench.sh wow.tpl
> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
>
> [INFO] Created DB named `db1'
> [INFO] Uploaded 5000 documents via _bulk_docs
> ()
> [INFO] Uploaded 5000 documents via _bulk_docs
> Building view.
> {"total_rows":50,"offset":0,"rows":[
> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
> ]}
>
> real 5m1.395s
> user 0m0.008s
> sys 0m0.010s
>
>
> After COUCHDB-1186 + better queueing patch
> (http://friendpaste.com/178nPFgfyyeGf2vtNRpL0w)
>
> fdmanana 00:14:25 ~/git/hub/slow_couchdb (master)> docs=50
> batch=5000 ./bench.sh wow.tpl
> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
>
> [INFO] Created DB named `db1'
> [INFO] Uploaded 5000 documents via _bulk_docs
> (..
Re: Please report your indexing speed
Jason, repeated my last test with the 1Kb docs (
https://gist.github.com/1930804, map function
http://friendpaste.com/5C99aqXocN6N6H1BAYIigs ) to cover branch 1.1.x
as well. Here are the full results (also in
https://gist.github.com/1930807):
Before COUCHDB-1186
fdmanana 23:21:05 ~/git/hub/slow_couchdb (master)> docs=50
batch=5000 ./bench.sh wow.tpl
Server: CouchDB/1.2.0a-a68a792-git (Erlang OTP/R14B03)
{"couchdb":"Welcome","version":"1.2.0a-a68a792-git"}
[INFO] Created DB named `db1'
[INFO] Uploaded 5000 documents via _bulk_docs
()
[INFO] Uploaded 5000 documents via _bulk_docs
Building view.
{"total_rows":50,"offset":0,"rows":[
{"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
]}
real5m6.676s
user0m0.009s
sys 0m0.010s
After COUCHDB-1186
fdmanana 23:50:07 ~/git/hub/slow_couchdb (master)> docs=50
batch=5000 ./bench.sh wow.tpl
Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
{"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
[INFO] Created DB named `db1'
[INFO] Uploaded 5000 documents via _bulk_docs
()
[INFO] Uploaded 5000 documents via _bulk_docs
Building view.
{"total_rows":50,"offset":0,"rows":[
{"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
]}
real5m1.395s
user0m0.008s
sys 0m0.010s
After COUCHDB-1186 + better queueing patch
(http://friendpaste.com/178nPFgfyyeGf2vtNRpL0w)
fdmanana 00:14:25 ~/git/hub/slow_couchdb (master)> docs=50
batch=5000 ./bench.sh wow.tpl
Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
{"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
[INFO] Created DB named `db1'
[INFO] Uploaded 5000 documents via _bulk_docs
()
[INFO] Uploaded 5000 documents via _bulk_docs
Building view.
{"total_rows":50,"offset":0,"rows":[
{"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
]}
real4m48.175s
user0m0.008s
sys 0m0.009s
CouchDB branch 1.1.x
fdmanana 08:16:58 ~/git/hub/slow_couchdb (master)> docs=50
batch=5000 ./bench.sh wow.tpl
Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03)
{"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
[INFO] Created DB named `db1'
[INFO] Uploaded 5000 documents via _bulk_docs
()
[INFO] Uploaded 5000 documents via _bulk_docs
Building view.
{"total_rows":50,"offset":0,"rows":[
{"id":"0001c0a1-edcb-4dbc-aa9d-533c73d980cb","key":["dwarf","assassin"],"value":[{"x":62038.32,"y":105825.29},{"x":90713.13,"y":128570.97},{"x":43836.37,"y":80517.12},{"x":71610.97,"y":143739.99},{"x":86038.39,"y":84731.8}]}
]}
real5m44.374s
user0m0.008s
sys 0m0.010s
Disk model APPLE SSD TS128C, quad core machine, 8Gb of ram.
On Tue, Feb 28, 2012 at 5:17 AM, Jason Smith wrote:
> Forgive the clean new thread. Hopefully it will not remain so.
>
> If you can, would you please clone https://github.com/jhs/slow_couchdb
>
> And build whatever Erlangs and CouchDB checkouts you see fit, and run
> the test. For example:
>
> docs=50 ./bench.sh small_doc.tpl
>
> That should run the test and, God willing, upload the results to a
> couch in the cloud. We should be able to use that information to
> identify who you are, whether you are on SSD, what Erlang and Couch
> build, and how fast it ran. Modulo bugs.
--
Filipe David Manana,
"Reasonable men adapt themselves to the world.
Unreasonable men adapt the world to themselves.
That's why all progress depends on unreasonable men."
Please report your indexing speed
Forgive the clean new thread. Hopefully it will not remain so. If you can, would you please clone https://github.com/jhs/slow_couchdb And build whatever Erlangs and CouchDB checkouts you see fit, and run the test. For example: docs=50 ./bench.sh small_doc.tpl That should run the test and, God willing, upload the results to a couch in the cloud. We should be able to use that information to identify who you are, whether you are on SSD, what Erlang and Couch build, and how fast it ran. Modulo bugs.
