Re: Frugal Erlang vs Resources Hungry CouchDB

2011-07-02 Thread Robert Dionne
On Jul 2, 2011, at 2:50 AM, Jason Smith wrote: > On Fri, Jul 1, 2011 at 10:23 PM, Jens Alfke wrote: >> >> On Jun 30, 2011, at 11:37 PM, Zdravko Gligic wrote: >> >>> But neither one even bothered trying to answer my question of whether >>> just the last updated header or perhaps the last few are

Re: Frugal Erlang vs Resources Hungry CouchDB

2011-07-01 Thread Jason Smith
On Fri, Jul 1, 2011 at 10:23 PM, Jens Alfke wrote: > > On Jun 30, 2011, at 11:37 PM, Zdravko Gligic wrote: > >> But neither one even bothered trying to answer my question of whether >> just the last updated header or perhaps the last few are ever used. > > Just the last one. But at any point in ti

Re: Frugal Erlang vs Resources Hungry CouchDB

2011-07-01 Thread Paul Davis
On Fri, Jul 1, 2011 at 2:37 AM, Zdravko Gligic wrote: > I mapped out a set of web 2.0 level questions: a) How many of the update headers are actually useful?  Is it just the last successfully written one or even just a few last ones ? > > Jason Smith tried reducing them to my 0.2 level:

Re: Frugal Erlang vs Resources Hungry CouchDB

2011-07-01 Thread Randall Leeds
On Jun 30, 2011 11:52 PM, "Zdravko Gligic" wrote: > > I mapped out a set of web 2.0 level questions: > >>> a) How many of the update headers are actually useful? Is it just the > >>> last successfully written one or even just a few last ones ? > > Jason Smith tried reducing them to my 0.2 level:

Re: Frugal Erlang vs Resources Hungry CouchDB

2011-07-01 Thread Adam Kocoloski
On Jul 1, 2011, at 11:23 AM, Jens Alfke wrote: > I am not sure whether the db looks up older revisions of documents by > starting from an earlier header (“going back in time”); I don’t think so, > because this would be inefficient (O(N)) for finding a specific revision of a > document. Instead

Re: Frugal Erlang vs Resources Hungry CouchDB

2011-07-01 Thread Jens Alfke
On Jun 30, 2011, at 11:37 PM, Zdravko Gligic wrote: > Then there is the whole issue of servers (where much of this makes > sense because of scalability) vs clients (where very little of this > makes any sense because there are no multi-user concurrency issues). Sure there are. I’ve spent my care

Re: Frugal Erlang vs Resources Hungry CouchDB

2011-07-01 Thread Jens Alfke
On Jun 30, 2011, at 11:37 PM, Zdravko Gligic wrote: > But neither one even bothered trying to answer my question of whether > just the last updated header or perhaps the last few are ever used. Just the last one. But at any point in time, the last one is vital for recovery. It just becomes usel

Re: Frugal Erlang vs Resources Hungry CouchDB

2011-07-01 Thread Robert Newson
Every header is potentially used but every header supplants every previous header. The reason we must write a header for each update is durability. If we did not do so, then a crash could undo an update, which is not acceptable for a database. Moreover, with the recommended production setting of de

Re: Frugal Erlang vs Resources Hungry CouchDB

2011-06-30 Thread Zdravko Gligic
I mapped out a set of web 2.0 level questions: >>> a) How many of the update headers are actually useful? Is it just the >>> last successfully written one or even just a few last ones ? Jason Smith tried reducing them to my 0.2 level: >> Excellent point. That's just it, isn't it? >> How useful ar

Re: Frugal Erlang vs Resources Hungry CouchDB

2011-06-30 Thread Jens Alfke
On Jun 30, 2011, at 5:11 PM, Jason Smith wrote: > On Fri, Jul 1, 2011 at 5:39 AM, Zdravko Gligic wrote: >> a) How many of the update headers are actually useful? Is it just the >> last successfully written one or even just a few last ones ? > > Excellent point. That's just it, isn't it? > How

Re: Frugal Erlang vs Resources Hungry CouchDB

2011-06-30 Thread Jason Smith
On Fri, Jul 1, 2011 at 5:39 AM, Zdravko Gligic wrote: > a) How many of the update headers are actually useful?  Is it just the > last successfully written one or even just a few last ones ? Excellent point. That's just it, isn't it? How useful are the lower rungs of a ladder? How useful is the f

Re: Frugal Erlang vs Resources Hungry CouchDB

2011-06-30 Thread Paul Davis
On Thu, Jun 30, 2011 at 6:39 PM, Zdravko Gligic wrote: > Robert Newson wrote: >>> > CouchDB *must* write an updated btree and an updated header to point > to the root of that btree every time you update a document, or it will > be lost if couch crashed right then. > << > > So, we have these 3 piec

Re: Frugal Erlang vs Resources Hungry CouchDB

2011-06-30 Thread Zdravko Gligic
Robert Newson wrote: >> CouchDB *must* write an updated btree and an updated header to point to the root of that btree every time you update a document, or it will be lost if couch crashed right then. << So, we have these 3 pieces of info that need to be written with every update of a document: 1)

Re: Frugal Erlang vs Resources Hungry CouchDB

2011-06-30 Thread Paul Davis
On Thu, Jun 30, 2011 at 12:29 PM, Jens Alfke wrote: > > On Jun 29, 2011, at 7:00 PM, sleepnova wrote: > >> I think what many people really concerned is the growing pattern of size as >> number of docs increase. (space complexity) >> (If it grows exponentially then that's not a good sign.) > > It’s

Re: Frugal Erlang vs Resources Hungry CouchDB

2011-06-30 Thread Jens Alfke
On Jun 29, 2011, at 7:00 PM, sleepnova wrote: > I think what many people really concerned is the growing pattern of size as > number of docs increase. (space complexity) > (If it grows exponentially then that's not a good sign.) It’s basically linear, assuming the database gets compacted periodi

Re: Frugal Erlang vs Resources Hungry CouchDB

2011-06-30 Thread Robert Newson
I'd say the essential thing that CouchDB "knows" during compaction that it does not know earlier is *your documents*. CouchDB *must* write an updated btree and an updated header to point to the root of that btree every time you update a document, or it will be lost if couch crashed right then. Whe

Re: Frugal Erlang vs Resources Hungry CouchDB

2011-06-30 Thread Randall Leeds
On Wed, Jun 29, 2011 at 19:13, Zdravko Gligic wrote: > If these three points are more or less correct ... > > 1) CouchDB keeps appending to the end of the file.  Fine. > > 2) It needs just as much disk space when doing a compaction.  Is that > extra space equivalent to the original uncompacted or

Re: Frugal Erlang vs Resources Hungry CouchDB

2011-06-29 Thread Zdravko Gligic
If these three points are more or less correct ... 1) CouchDB keeps appending to the end of the file. Fine. 2) It needs just as much disk space when doing a compaction. Is that extra space equivalent to the original uncompacted or the final compacted version? 3) Compaction is similar to replic

Re: Frugal Erlang vs Resources Hungry CouchDB

2011-06-29 Thread sleepnova
I think what many people really concerned is the growing pattern of size as number of docs increase. (space complexity) (If it grows exponentially then that's not a good sign.) So is there any official/non-official, theoretically/benchmark showing this characteristic? 2011/6/30 Paul Davis > > Te

Re: Frugal Erlang vs Resources Hungry CouchDB

2011-06-29 Thread Dale Harvey
I was about to post some of what Paul just did On mobile devices the bottlenecks are in CPU and flash storage, erlang (and therefore couch) dont have a true idle state, but they do very well at limiting battery usage on devices, the Android version is now down to 4/5 MB of flash storage which is c

Re: Frugal Erlang vs Resources Hungry CouchDB

2011-06-29 Thread Jens Alfke
On Jun 29, 2011, at 4:59 PM, Zdravko Gligic wrote: > In many places I have read how Erlang runs on small devices and how > (as a result) it is very frugal with resources. I think that I have > read that or at least something to that effect. I’m not an Erlang expert, but the “typical” use-case E

Re: Frugal Erlang vs Resources Hungry CouchDB

2011-06-29 Thread Paul Davis
On Wed, Jun 29, 2011 at 7:59 PM, Zdravko Gligic wrote: > Hi Folks, > > In many places I have read how Erlang runs on small devices and how > (as a result) it is very frugal with resources.  I think that I have > read that or at least something to that effect.  However, none of that > seems to appl

Frugal Erlang vs Resources Hungry CouchDB

2011-06-29 Thread Zdravko Gligic
Hi Folks, In many places I have read how Erlang runs on small devices and how (as a result) it is very frugal with resources. I think that I have read that or at least something to that effect. However, none of that seems to apply to CouchDB. I believe that I read somewhere that the length of k