I have been working with a very similar problem, actually. A large set
of records (40k+), building views from scratch.
My experience was that I just needed to let couchdb build the view. It
can take several minutes, and the CPU usage will be high. You should
see both the beam and couchjs processes working while the view is
building. If you're accessing a view via Futon, it's likely the
browser will time-out the request before the build is finished. The
build process *will* continue on the server side, though. If you let
the build finish, the next time you query the view, it will return the
data immediately.
To mitigate this problem, I'm now updating the view every time I do an
insert (I bulk-add 20 records per minute). This only requires that the
new data be added to the view, so building at this point is a short
process.
(big thanks to the folks on #couchdb for helping me with this problem!)
--
Ed Finkler
http://funkatron.com
AIM: funka7ron
ICQ: 3922133
Skype: funka7ron
On Jul 31, 2008, at 5:10 PM, Demetrius Nunes wrote:
Hi there,
I was having a great time playing aroung with CouchDB. It seems like a
perfect fit for a future system that we'll be building fairly soon.
But then, I've just created a CouchDB database, importing 20.000
records
from an old relational database into it.
When I go into Futon, I can see the database is there, with 20.899
documents
and 125.2 MB in size.
Clicking on it, I can navigate thru the "All Documents" pretty
quickly (10
documents per page).
The problem is when I try to create a custom view. Just as I enter the
custom view page in Futon, the server hangs and locks up my CPU at 90%
usage. I waited several minutes for it to cool off but the process
was still
there and I had no response at all.
I then tried to create a view programatically, using REST/JSON and I
get the
same result.
I am running CouchDB 0.8.0 on Ubuntu 8.0.4.
Is CouchDB not ready for a dataset of this size yet?
Thanks and best regards,
Dema
--
____________________________
http://www.demetriusnunes.com
