Hi Joan, Robert,
I appreciate your comments, thanks! Answers inline:
> * What version of CouchDB?
Unfortunately we are still stuck with 1.6.1
> * Do you regularly compact your databases?
Probably in a daily bases for both the databases and the views. We
started using the compact_daemon
as of a couple of months ago, with configs like:
"""
[compactions]
_default = [{db_fragmentation, "70%"}, {view_fragmentation, "60%"},
{from, "20:00"}, {to, "05:00"}]
[compaction_daemon]
check_interval = 3600
min_file_size = 209715200
"""
> * Have you looked for conflicted documents recently?
Not yet. Can you please remind me whether there is any straight
forward way to look for conflicts? Or
do I need to create a view for it?
> and note that view compaction is a separate operation to database compaction,
> and is necessary for performance.
Yes, views are regularly compacted as well.
Early this morning we had 3M documents, and I thought it could somehow
be related to the total amount of docs.
Given that all that data is temporary, and most of it was still
available on the other couchdb instances, we dumped the
database and views. Final result was around 1.5M docs getting
replicated within an hour or so, views took slightly longer
to build (1.5h). Two hours late, we got hit by the super slow views
indexing and low rate view response (kB/s).
I read about separating the views into multiple design docs, which - I
guess - is going to be the next thing I will try.
One more note, all those views are written in erlang, and the config is:
[native_query_servers]
erlang = {couch_native_process, start_link, []}
so I'd expect the best performance possible in such setup.
Thanks,
Alan.
On Wed, May 20, 2020 at 7:25 PM Robert Samuel Newson <rnew...@apache.org> wrote:
>
> and note that view compaction is a separate operation to database compaction,
> and is necessary for performance.
>
> in 3.0 both of these types of compaction are automated.
>
> B.
>
> > On 20 May 2020, at 18:23, Joan Touzet <woh...@apache.org> wrote:
> >
> > HI Alan,
> >
> > * What version of CouchDB?
> > * Do you regularly compact your databases?
> > * Have you looked for conflicted documents recently?
> >
> > -Joan
> >
> > On 2020-05-20 10:02, Alan Malta wrote:
> >> Hi everyone,
> >> it's been more than a week that I have been debugging a strange
> >> performance problem with CouchDB; mainly affecting couchdb views.
> >> About my Couch setup, I have one central couch instance and around 5
> >> to 15 other instances replicating documents to it. In addition to the
> >> replication, each of those other couch instances are also running a
> >> service that posts documents to the central one via '_bulk_docs' API.
> >> It's important to note that this model is deployed in production for
> >> many years now.
> >> What started to happen is that the indexing of the views became very
> >> very slow, like < 1k changes within 10min. Making GET calls to the
> >> views (either with or without reduce function), I also see a poor
> >> response rate (a few tens kilobytes, either remotely or localhost).
> >> Has anyone ever faced such slowness with CouchDB (views)? Would you
> >> have any recommendations on where I should start looking and tests to
> >> be performed? I have already ruled out problems with the virtual
> >> machine and the hypervisor (load is normal for months). I have also
> >> already recreated the views from scratch; recreated the database from
> >> scratch (dumping deleted docs). I have also created a view to see
> >> whether there were any large documents, and the biggest one is only
> >> .5MB.
> >> When I replicated the database from scratch today, CouchDB indexed
> >> around 1.5M docs in an hour or so; while it's been the last 2h
> >> indexing 26k changes to the database...
> >> Any help or pointers would be very much appreciated here.
> >> Thanks,
> >> Alan.
>
