Hi Jan, Thanks for your quick reply to my question. I have some answers to your questions and some new information that I got from running couched interactively.
> Are there any other things going on on the VM, when you do this? The VM also hosts a MySQL server, but I see no evidence that this is a contributing cause for the couch issue. > > Can you reliably reproduce this behavior? I can reliably reproduce it. > > Are there other correlating factors (like does this always happen at the same > time / due to a cronjob, etc)? It can be repeated by re-starting couched and re-requesting the compaction on all databases. It is not time-dependent. > > Can you set your CouchDB log level to debug and see if that gets you more > info? (curl -X PUT http://[user:pass@]127.0.0.1:5984/_config/log/level -d > '"debug"’). (see below) > > Is it possible for you to share these database files (publicly or in private)? The databases contain health data and I am unable to share them. > > What are your disk usage levels before/during compaction? Plenty of disk in this case. The compacted data is in the 2G range. The problem does not seem to be storage-size related. We are able to compact during regular operation when using a 40G instance volume. Unable to compact when using a 120G EBS volume. > > Are you getting anything in the system log(s)? That is what is odd. There is nothing in the system logs or the couchdb logs. Bonus data: I ran the debug experiment with couchdb running interactively. The session text is below, but note that I also got the following error message and an erlang core dump: Crash dump was written to: erl_crash.dump eheap_alloc: Cannot allocate 156725600 bytes of memory (of type "old_heap"). Aborted (core dumped) The dump is ~500MB. Here is the session text: [gtarsa@prod-db01 ~]$ sudo /usr/local/bin/couchdb Apache CouchDB 1.6.1 (LogLevel=info) is starting. Apache CouchDB has started. Time to relax. [info] [<0.31.0>] Apache CouchDB has started on http://0.0.0.0:5984/ [info] [<0.120.0>] 127.0.0.1 - - GET /_config/level 200 [info] [<0.678.0>] 127.0.0.1 - - GET /_config/log/level 200 [error] [<0.803.0>] attempted upload of invalid JSON (set log_level to debug to log it) [info] [<0.803.0>] 127.0.0.1 - - PUT /_config/log/level 400 =SUPERVISOR REPORT==== 8-Mar-2016::11:09:22 === Supervisor: {local,couch_primary_services} Context: child_terminated Reason: normal Offender: [{pid,<0.92.0>}, {name,couch_log}, {mfargs,{couch_log,start_link,[]}}, {restart_type,permanent}, {shutdown,brutal_kill}, {child_type,worker}] [debug] [<0.117.0>] 'PUT' /_config/log/level {1,1} from "127.0.0.1" Headers: [{'Accept',"*/*"}, {'Content-Length',"7"}, {'Content-Type',"application/x-www-form-urlencoded"}, {'Host',"127.0.0.1:5984"}, {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}] [debug] [<0.117.0>] OAuth Params: [] =SUPERVISOR REPORT==== 8-Mar-2016::11:09:25 === Supervisor: {local,couch_primary_services} Context: child_terminated Reason: normal Offender: [{pid,<0.828.0>}, {name,couch_log}, {mfargs,{couch_log,start_link,[]}}, {restart_type,permanent}, {shutdown,brutal_kill}, {child_type,worker}] [debug] [<0.826.0>] 'GET' /_all_dbs {1,1} from "127.0.0.1" Headers: [{'Accept',"*/*"}, {'Content-Type',"application/json"}, {'Host',"localhost:5984"}, {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}] [debug] [<0.826.0>] OAuth Params: [] [info] [<0.826.0>] 127.0.0.1 - - GET /_all_dbs 200 [debug] [<0.833.0>] 'POST' /biometrics/_compact {1,1} from "127.0.0.1" Headers: [{'Accept',"*/*"}, {'Content-Type',"application/json"}, {'Host',"localhost:5984"}, {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}] [debug] [<0.833.0>] OAuth Params: [] [info] [<0.872.0>] Starting compaction for db "biometrics" [debug] [<0.877.0>] Compaction process spawned for db "biometrics" [info] [<0.833.0>] 127.0.0.1 - - POST /biometrics/_compact 202 [debug] [<0.115.0>] 'POST' /biometrics/_view_cleanup {1,1} from "127.0.0.1" Headers: [{'Accept',"*/*"}, {'Content-Type',"application/json"}, {'Host',"localhost:5984"}, {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}] [debug] [<0.115.0>] OAuth Params: [] [info] [<0.115.0>] 127.0.0.1 - - POST /biometrics/_view_cleanup 202 [debug] [<0.114.0>] 'POST' /diabetes/_compact {1,1} from "127.0.0.1" Headers: [{'Accept',"*/*"}, {'Content-Type',"application/json"}, {'Host',"localhost:5984"}, {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}] [debug] [<0.114.0>] OAuth Params: [] [info] [<0.890.0>] Starting compaction for db "diabetes" [debug] [<0.895.0>] Compaction process spawned for db "diabetes" [info] [<0.114.0>] 127.0.0.1 - - POST /diabetes/_compact 202 [debug] [<0.113.0>] 'POST' /diabetes/_view_cleanup {1,1} from "127.0.0.1" Headers: [{'Accept',"*/*"}, {'Content-Type',"application/json"}, {'Host',"localhost:5984"}, {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}] [debug] [<0.113.0>] OAuth Params: [] [info] [<0.113.0>] 127.0.0.1 - - POST /diabetes/_view_cleanup 202 [debug] [<0.112.0>] 'POST' /fitness/_compact {1,1} from "127.0.0.1" Headers: [{'Accept',"*/*"}, {'Content-Type',"application/json"}, {'Host',"localhost:5984"}, {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}] [debug] [<0.112.0>] OAuth Params: [] [info] [<0.909.0>] Starting compaction for db "fitness" [debug] [<0.914.0>] Compaction process spawned for db "fitness" [info] [<0.112.0>] 127.0.0.1 - - POST /fitness/_compact 202 [debug] [<0.111.0>] 'POST' /fitness/_view_cleanup {1,1} from "127.0.0.1" Headers: [{'Accept',"*/*"}, {'Content-Type',"application/json"}, {'Host',"localhost:5984"}, {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}] [debug] [<0.111.0>] OAuth Params: [] [info] [<0.111.0>] 127.0.0.1 - - POST /fitness/_view_cleanup 202 [debug] [<0.110.0>] 'POST' /nutrition/_compact {1,1} from "127.0.0.1" Headers: [{'Accept',"*/*"}, {'Content-Type',"application/json"}, {'Host',"localhost:5984"}, {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}] [debug] [<0.110.0>] OAuth Params: [] [info] [<0.927.0>] Starting compaction for db "nutrition" [debug] [<0.932.0>] Compaction process spawned for db "nutrition" [info] [<0.110.0>] 127.0.0.1 - - POST /nutrition/_compact 202 [debug] [<0.109.0>] 'POST' /nutrition/_view_cleanup {1,1} from "127.0.0.1" Headers: [{'Accept',"*/*"}, {'Content-Type',"application/json"}, {'Host',"localhost:5984"}, {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}] [debug] [<0.109.0>] OAuth Params: [] [info] [<0.109.0>] 127.0.0.1 - - POST /nutrition/_view_cleanup 202 [debug] [<0.108.0>] 'POST' /routine/_compact {1,1} from "127.0.0.1" Headers: [{'Accept',"*/*"}, {'Content-Type',"application/json"}, {'Host',"localhost:5984"}, {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}] [debug] [<0.108.0>] OAuth Params: [] [info] [<0.945.0>] Starting compaction for db "routine" [debug] [<0.950.0>] Compaction process spawned for db "routine" [info] [<0.108.0>] 127.0.0.1 - - POST /routine/_compact 202 [debug] [<0.123.0>] 'POST' /routine/_view_cleanup {1,1} from "127.0.0.1" Headers: [{'Accept',"*/*"}, {'Content-Type',"application/json"}, {'Host',"localhost:5984"}, {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}] [debug] [<0.123.0>] OAuth Params: [] [info] [<0.123.0>] 127.0.0.1 - - POST /routine/_view_cleanup 202 [debug] [<0.122.0>] 'POST' /sleep/_compact {1,1} from "127.0.0.1" Headers: [{'Accept',"*/*"}, {'Content-Type',"application/json"}, {'Host',"localhost:5984"}, {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}] [debug] [<0.122.0>] OAuth Params: [] [info] [<0.963.0>] Starting compaction for db "sleep" [debug] [<0.87.0>] New task status for <0.895.0>: [{changes_done,113}, {database,<<"diabetes">>}, {progress,100}, {started_on,1457453394}, {total_changes,113}, {type,database_compaction}, {updated_on,1457453395}] [debug] [<0.968.0>] Compaction process spawned for db "sleep" [info] [<0.122.0>] 127.0.0.1 - - POST /sleep/_compact 202 [debug] [<0.385.0>] 'POST' /sleep/_view_cleanup {1,1} from "127.0.0.1" Headers: [{'Accept',"*/*"}, {'Content-Type',"application/json"}, {'Host',"localhost:5984"}, {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}] [debug] [<0.385.0>] OAuth Params: [] [debug] [<0.890.0>] CouchDB swapping files /usr/local/var/lib/couchdb/diabetes.couch and /usr/local/var/lib/couchdb/diabetes.couch.compact. [info] [<0.385.0>] 127.0.0.1 - - POST /sleep/_view_cleanup 202 [info] [<0.890.0>] Compaction for db "diabetes" completed. [debug] [<0.635.0>] 'POST' /tobacco_cessation/_compact {1,1} from "127.0.0.1" Headers: [{'Accept',"*/*"}, {'Content-Type',"application/json"}, {'Host',"localhost:5984"}, {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}] [debug] [<0.635.0>] OAuth Params: [] [info] [<0.987.0>] Starting compaction for db "tobacco_cessation" [debug] [<0.992.0>] Compaction process spawned for db "tobacco_cessation" [info] [<0.635.0>] 127.0.0.1 - - POST /tobacco_cessation/_compact 202 [debug] [<0.87.0>] New task status for <0.992.0>: [{changes_done,1}, {database, <<"tobacco_cessation">>}, {progress,100}, {started_on,1457453395}, {total_changes,1}, {type,database_compaction}, {updated_on,1457453395}] [debug] [<0.640.0>] 'POST' /tobacco_cessation/_view_cleanup {1,1} from "127.0.0.1" Headers: [{'Accept',"*/*"}, {'Content-Type',"application/json"}, {'Host',"localhost:5984"}, {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}] [debug] [<0.640.0>] OAuth Params: [] [info] [<0.640.0>] 127.0.0.1 - - POST /tobacco_cessation/_view_cleanup 202 [debug] [<0.987.0>] CouchDB swapping files /usr/local/var/lib/couchdb/tobacco_cessation.couch and /usr/local/var/lib/couchdb/tobacco_cessation.couch.compact. [info] [<0.987.0>] Compaction for db "tobacco_cessation" completed. [debug] [<0.815.0>] 'POST' /trackers/_compact {1,1} from "127.0.0.1" Headers: [{'Accept',"*/*"}, {'Content-Type',"application/json"}, {'Host',"localhost:5984"}, {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}] [debug] [<0.815.0>] OAuth Params: [] [info] [<0.1011.0>] Starting compaction for db "trackers" [debug] [<0.1016.0>] Compaction process spawned for db "trackers" [info] [<0.815.0>] 127.0.0.1 - - POST /trackers/_compact 202 [debug] [<0.866.0>] 'POST' /trackers/_view_cleanup {1,1} from "127.0.0.1" Headers: [{'Accept',"*/*"}, {'Content-Type',"application/json"}, {'Host',"localhost:5984"}, {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}] [debug] [<0.866.0>] OAuth Params: [] [info] [<0.866.0>] 127.0.0.1 - - POST /trackers/_view_cleanup 202 [debug] [<0.867.0>] 'POST' /users/_compact {1,1} from "127.0.0.1" Headers: [{'Accept',"*/*"}, {'Content-Type',"application/json"}, {'Host',"localhost:5984"}, {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}] [debug] [<0.867.0>] OAuth Params: [] [info] [<0.1029.0>] Starting compaction for db "users" [debug] [<0.1034.0>] Compaction process spawned for db "users" [info] [<0.867.0>] 127.0.0.1 - - POST /users/_compact 202 [debug] [<0.87.0>] New task status for <0.1034.0>: [{changes_done,0}, {database,<<"users">>}, {progress,0}, {started_on,1457453395}, {total_changes,0}, {type,database_compaction}, {updated_on,1457453395}] [debug] [<0.884.0>] 'POST' /users/_view_cleanup {1,1} from "127.0.0.1" Headers: [{'Accept',"*/*"}, {'Content-Type',"application/json"}, {'Host',"localhost:5984"}, {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}] [debug] [<0.884.0>] OAuth Params: [] [info] [<0.884.0>] 127.0.0.1 - - POST /users/_view_cleanup 202 [debug] [<0.1029.0>] CouchDB swapping files /usr/local/var/lib/couchdb/users.couch and /usr/local/var/lib/couchdb/users.couch.compact. [info] [<0.1029.0>] Compaction for db "users" completed. [debug] [<0.885.0>] 'POST' /weight/_compact {1,1} from "127.0.0.1" Headers: [{'Accept',"*/*"}, {'Content-Type',"application/json"}, {'Host',"localhost:5984"}, {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}] [debug] [<0.885.0>] OAuth Params: [] [info] [<0.1053.0>] Starting compaction for db "weight" [debug] [<0.1058.0>] Compaction process spawned for db "weight" [info] [<0.885.0>] 127.0.0.1 - - POST /weight/_compact 202 [debug] [<0.902.0>] 'POST' /weight/_view_cleanup {1,1} from "127.0.0.1" Headers: [{'Accept',"*/*"}, {'Content-Type',"application/json"}, {'Host',"localhost:5984"}, {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}] [debug] [<0.902.0>] OAuth Params: [] [info] [<0.902.0>] 127.0.0.1 - - POST /weight/_view_cleanup 202 Crash dump was written to: erl_crash.dump eheap_alloc: Cannot allocate 156725600 bytes of memory (of type "old_heap"). [gtarsa@prod-db01 ~]$ > On Mar 7, 2016, at 4:09 PM, Jan Lehnardt <[email protected]> wrote: > > Heya Greg, > > this should definitely not happen at all, regardless of AWS storage type. > > Are there any other things going on on the VM, when you do this? > > Can you reliably reproduce this behaviour? > > Are there other correlating factors (like does this always happen at the same > time / due to a cronjob, etc)? > > Can you set your CouchDB log level to debug and see if that gets you more > info? (curl -X PUT http://[user:pass@]127.0.0.1:5984/_config/log/level -d > '"debug"'). > > Is it possible for you to share these database files (publicly or in private)? > > What are your disk usage levels before/during compaction? > > Are you getting anything in the system log(s)? > > Best > Jan > -- > Professional Support for Apache CouchDB: > https://neighbourhood.ie/couchdb-support/ > > >> On 07 Mar 2016, at 21:27, Greg Tarsa <[email protected]> wrote: >> >> We have a set of couchdb databases that we use to collect user information >> for various purposes. I am inheriting this configuration from a predecessor >> and am relatively new to couchdb. >> >> Whenever we attempt to compact the databases, the server crashes without any >> messages either in the couchdb log or the system logs. This is running in >> an AWS instance with an EBS volume. >> >> Experiments have shown that if the instance is configured with instance >> storage (ephemeral storage that disappears when the instance disappears) >> then this operation works properly. But we would like to use larger >> volumes and have persistence. >> >> When the instance is configured with an external EBS volume, then we see the >> server crash described above. >> >> I have searched the web for “couchdb compaction crash no log” and not found >> anything helpful. >> >> It seems like compacting while running should not be failing at all, much >> less silently, so I am looking for insights to the problem, or solutions if >> such exist. >> >> Configuration and log info is below. >> >> Any help would be appreciated. >> >> Thanks, >> Greg >> >> >> --------------------------------------------------------- >> >> CouchDB version: 1.6.1 >> OS: RHEL 6.6 >> >> --------------------------------------------------------- >> >> Here is a directory of the databases as the time of the crash: >> >> cat bad.couch.dbinfo.txt >> total 15400740 >> 12 -rw-r--r--. 1 couchdb couchdb 8297 Jan 20 16:31 _users.couch >> 16 -rw-r--r--. 1 couchdb couchdb 12393 Jan 20 16:33 _replicator.couch >> 21060 -rw-r--r--. 1 couchdb couchdb 21557368 Mar 7 11:57 biometrics.couch >> 781136 -rw-r--r--. 1 couchdb couchdb 799875192 Mar 7 12:00 fitness.couch >> 954244 -rw-r--r--. 1 couchdb couchdb 977137784 Mar 7 12:05 nutrition.couch >> 8419624 -rw-r--r--. 1 couchdb couchdb 8621678721 Mar 7 12:06 routine.couch >> 390796 -rw-r--r--. 1 couchdb couchdb 400167032 Mar 7 12:06 sleep.couch >> 217932 -rw-r--r--. 1 couchdb couchdb 223154296 Mar 7 12:06 weight.couch >> 4614884 -rw-r--r--. 1 couchdb couchdb 4725629060 Mar 7 12:06 trackers.couch >> 4 -rw-r--r--. 1 couchdb couchdb 79 Mar 7 12:41 >> fitness.couch.compact >> 4 -rw-r--r--. 1 couchdb couchdb 79 Mar 7 12:41 >> nutrition.couch.compact >> 4 -rw-r--r--. 1 couchdb couchdb 79 Mar 7 12:41 >> routine.couch.compact >> 64 -rw-r--r--. 1 couchdb couchdb 61551 Mar 7 12:41 diabetes.couch >> 4 -rw-r--r--. 1 couchdb couchdb 79 Mar 7 12:41 >> sleep.couch.compact >> 12 -rw-r--r--. 1 couchdb couchdb 8300 Mar 7 12:41 >> tobacco_cessation.couch >> 4 -rw-r--r--. 1 couchdb couchdb 79 Mar 7 12:41 users.couch >> 4 -rw-r--r--. 1 couchdb couchdb 79 Mar 7 12:41 >> weight.couch.compact >> 152 -rw-r--r--. 1 couchdb couchdb 151797 Mar 7 12:42 >> trackers.couch.compact >> 784 -rw-r--r--. 1 couchdb couchdb 801865 Mar 7 12:42 >> biometrics.couch.compact >> >> --------------------------------------------------------- >> >> Here is the contents of the log at the time of the crash: >> >> [Mon, 07 Mar 2016 17:25:32 GMT] [info] [<0.31.0>] Apache CouchDB has started >> on http://0.0.0.0:5984/ >> [Mon, 07 Mar 2016 17:25:32 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET >> /_active_tasks 200 >> [Mon, 07 Mar 2016 17:25:33 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET >> /favicon.ico 200 >> [Mon, 07 Mar 2016 17:25:34 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET >> /_active_tasks 200 >> [Mon, 07 Mar 2016 17:25:34 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET >> /favicon.ico 200 >> [Mon, 07 Mar 2016 17:25:34 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET >> /_active_tasks 200 >> [Mon, 07 Mar 2016 17:25:34 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET >> /favicon.ico 200 >> [Mon, 07 Mar 2016 17:25:35 GMT] [info] [<0.123.0>] 10.1.1.12 - - GET /users/ >> 200 >> [Mon, 07 Mar 2016 17:25:37 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET >> /_active_tasks 200 >> [Mon, 07 Mar 2016 17:25:37 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET >> /favicon.ico 200 >> [Mon, 07 Mar 2016 17:25:38 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET >> /_active_tasks 200 >> [Mon, 07 Mar 2016 17:25:38 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET >> /favicon.ico 200 >> [Mon, 07 Mar 2016 17:25:40 GMT] [info] [<0.123.0>] 10.1.1.12 - - GET /users/ >> 200 >> [Mon, 07 Mar 2016 17:25:45 GMT] [info] [<0.123.0>] 10.1.1.12 - - GET /users/ >> 200 >> [Mon, 07 Mar 2016 17:25:50 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET >> /_active_tasks 200 >> [Mon, 07 Mar 2016 17:25:50 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET >> /favicon.ico 200 >> [Mon, 07 Mar 2016 17:25:50 GMT] [info] [<0.123.0>] 10.1.1.12 - - GET /users/ >> 200 >> [Mon, 07 Mar 2016 17:25:52 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET >> /_active_tasks 200 >> [Mon, 07 Mar 2016 17:25:52 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET >> /favicon.ico 200 >> [Mon, 07 Mar 2016 17:25:54 GMT] [info] [<0.114.0>] 10.1.1.12 - - GET >> /users/_changes?feed=continuous&style=all_docs&since=0&heartbeat=10000 200 >> [Mon, 07 Mar 2016 17:25:54 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET >> /_active_tasks 200 >> [Mon, 07 Mar 2016 17:25:54 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET >> /favicon.ico 200 >> [Mon, 07 Mar 2016 17:25:55 GMT] [info] [<0.123.0>] 10.1.1.12 - - GET /users/ >> 200 >> [Mon, 07 Mar 2016 17:26:00 GMT] [info] [<0.123.0>] 10.1.1.12 - - GET /users/ >> 200 >> [Mon, 07 Mar 2016 17:26:00 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET >> /_active_tasks 200 >> [Mon, 07 Mar 2016 17:26:01 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET >> /favicon.ico 200 >> [Mon, 07 Mar 2016 17:26:05 GMT] [info] [<0.123.0>] 10.1.1.12 - - GET /users/ >> 200 >> ... [numerous GET /users/ 200 messages removed for brevity] ... >> [Mon, 07 Mar 2016 17:41:51 GMT] [info] [<0.123.0>] 10.1.1.12 - - GET /users/ >> 200 >> [Mon, 07 Mar 2016 17:41:52 GMT] [info] [<0.152.0>] 127.0.0.1 - - GET >> /_all_dbs 200 >> [Mon, 07 Mar 2016 17:41:52 GMT] [info] [<0.1157.0>] Starting compaction for >> db "biometrics" >> [Mon, 07 Mar 2016 17:41:52 GMT] [info] [<0.151.0>] 127.0.0.1 - - POST >> /biometrics/_compact 202 >> [Mon, 07 Mar 2016 17:41:52 GMT] [info] [<0.150.0>] 127.0.0.1 - - POST >> /biometrics/_view_cleanup 202 >> [Mon, 07 Mar 2016 17:41:52 GMT] [info] [<0.1175.0>] Starting compaction for >> db "diabetes" >> [Mon, 07 Mar 2016 17:41:52 GMT] [info] [<0.198.0>] 127.0.0.1 - - POST >> /diabetes/_compact 202 >> [Mon, 07 Mar 2016 17:41:52 GMT] [info] [<0.197.0>] 127.0.0.1 - - POST >> /diabetes/_view_cleanup 202 >> [Mon, 07 Mar 2016 17:41:52 GMT] [info] [<0.1193.0>] Starting compaction for >> db "fitness" >> [Mon, 07 Mar 2016 17:41:52 GMT] [info] [<0.118.0>] 127.0.0.1 - - POST >> /fitness/_compact 202 >> [Mon, 07 Mar 2016 17:41:52 GMT] [info] [<0.119.0>] 127.0.0.1 - - POST >> /fitness/_view_cleanup 202 >> [Mon, 07 Mar 2016 17:41:52 GMT] [info] [<0.1211.0>] Starting compaction for >> db "nutrition" >> [Mon, 07 Mar 2016 17:41:52 GMT] [info] [<0.120.0>] 127.0.0.1 - - POST >> /nutrition/_compact 202 >> [Mon, 07 Mar 2016 17:41:52 GMT] [info] [<0.121.0>] 127.0.0.1 - - POST >> /nutrition/_view_cleanup 202 >> [Mon, 07 Mar 2016 17:41:53 GMT] [info] [<0.1229.0>] Starting compaction for >> db "routine" >> [Mon, 07 Mar 2016 17:41:53 GMT] [info] [<0.122.0>] 127.0.0.1 - - POST >> /routine/_compact 202 >> [Mon, 07 Mar 2016 17:41:53 GMT] [info] [<0.115.0>] 127.0.0.1 - - POST >> /routine/_view_cleanup 202 >> [Mon, 07 Mar 2016 17:41:53 GMT] [info] [<0.1175.0>] Compaction for db >> "diabetes" completed. >> [Mon, 07 Mar 2016 17:41:53 GMT] [info] [<0.1254.0>] Starting compaction for >> db "sleep" >> [Mon, 07 Mar 2016 17:41:53 GMT] [info] [<0.116.0>] 127.0.0.1 - - POST >> /sleep/_compact 202 >> [Mon, 07 Mar 2016 17:41:53 GMT] [info] [<0.117.0>] 127.0.0.1 - - POST >> /sleep/_view_cleanup 202 >> [Mon, 07 Mar 2016 17:41:53 GMT] [info] [<0.1272.0>] Starting compaction for >> db "tobacco_cessation" >> [Mon, 07 Mar 2016 17:41:53 GMT] [info] [<0.184.0>] 127.0.0.1 - - POST >> /tobacco_cessation/_compact 202 >> [Mon, 07 Mar 2016 17:41:53 GMT] [info] [<0.183.0>] 127.0.0.1 - - POST >> /tobacco_cessation/_view_cleanup 202 >> [Mon, 07 Mar 2016 17:41:53 GMT] [info] [<0.1290.0>] Starting compaction for >> db "trackers" >> [Mon, 07 Mar 2016 17:41:53 GMT] [info] [<0.182.0>] 127.0.0.1 - - POST >> /trackers/_compact 202 >> [Mon, 07 Mar 2016 17:41:53 GMT] [info] [<0.1272.0>] Compaction for db >> "tobacco_cessation" completed. >> [Mon, 07 Mar 2016 17:41:53 GMT] [info] [<0.1151.0>] 127.0.0.1 - - POST >> /trackers/_view_cleanup 202 >> [Mon, 07 Mar 2016 17:41:53 GMT] [info] [<0.162.0>] Starting compaction for >> db "users" >> [Mon, 07 Mar 2016 17:41:53 GMT] [info] [<0.1152.0>] 127.0.0.1 - - POST >> /users/_compact 202 >> [Mon, 07 Mar 2016 17:41:53 GMT] [info] [<0.1168.0>] 127.0.0.1 - - POST >> /users/_view_cleanup 202 >> [Mon, 07 Mar 2016 17:41:53 GMT] [info] [<0.162.0>] Compaction for db "users" >> completed. >> [Mon, 07 Mar 2016 17:41:53 GMT] [info] [<0.1329.0>] Starting compaction for >> db "weight" >> [Mon, 07 Mar 2016 17:41:53 GMT] [info] [<0.1170.0>] 127.0.0.1 - - POST >> /weight/_compact 202 >> [Mon, 07 Mar 2016 17:41:53 GMT] [info] [<0.1187.0>] 127.0.0.1 - - POST >> /weight/_view_cleanup 202 >> [Mon, 07 Mar 2016 17:41:56 GMT] [info] [<0.123.0>] 10.1.1.12 - - GET /users/ >> 200 >> [Mon, 07 Mar 2016 17:42:01 GMT] [info] [<0.123.0>] 10.1.1.12 - - GET /users/ >> 200 >> [Mon, 07 Mar 2016 17:42:06 GMT] [info] [<0.123.0>] 10.1.1.12 - - GET /users/ >> 200 >> [Mon, 07 Mar 2016 17:42:11 GMT] [info] [<0.123.0>] 10.1.1.12 - - GET /users/ >> 200 >> >> -------------------------------------------------- >> > >
