@Victor Thanks Victor, sounds like the right thing to do. Will do. On 26 December 2017 at 09:18, Víctor Torre <[email protected]> wrote:
> Hi Sinan, > > I recommend you to destroy the server which has been hacked. If someone > wrote a mining program in it, maybe they've persisted some malicious code > as well (rootkit, etc) > > Definetly, If you detect a database node compromised you must destroy it > and load a backup. > > > > > > > > On Thu, Dec 21, 2017 at 6:27 PM, Sinan Gabel <[email protected]> > wrote: > > > PS Just want to say that by removing the cron job as described the crypto > > mining problem is removed and the 100% CPU problem is resolved. (I have > had > > responses from other users experiencing the exact same problem.) > > > > $ sudo -i > > $ su couchdb > > $ crontab -e > > > > Then remove the line in the cron. If you use the nano editor the command > > are "ctrl-k" to remove lines, and "ctrl-o" to write the file, and > "ctrl-x" > > to exit the crontab editing. And thus the problem should be resolved. > Else > > just write to me. > > > > > > On 11 December 2017 at 17:49, Sinan Gabel <[email protected]> wrote: > > > >> I just found this, it may solve the problem by removing it. It is a > >> script that starts the crypto mining program as user couchdb. > >> > >> [image: Inline images 1] > >> > >> On 11 December 2017 at 15:04, Sinan Gabel <[email protected]> > wrote: > >> > >>> > >>> > >>> I have now checked it, and it is as you @Robert perceived, a crypto > >>> mining program. > >>> > >>> It is a program "fs-manager" with config.json as given below that is > >>> restarted every 3 hours or so (8:41, 11:41 etc.) unless it keeps > running. > >>> > >>> I updated couchdb to latest version and changed all admin passwords but > >>> that has not solved it, so it hidden somewhere on the server. > >>> > >>> Anyone have a clue to how to remove this hack completely? > >>> > >>> > >>> > >>> $ sudo find / -name "fs-manager" > >>> > >>> => > >>> > >>> /var/tmp/.X1M-Unix/fs-manager > >>> > >>> > >>> $ ls -al > >>> > >>> => > >>> > >>> > >>> $ less config.json > >>> > >>> => > >>> > >>> { "algo": "cryptonight", "av": 0, "background": true, "colors": false, > >>> "cpu-affinity": null, "cpu-priority": null, "donate-level": 2, > "log-file": > >>> "xmrig.log", "max-cpu-usage": 85, "print-time": 60, "retries": 2, > >>> "retry-pause": 3, "safe": false, "syslog": false, "threads": null, > "pools": > >>> [ { "url": "pool-proxy.com:8080", "user": "user", "pass": "x", > >>> "keepalive": true, "nicehash": false } ]} > >>> > >>> > >>> > >>> > >>> On 10 December 2017 at 18:11, Robert Samuel Newson <[email protected] > > > >>> wrote: > >>> > >>>> fs-manager is not part of couchdb. You should check that you've not > >>>> been hacked. See https://justi.cz/security/2017 > >>>> /11/14/couchdb-rce-npm.html <https://justi.cz/security/201 > >>>> 7/11/14/couchdb-rce-npm.html>. > >>>> > >>>> I've seen another case where a user's couchdb installation was > >>>> compromised and a bitcoin mining tool installed. This would > (obviously) use > >>>> all your cpu, and I think fs-manager was part of that. > >>>> > >>>> B. > >>>> > >>>> > On 10 Dec 2017, at 17:07, Sinan Gabel <[email protected]> > wrote: > >>>> > > >>>> > Same problem with latest couchdb clustered version. What is the > >>>> process " > >>>> > */.fs-manager*" doing? > >>>> > > >>>> > Googling it returned something about a python package called > >>>> "fs-manager"!? > >>>> > > >>>> > For now I have killed the *couchdb fs-manager* process, and couchdb > >>>> still > >>>> > seems to be working fine. > >>>> > > >>>> > On 10 December 2017 at 16:50, Sinan Gabel <[email protected]> > >>>> wrote: > >>>> > > >>>> >> PS I have just now upgraded to latest couchdb clustered version, I > >>>> will > >>>> >> see if that solves the 100% CPU usage problem. > >>>> >> > >>>> >> On 10 December 2017 at 15:28, Sinan Gabel <[email protected]> > >>>> wrote: > >>>> >> > >>>> >>> Hi, > >>>> >>> > >>>> >>> I do not have a solution but also experiencing the 100% CPU usage. > >>>> Here's > >>>> >>> a .png screen shot of the processes running (my case): > >>>> >>> > >>>> >>> > >>>> >>> On 10 December 2017 at 02:00, Geoffrey Cox <[email protected]> > >>>> wrote: > >>>> >>> > >>>> >>>> Well, I'm back at this and here is the latest info and I think it > >>>> may be > >>>> >>>> related to writes and the _global_changes database: > >>>> >>>> > >>>> >>>> 1. I run my production env and one of my nodes becomes the > >>>> "workhorse" > >>>> >>>> node with 100% CPU > >>>> >>>> 2. I stop all my production code from generating any more > CouchDB > >>>> >>>> requests and eventually the workhorse node goes back to 0% CPU > >>>> >>>> 3. I can then issue writes on a single database (really any > >>>> database > >>>> >>>> and > >>>> >>>> ANY node--not just the workhorse node) and the workhorse node > >>>> will > >>>> >>>> kick > >>>> >>>> back up to 100% CPU. If I stop the writes, the workhorse node > >>>> will > >>>> >>>> return > >>>> >>>> to 0% CPU. > >>>> >>>> 4. And now the punch line: if I delete the _global_changes > >>>> database, > >>>> >>>> the > >>>> >>>> CPU drops down to 0% even if I am issuing writes! Pure cray > cray > >>>> >>>> > >>>> >>>> Any thoughts? > >>>> >>>> > >>>> >>>> (Sorry, still working on a reproducible env for everyone) > >>>> >>>> > >>>> >>>> On Wed, Dec 6, 2017 at 6:56 AM Geoffrey Cox <[email protected]> > >>>> wrote: > >>>> >>>> > >>>> >>>> > >>>> >>>> > >>>> >>> > >>>> >> > >>>> > >>>> > >>> > >> > > > > > -- > [image: Cabify - Your private Driver] <http://www.cabify.com/> > > *Victor Torre* > DevOps > Madrid, Spain > > [email protected] > > > [image: Facebook] <http://cbify.com/fb_ES>[image: Twitter] > <http://cbify.com/tw_ES>[image: Instagram] <http://cbify.com/in_ES> > > -- > Este mensaje y cualquier archivo adjunto va dirigido exclusivamente a su > destinatario, pudiendo contener información confidencial sometida a secreto > profesional. No está permitida su reproducción o distribución sin la > autorización expresa de Cabify. Si usted no es el destinatario final por > favor elimínelo e infórmenos por esta vía. > > This message and any attached file are intended exclusively for the > addressee, and it may be confidential. You are not allowed to copy or > disclose it without Cabify's prior written authorization. If you are not > the intended recipient please delete it from your system and notify us by > e-mail. >
