New question #252717 on Graphite: https://answers.launchpad.net/graphite/+question/252717
Hi all, I have one host setup as my graphite installation (one relay, 2 writers). Everything was working peachy until I increased the number of open files allowed for the account running the carbon daemons to open more files (32k). I was getting EMFILE in my relay console.log, hence the change. After I restart the carbon daemons (relay and two writers), after about 20 minutes (metrics are written to disk fine), zero metrics are written, and I cannot see them in the web interface. The relay is still receiving metrics (verified via strace), but I don't see the writers writing any data. I've enabled write logging on the writers, and they even say that writes are being made to the writer logfile. However, if I use whisper-dump.py, the last recorded metric is within that 20 minute window mentioned above. I monitor the disks with iostat, and after that 20 minute period, everything goes dark. I am willing to bet this has to do with open files, but I'm just stuck on what to look for, or better ways to debug the carbon relay/writers. Anyone have any idea? THanks! -- You received this question notification because you are a member of graphite-dev, which is an answer contact for Graphite. _______________________________________________ Mailing list: https://launchpad.net/~graphite-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~graphite-dev More help : https://help.launchpad.net/ListHelp
