Question #242463 on Graphite changed: https://answers.launchpad.net/graphite/+question/242463
Sumeet Grover posted a new comment: Hi All, Thanks for your comments, and especially the pointer to MTU. I've checked the MTU for our ethernet sockets on the Graphite server is 1500 bytes, and the time the metrics got corrupted, there were more than 2,000,000 committed points in Carbon every second. However, our rmem_max kernel parameter is around 220KiB, effectively around 110KiB. We are running Carbon 0.9.10. I ran two simulation tests to find out if it is Carbon, which corrupts the metrics. The answer was NO. Since then we have reverted back to the theory that the problem is highly likely to be with the processes that feed into Graphite. Below are my test results, which proved that given a high volume of messages or messages with invalid values, Carbon actually does not distort them: SIMULATION TEST 1 Duration: 1h30m Metrics Received by Carbon: Over 2,000,000 Committed Points: Over 150,000 Cache Queues: Over 45,000 RAM Usage: Over 200MiB Test metrics were sent with valid values, i.e. INT/DECIMAL values. SIMULATION TEST 2 Duration: 1h30m Metrics Received by Carbon: Over 300,000 Committed Points: Over 100,000 Cache Queues: Over 45,000 RAM Usage: Over 200MiB Test metrics were sent with random values which were invalid, i.e. alpha numeric. Carbon ignored these values, proving our initial theory wrong - i.e. an invalid values leads to distortion of metrics in Carbon. -- You received this question notification because you are a member of graphite-dev, which is an answer contact for Graphite. _______________________________________________ Mailing list: https://launchpad.net/~graphite-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~graphite-dev More help : https://help.launchpad.net/ListHelp
