Saqeeb Shaikh created ATLAS-3525: ------------------------------------ Summary: Hive server runs into OOM with huge heap trace of atlas logger. Key: ATLAS-3525 URL: https://issues.apache.org/jira/browse/ATLAS-3525 Project: Atlas Issue Type: Bug Affects Versions: 0.8.1 Reporter: Saqeeb Shaikh Assignee: Saqeeb Shaikh
Atlas hook produces large message on Kafka, which Kafka fails with RecordTooLargeException :: {code:java} hook.AtlasHook (AtlasHook.java:notifyEntitiesInternal(138)) - Failed to send notification - attempt #1; error=java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.RecordTooLargeException: The message is 625562624 bytes when serialized which is larger than the maximum request size you have configured with the max.request.size configuration.{code} The failed messages are logged in a log file, and thus Atlas's logger has a huge trace on the heap. Atlas hook performs an in memory compression of large messages ( handled in https://issues.apache.org/jira/browse/ATLAS-2075 ). But due to insufficient heap space available this in memory compression fails. The purpose of this Jira is to figure out a way to handle this compression in a manner that it doesn't spikes the heap usage. Temporary work-around for this can be increasing the heap size to double or triple or not logging the failed message, instead just log the size of the failed message. -- This message was sent by Atlassian Jira (v8.3.4#803005)