[ https://issues.apache.org/jira/browse/CARBONDATA-241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15493819#comment-15493819 ]
ASF GitHub Bot commented on CARBONDATA-241: ------------------------------------------- Github user gvramana commented on a diff in the pull request: https://github.com/apache/incubator-carbondata/pull/158#discussion_r79003748 --- Diff: processing/src/main/java/org/apache/carbondata/lcm/status/SegmentStatusManager.java --- @@ -102,6 +91,60 @@ public long getTableStatusLastModifiedTime() throws IOException { /** * get valid segment for given table + * + * @return + * @throws IOException + */ + public InvalidSegmentsInfo getInvalidSegments() throws IOException { --- End diff -- This requires reading SegmentInfo twice, once for valid blocks and next for Invalid Blocks. Instead send a single class containting ValidAndInvalidBlocks. > OOM error during query execution in long run > -------------------------------------------- > > Key: CARBONDATA-241 > URL: https://issues.apache.org/jira/browse/CARBONDATA-241 > Project: CarbonData > Issue Type: Bug > Reporter: kumar vishal > Assignee: kumar vishal > > **Problem:** During long run query execution is taking more time and it is > throwing out of memory issue. > **Reason**: In compaction we are compacting segments and each segment > metadata is loaded in memory. So after compaction compacted segments are > invalid but its meta data is not removed from memory because of this > duplicate metadata is pile up and it is taking more memory and after few days > query exeution is throwing OOM > **Solution**: Need to remove invalid blocks from memory > -- This message was sent by Atlassian JIRA (v6.3.4#6332)