[ https://issues.apache.org/jira/browse/CARBONDATA-1522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manish Gupta resolved CARBONDATA-1522. -------------------------------------- Resolution: Fixed Fix Version/s: 1.4.0 > 6. Loading aggregation tables for streaming data tables. > -------------------------------------------------------- > > Key: CARBONDATA-1522 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1522 > Project: CarbonData > Issue Type: Sub-task > Reporter: Ravindra Pesala > Assignee: Kunal Kapoor > Priority: Major > Fix For: 1.4.0 > > Time Spent: 9h 40m > Remaining Estimate: 0h > > we can finish the segment load after receives configurable amount of data > and create a new segment to load new streaming data. While finishing the > segment we can trigger the agg tables on it. So there would not be any agg > tables on ongoing streaming segment but querying can be done on the streaming > segment of actual table. > For example user configures stream_segment size as 1 GB, so for every 1 GB of > stream data it receives it creates a new segment and finishes the current > segment. While finishing the current segment we can trigger agg table loading > and compaction of segments. > While querying of data we change the query plan to apply union of agg table > and streaming segment of actual table to get the current data. -- This message was sent by Atlassian JIRA (v7.6.3#76005)