[
https://issues.apache.org/jira/browse/CARBONDATA-1109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yadong Qi updated CARBONDATA-1109:
----------------------------------
Affects Version/s: 1.2.0
> Page lost in load process when last page is not be consumed at the end
> ----------------------------------------------------------------------
>
> Key: CARBONDATA-1109
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1109
> Project: CarbonData
> Issue Type: Bug
> Affects Versions: 1.2.0
> Reporter: Yadong Qi
>
> First, we use Producer-Consumer model in the write step, we have n(default
> value is 2 and can be configured) producers and one consumer. The task of
> generate last page(less than 32000) is added to thread pool at the end, but
> can't be guaranteed to be finished and add to BlockletDataHolder at the end.
> Because we have n tasks running concurrently.
> Second, we have 2 ways to invoke `writeDataToFile`, one is the size of
> `DataWriterHolder` reach the size of blocklet and two is page is the last
> page.
> So if the last page is not be consumed at the end, we lost the page which be
> consumed after last page.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
