[
https://issues.apache.org/jira/browse/CARBONDATA-247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravindra Pesala updated CARBONDATA-247:
---------------------------------------
Fix Version/s: 0.1.1-incubating
> Higher MAXCOLUMNS value in load DML options is leading to out of memory error
> -----------------------------------------------------------------------------
>
> Key: CARBONDATA-247
> URL: https://issues.apache.org/jira/browse/CARBONDATA-247
> Project: CarbonData
> Issue Type: Bug
> Components: data-load
> Reporter: dhatchayani
> Priority: Minor
> Fix For: 0.2.0-incubating, 0.1.1-incubating
>
>
> When a higher value lets say Integer max value is configured for maxcolumns
> option in load DML and executor memory is less, then in that case
> UnivocityCsvParser throws an out of memory error when it tries to create an
> array of size of maxColumns option value.
> java.lang.OutOfMemoryError: Java heap space
> at
> com.univocity.parsers.common.ParserOutput.<init>(ParserOutput.java:86)
> at
> com.univocity.parsers.common.AbstractParser.<init>(AbstractParser.java:66)
> at com.univocity.parsers.csv.CsvParser.<init>(CsvParser.java:50)
> at
> org.apache.carbondata.processing.csvreaderstep.UnivocityCsvParser.initialize(UnivocityCsvParser.java:114)
> at
> org.apache.carbondata.processing.csvreaderstep.CsvInput.doProcessUnivocity(CsvInput.java:427)
> at
> org.apache.carbondata.processing.csvreaderstep.CsvInput.access$100(CsvInput.java:60)
> at
> org.apache.carbondata.processing.csvreaderstep.CsvInput$1.call(CsvInput.java:389)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
