[jira] [Updated] (CARBONDATA-247) Higher MAXCOLUMNS value in load DML options is leading to out of memory error
[ https://issues.apache.org/jira/browse/CARBONDATA-247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Ramana G updated CARBONDATA-247: Component/s: data-load > Higher MAXCOLUMNS value in load DML options is leading to out of memory error > - > > Key: CARBONDATA-247 > URL: https://issues.apache.org/jira/browse/CARBONDATA-247 > Project: CarbonData > Issue Type: Bug > Components: data-load >Reporter: dhatchayani >Priority: Minor > Fix For: 0.2.0-incubating > > > When a higher value lets say Integer max value is configured for maxcolumns > option in load DML and executor memory is less, then in that case > UnivocityCsvParser throws an out of memory error when it tries to create an > array of size of maxColumns option value. > java.lang.OutOfMemoryError: Java heap space > at > com.univocity.parsers.common.ParserOutput.(ParserOutput.java:86) > at > com.univocity.parsers.common.AbstractParser.(AbstractParser.java:66) > at com.univocity.parsers.csv.CsvParser.(CsvParser.java:50) > at > org.apache.carbondata.processing.csvreaderstep.UnivocityCsvParser.initialize(UnivocityCsvParser.java:114) > at > org.apache.carbondata.processing.csvreaderstep.CsvInput.doProcessUnivocity(CsvInput.java:427) > at > org.apache.carbondata.processing.csvreaderstep.CsvInput.access$100(CsvInput.java:60) > at > org.apache.carbondata.processing.csvreaderstep.CsvInput$1.call(CsvInput.java:389) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CARBONDATA-247) Higher MAXCOLUMNS value in load DML options is leading to out of memory error
[ https://issues.apache.org/jira/browse/CARBONDATA-247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala updated CARBONDATA-247: --- Fix Version/s: 0.1.1-incubating > Higher MAXCOLUMNS value in load DML options is leading to out of memory error > - > > Key: CARBONDATA-247 > URL: https://issues.apache.org/jira/browse/CARBONDATA-247 > Project: CarbonData > Issue Type: Bug > Components: data-load >Reporter: dhatchayani >Priority: Minor > Fix For: 0.2.0-incubating, 0.1.1-incubating > > > When a higher value lets say Integer max value is configured for maxcolumns > option in load DML and executor memory is less, then in that case > UnivocityCsvParser throws an out of memory error when it tries to create an > array of size of maxColumns option value. > java.lang.OutOfMemoryError: Java heap space > at > com.univocity.parsers.common.ParserOutput.(ParserOutput.java:86) > at > com.univocity.parsers.common.AbstractParser.(AbstractParser.java:66) > at com.univocity.parsers.csv.CsvParser.(CsvParser.java:50) > at > org.apache.carbondata.processing.csvreaderstep.UnivocityCsvParser.initialize(UnivocityCsvParser.java:114) > at > org.apache.carbondata.processing.csvreaderstep.CsvInput.doProcessUnivocity(CsvInput.java:427) > at > org.apache.carbondata.processing.csvreaderstep.CsvInput.access$100(CsvInput.java:60) > at > org.apache.carbondata.processing.csvreaderstep.CsvInput$1.call(CsvInput.java:389) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
