[jira] [Commented] (CARBONDATA-1654) NullPointerException when insert overwrite table
[ https://issues.apache.org/jira/browse/CARBONDATA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16226343#comment-16226343 ] Ashwini K commented on CARBONDATA-1654: --- Hi , I created the same tables you shared and tried to insert overwrite . It executed with out any error . I am using hadoop 2.7.2 and spark 2.1 and executed using beeline . From the error you have shared there seem to be ssh configuration problem i had the similar error some time back . Can you check on your ssh configuration? > NullPointerException when insert overwrite table > > > Key: CARBONDATA-1654 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1654 > Project: CarbonData > Issue Type: Bug > Components: data-load >Affects Versions: 1.2.0 > Environment: spark 2.1.1 carbondata 1.2.0 >Reporter: cen yuhai >Priority: Critical > > carbon.sql("insert overwrite table carbondata_table select * from hive_table > where dt = '2017-10-10' ").collect > carbondata wanto find directory Segment_1, but there is Segment_2 > {code} > [Stage 0:> (0 + 504) / > 504]17/10/28 19:11:28 WARN [org.glassfish.jersey.internal.Errors(191) -- > SparkUI-174]: The following warnings have been detected: WARNING: The > (sub)resource method stageData in > org.apache.spark.status.api.v1.OneStageResource contains empty path > annotation. > 17/10/28 19:25:20 ERROR > [org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile(141) > -- main]: main Exception occurred:File does not exist: > hdfs://bipcluster/user/master/carbon/store/dm_test/carbondata_table/Fact/Part0/Segment_1 > 17/10/28 19:25:22 ERROR > [org.apache.spark.sql.execution.command.LoadTable(143) -- main]: main > java.lang.NullPointerException > at > org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.isDirectory(AbstractDFSCarbonFile.java:88) > at > org.apache.carbondata.core.util.CarbonUtil.deleteRecursive(CarbonUtil.java:364) > at > org.apache.carbondata.core.util.CarbonUtil.access$100(CarbonUtil.java:93) > at > org.apache.carbondata.core.util.CarbonUtil$2.run(CarbonUtil.java:326) > at > org.apache.carbondata.core.util.CarbonUtil$2.run(CarbonUtil.java:322) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) > at > org.apache.carbondata.core.util.CarbonUtil.deleteFoldersAndFiles(CarbonUtil.java:322) > at > org.apache.carbondata.spark.load.CarbonLoaderUtil.recordLoadMetadata(CarbonLoaderUtil.java:331) > at > org.apache.carbondata.spark.rdd.CarbonDataRDDFactory$.updateStatus$1(CarbonDataRDDFactory.scala:595) > at > org.apache.carbondata.spark.rdd.CarbonDataRDDFactory$.loadCarbonData(CarbonDataRDDFactory.scala:1107) > at > org.apache.spark.sql.execution.command.LoadTable.processData(carbonTableSchema.scala:1046) > at > org.apache.spark.sql.execution.command.LoadTable.run(carbonTableSchema.scala:754) > at > org.apache.spark.sql.execution.command.LoadTableByInsert.processData(carbonTableSchema.scala:651) > at > org.apache.spark.sql.execution.command.LoadTableByInsert.run(carbonTableSchema.scala:637) > at > org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58) > at > org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56) > at > org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:67) > at org.apache.spark.sql.Dataset.(Dataset.scala:180) > at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65) > at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:619) > at > $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:36) > at > $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:41) > at $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:43) > at $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:45) > at $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw.(:47) > at $line23.$read$$iw$$iw$$iw$$iw$$iw.(:49) > at $line23.$read$$iw$$iw$$iw$$iw.(:51) > at $line23.$read$$iw$$iw$$iw.(:53) > at $line23.$read$$iw$$iw.(:55) > at $line23.$read$$iw.(:57) > at $line23.$read.(:59) > at $line23.$read$.(:63) > at $line23.$read$.() > at $line23.$eval$.$print$lzycompute(:7) > at $line23.$eval$.$print(:6) > at $line23.$eval.$print() > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > a
[jira] [Commented] (CARBONDATA-1654) NullPointerException when insert overwrite table
[ https://issues.apache.org/jira/browse/CARBONDATA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16226131#comment-16226131 ] cen yuhai commented on CARBONDATA-1654: --- I update the description > NullPointerException when insert overwrite table > > > Key: CARBONDATA-1654 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1654 > Project: CarbonData > Issue Type: Bug > Components: data-load >Affects Versions: 1.2.0 > Environment: spark 2.1.1 carbondata 1.2.0 >Reporter: cen yuhai >Priority: Critical > > carbon.sql("insert overwrite table carbondata_table select * from hive_table > where dt = '2017-10-10' ").collect > carbondata wanto find directory Segment_1, but there is Segment_2 > {code} > [Stage 0:> (0 + 504) / > 504]17/10/28 19:11:28 WARN [org.glassfish.jersey.internal.Errors(191) -- > SparkUI-174]: The following warnings have been detected: WARNING: The > (sub)resource method stageData in > org.apache.spark.status.api.v1.OneStageResource contains empty path > annotation. > 17/10/28 19:25:20 ERROR > [org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile(141) > -- main]: main Exception occurred:File does not exist: > hdfs://bipcluster/user/master/carbon/store/dm_test/carbondata_table/Fact/Part0/Segment_1 > 17/10/28 19:25:22 ERROR > [org.apache.spark.sql.execution.command.LoadTable(143) -- main]: main > java.lang.NullPointerException > at > org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.isDirectory(AbstractDFSCarbonFile.java:88) > at > org.apache.carbondata.core.util.CarbonUtil.deleteRecursive(CarbonUtil.java:364) > at > org.apache.carbondata.core.util.CarbonUtil.access$100(CarbonUtil.java:93) > at > org.apache.carbondata.core.util.CarbonUtil$2.run(CarbonUtil.java:326) > at > org.apache.carbondata.core.util.CarbonUtil$2.run(CarbonUtil.java:322) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) > at > org.apache.carbondata.core.util.CarbonUtil.deleteFoldersAndFiles(CarbonUtil.java:322) > at > org.apache.carbondata.spark.load.CarbonLoaderUtil.recordLoadMetadata(CarbonLoaderUtil.java:331) > at > org.apache.carbondata.spark.rdd.CarbonDataRDDFactory$.updateStatus$1(CarbonDataRDDFactory.scala:595) > at > org.apache.carbondata.spark.rdd.CarbonDataRDDFactory$.loadCarbonData(CarbonDataRDDFactory.scala:1107) > at > org.apache.spark.sql.execution.command.LoadTable.processData(carbonTableSchema.scala:1046) > at > org.apache.spark.sql.execution.command.LoadTable.run(carbonTableSchema.scala:754) > at > org.apache.spark.sql.execution.command.LoadTableByInsert.processData(carbonTableSchema.scala:651) > at > org.apache.spark.sql.execution.command.LoadTableByInsert.run(carbonTableSchema.scala:637) > at > org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58) > at > org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56) > at > org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:67) > at org.apache.spark.sql.Dataset.(Dataset.scala:180) > at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65) > at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:619) > at > $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:36) > at > $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:41) > at $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:43) > at $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:45) > at $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw.(:47) > at $line23.$read$$iw$$iw$$iw$$iw$$iw.(:49) > at $line23.$read$$iw$$iw$$iw$$iw.(:51) > at $line23.$read$$iw$$iw$$iw.(:53) > at $line23.$read$$iw$$iw.(:55) > at $line23.$read$$iw.(:57) > at $line23.$read.(:59) > at $line23.$read$.(:63) > at $line23.$read$.() > at $line23.$eval$.$print$lzycompute(:7) > at $line23.$eval$.$print(:6) > at $line23.$eval.$print() > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at > scala.tools.nsc.interpreter.IMain$ReadEvalPrint
[jira] [Commented] (CARBONDATA-1654) NullPointerException when insert overwrite table
[ https://issues.apache.org/jira/browse/CARBONDATA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16224862#comment-16224862 ] Ashwini K commented on CARBONDATA-1654: --- pls share the table definition > NullPointerException when insert overwrite table > > > Key: CARBONDATA-1654 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1654 > Project: CarbonData > Issue Type: Bug > Components: data-load >Affects Versions: 1.2.0 > Environment: spark 2.1.1 carbondata 1.2.0 >Reporter: cen yuhai >Priority: Critical > > carbon.sql("insert overwrite table carbondata_table select * from hive_table > where dt = '2017-10-10' ").collect > carbondata wanto find directory Segment_1, but there is Segment_2 > {code} > [Stage 0:> (0 + 504) / > 504]17/10/28 19:11:28 WARN [org.glassfish.jersey.internal.Errors(191) -- > SparkUI-174]: The following warnings have been detected: WARNING: The > (sub)resource method stageData in > org.apache.spark.status.api.v1.OneStageResource contains empty path > annotation. > 17/10/28 19:25:20 ERROR > [org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile(141) > -- main]: main Exception occurred:File does not exist: > hdfs://bipcluster/user/master/carbon/store/dm_test/carbondata_table/Fact/Part0/Segment_1 > 17/10/28 19:25:22 ERROR > [org.apache.spark.sql.execution.command.LoadTable(143) -- main]: main > java.lang.NullPointerException > at > org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.isDirectory(AbstractDFSCarbonFile.java:88) > at > org.apache.carbondata.core.util.CarbonUtil.deleteRecursive(CarbonUtil.java:364) > at > org.apache.carbondata.core.util.CarbonUtil.access$100(CarbonUtil.java:93) > at > org.apache.carbondata.core.util.CarbonUtil$2.run(CarbonUtil.java:326) > at > org.apache.carbondata.core.util.CarbonUtil$2.run(CarbonUtil.java:322) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) > at > org.apache.carbondata.core.util.CarbonUtil.deleteFoldersAndFiles(CarbonUtil.java:322) > at > org.apache.carbondata.spark.load.CarbonLoaderUtil.recordLoadMetadata(CarbonLoaderUtil.java:331) > at > org.apache.carbondata.spark.rdd.CarbonDataRDDFactory$.updateStatus$1(CarbonDataRDDFactory.scala:595) > at > org.apache.carbondata.spark.rdd.CarbonDataRDDFactory$.loadCarbonData(CarbonDataRDDFactory.scala:1107) > at > org.apache.spark.sql.execution.command.LoadTable.processData(carbonTableSchema.scala:1046) > at > org.apache.spark.sql.execution.command.LoadTable.run(carbonTableSchema.scala:754) > at > org.apache.spark.sql.execution.command.LoadTableByInsert.processData(carbonTableSchema.scala:651) > at > org.apache.spark.sql.execution.command.LoadTableByInsert.run(carbonTableSchema.scala:637) > at > org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58) > at > org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56) > at > org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:67) > at org.apache.spark.sql.Dataset.(Dataset.scala:180) > at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65) > at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:619) > at > $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:36) > at > $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:41) > at $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:43) > at $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:45) > at $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw.(:47) > at $line23.$read$$iw$$iw$$iw$$iw$$iw.(:49) > at $line23.$read$$iw$$iw$$iw$$iw.(:51) > at $line23.$read$$iw$$iw$$iw.(:53) > at $line23.$read$$iw$$iw.(:55) > at $line23.$read$$iw.(:57) > at $line23.$read.(:59) > at $line23.$read$.(:63) > at $line23.$read$.() > at $line23.$eval$.$print$lzycompute(:7) > at $line23.$eval$.$print(:6) > at $line23.$eval.$print() > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at > scala.tools.nsc.interpreter.IMain$ReadE
[jira] [Commented] (CARBONDATA-1654) NullPointerException when insert overwrite table
[ https://issues.apache.org/jira/browse/CARBONDATA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16224453#comment-16224453 ] cen yuhai commented on CARBONDATA-1654: --- No, I can't. Why I should upate schema? > NullPointerException when insert overwrite table > > > Key: CARBONDATA-1654 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1654 > Project: CarbonData > Issue Type: Bug > Components: data-load >Affects Versions: 1.2.0 > Environment: spark 2.1.1 carbondata 1.2.0 >Reporter: cen yuhai >Priority: Critical > > carbon.sql("insert overwrite table carbondata_table select * from hive_table > where dt = '2017-10-10' ").collect > carbondata wanto find directory Segment_1, but there is Segment_2 > {code} > [Stage 0:> (0 + 504) / > 504]17/10/28 19:11:28 WARN [org.glassfish.jersey.internal.Errors(191) -- > SparkUI-174]: The following warnings have been detected: WARNING: The > (sub)resource method stageData in > org.apache.spark.status.api.v1.OneStageResource contains empty path > annotation. > 17/10/28 19:25:20 ERROR > [org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile(141) > -- main]: main Exception occurred:File does not exist: > hdfs://bipcluster/user/master/carbon/store/dm_test/carbondata_table/Fact/Part0/Segment_1 > 17/10/28 19:25:22 ERROR > [org.apache.spark.sql.execution.command.LoadTable(143) -- main]: main > java.lang.NullPointerException > at > org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.isDirectory(AbstractDFSCarbonFile.java:88) > at > org.apache.carbondata.core.util.CarbonUtil.deleteRecursive(CarbonUtil.java:364) > at > org.apache.carbondata.core.util.CarbonUtil.access$100(CarbonUtil.java:93) > at > org.apache.carbondata.core.util.CarbonUtil$2.run(CarbonUtil.java:326) > at > org.apache.carbondata.core.util.CarbonUtil$2.run(CarbonUtil.java:322) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) > at > org.apache.carbondata.core.util.CarbonUtil.deleteFoldersAndFiles(CarbonUtil.java:322) > at > org.apache.carbondata.spark.load.CarbonLoaderUtil.recordLoadMetadata(CarbonLoaderUtil.java:331) > at > org.apache.carbondata.spark.rdd.CarbonDataRDDFactory$.updateStatus$1(CarbonDataRDDFactory.scala:595) > at > org.apache.carbondata.spark.rdd.CarbonDataRDDFactory$.loadCarbonData(CarbonDataRDDFactory.scala:1107) > at > org.apache.spark.sql.execution.command.LoadTable.processData(carbonTableSchema.scala:1046) > at > org.apache.spark.sql.execution.command.LoadTable.run(carbonTableSchema.scala:754) > at > org.apache.spark.sql.execution.command.LoadTableByInsert.processData(carbonTableSchema.scala:651) > at > org.apache.spark.sql.execution.command.LoadTableByInsert.run(carbonTableSchema.scala:637) > at > org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58) > at > org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56) > at > org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:67) > at org.apache.spark.sql.Dataset.(Dataset.scala:180) > at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65) > at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:619) > at > $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:36) > at > $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:41) > at $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:43) > at $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:45) > at $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw.(:47) > at $line23.$read$$iw$$iw$$iw$$iw$$iw.(:49) > at $line23.$read$$iw$$iw$$iw$$iw.(:51) > at $line23.$read$$iw$$iw$$iw.(:53) > at $line23.$read$$iw$$iw.(:55) > at $line23.$read$$iw.(:57) > at $line23.$read.(:59) > at $line23.$read$.(:63) > at $line23.$read$.() > at $line23.$eval$.$print$lzycompute(:7) > at $line23.$eval$.$print(:6) > at $line23.$eval.$print() > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at > scala.tools.nsc.interpreter.IMai
[jira] [Commented] (CARBONDATA-1654) NullPointerException when insert overwrite table
[ https://issues.apache.org/jira/browse/CARBONDATA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16224449#comment-16224449 ] Ashwini K commented on CARBONDATA-1654: --- hi Can you please update carbondata_table schema ? > NullPointerException when insert overwrite table > > > Key: CARBONDATA-1654 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1654 > Project: CarbonData > Issue Type: Bug > Components: data-load >Affects Versions: 1.2.0 > Environment: spark 2.1.1 carbondata 1.2.0 >Reporter: cen yuhai >Priority: Critical > > carbon.sql("insert overwrite table carbondata_table select * from hive_table > where dt = '2017-10-10' ").collect > carbondata wanto find directory Segment_1, but there is Segment_2 > {code} > [Stage 0:> (0 + 504) / > 504]17/10/28 19:11:28 WARN [org.glassfish.jersey.internal.Errors(191) -- > SparkUI-174]: The following warnings have been detected: WARNING: The > (sub)resource method stageData in > org.apache.spark.status.api.v1.OneStageResource contains empty path > annotation. > 17/10/28 19:25:20 ERROR > [org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile(141) > -- main]: main Exception occurred:File does not exist: > hdfs://bipcluster/user/master/carbon/store/dm_test/carbondata_table/Fact/Part0/Segment_1 > 17/10/28 19:25:22 ERROR > [org.apache.spark.sql.execution.command.LoadTable(143) -- main]: main > java.lang.NullPointerException > at > org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.isDirectory(AbstractDFSCarbonFile.java:88) > at > org.apache.carbondata.core.util.CarbonUtil.deleteRecursive(CarbonUtil.java:364) > at > org.apache.carbondata.core.util.CarbonUtil.access$100(CarbonUtil.java:93) > at > org.apache.carbondata.core.util.CarbonUtil$2.run(CarbonUtil.java:326) > at > org.apache.carbondata.core.util.CarbonUtil$2.run(CarbonUtil.java:322) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) > at > org.apache.carbondata.core.util.CarbonUtil.deleteFoldersAndFiles(CarbonUtil.java:322) > at > org.apache.carbondata.spark.load.CarbonLoaderUtil.recordLoadMetadata(CarbonLoaderUtil.java:331) > at > org.apache.carbondata.spark.rdd.CarbonDataRDDFactory$.updateStatus$1(CarbonDataRDDFactory.scala:595) > at > org.apache.carbondata.spark.rdd.CarbonDataRDDFactory$.loadCarbonData(CarbonDataRDDFactory.scala:1107) > at > org.apache.spark.sql.execution.command.LoadTable.processData(carbonTableSchema.scala:1046) > at > org.apache.spark.sql.execution.command.LoadTable.run(carbonTableSchema.scala:754) > at > org.apache.spark.sql.execution.command.LoadTableByInsert.processData(carbonTableSchema.scala:651) > at > org.apache.spark.sql.execution.command.LoadTableByInsert.run(carbonTableSchema.scala:637) > at > org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58) > at > org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56) > at > org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:67) > at org.apache.spark.sql.Dataset.(Dataset.scala:180) > at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65) > at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:619) > at > $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:36) > at > $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:41) > at $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:43) > at $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:45) > at $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw.(:47) > at $line23.$read$$iw$$iw$$iw$$iw$$iw.(:49) > at $line23.$read$$iw$$iw$$iw$$iw.(:51) > at $line23.$read$$iw$$iw$$iw.(:53) > at $line23.$read$$iw$$iw.(:55) > at $line23.$read$$iw.(:57) > at $line23.$read.(:59) > at $line23.$read$.(:63) > at $line23.$read$.() > at $line23.$eval$.$print$lzycompute(:7) > at $line23.$eval$.$print(:6) > at $line23.$eval.$print() > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at > scala.tools.nsc.inter
[jira] [Commented] (CARBONDATA-1654) NullPointerException when insert overwrite table
[ https://issues.apache.org/jira/browse/CARBONDATA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16224446#comment-16224446 ] Ashwini K commented on CARBONDATA-1654: --- hi Can you please update carbondata_table schema ? > NullPointerException when insert overwrite table > > > Key: CARBONDATA-1654 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1654 > Project: CarbonData > Issue Type: Bug > Components: data-load >Affects Versions: 1.2.0 > Environment: spark 2.1.1 carbondata 1.2.0 >Reporter: cen yuhai >Priority: Critical > > carbon.sql("insert overwrite table carbondata_table select * from hive_table > where dt = '2017-10-10' ").collect > carbondata wanto find directory Segment_1, but there is Segment_2 > {code} > [Stage 0:> (0 + 504) / > 504]17/10/28 19:11:28 WARN [org.glassfish.jersey.internal.Errors(191) -- > SparkUI-174]: The following warnings have been detected: WARNING: The > (sub)resource method stageData in > org.apache.spark.status.api.v1.OneStageResource contains empty path > annotation. > 17/10/28 19:25:20 ERROR > [org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile(141) > -- main]: main Exception occurred:File does not exist: > hdfs://bipcluster/user/master/carbon/store/dm_test/carbondata_table/Fact/Part0/Segment_1 > 17/10/28 19:25:22 ERROR > [org.apache.spark.sql.execution.command.LoadTable(143) -- main]: main > java.lang.NullPointerException > at > org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.isDirectory(AbstractDFSCarbonFile.java:88) > at > org.apache.carbondata.core.util.CarbonUtil.deleteRecursive(CarbonUtil.java:364) > at > org.apache.carbondata.core.util.CarbonUtil.access$100(CarbonUtil.java:93) > at > org.apache.carbondata.core.util.CarbonUtil$2.run(CarbonUtil.java:326) > at > org.apache.carbondata.core.util.CarbonUtil$2.run(CarbonUtil.java:322) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) > at > org.apache.carbondata.core.util.CarbonUtil.deleteFoldersAndFiles(CarbonUtil.java:322) > at > org.apache.carbondata.spark.load.CarbonLoaderUtil.recordLoadMetadata(CarbonLoaderUtil.java:331) > at > org.apache.carbondata.spark.rdd.CarbonDataRDDFactory$.updateStatus$1(CarbonDataRDDFactory.scala:595) > at > org.apache.carbondata.spark.rdd.CarbonDataRDDFactory$.loadCarbonData(CarbonDataRDDFactory.scala:1107) > at > org.apache.spark.sql.execution.command.LoadTable.processData(carbonTableSchema.scala:1046) > at > org.apache.spark.sql.execution.command.LoadTable.run(carbonTableSchema.scala:754) > at > org.apache.spark.sql.execution.command.LoadTableByInsert.processData(carbonTableSchema.scala:651) > at > org.apache.spark.sql.execution.command.LoadTableByInsert.run(carbonTableSchema.scala:637) > at > org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58) > at > org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56) > at > org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:67) > at org.apache.spark.sql.Dataset.(Dataset.scala:180) > at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65) > at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:619) > at > $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:36) > at > $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:41) > at $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:43) > at $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:45) > at $line23.$read$$iw$$iw$$iw$$iw$$iw$$iw.(:47) > at $line23.$read$$iw$$iw$$iw$$iw$$iw.(:49) > at $line23.$read$$iw$$iw$$iw$$iw.(:51) > at $line23.$read$$iw$$iw$$iw.(:53) > at $line23.$read$$iw$$iw.(:55) > at $line23.$read$$iw.(:57) > at $line23.$read.(:59) > at $line23.$read$.(:63) > at $line23.$read$.() > at $line23.$eval$.$print$lzycompute(:7) > at $line23.$eval$.$print(:6) > at $line23.$eval.$print() > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at > scala.tools.nsc.inter