[ https://issues.apache.org/jira/browse/CARBONDATA-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16146975#comment-16146975 ]
Pallavi Singh commented on CARBONDATA-1422: ------------------------------------------- Hi Zhichao Zhang, Here is the stack-trace of the error: 17/08/30 15:12:30 INFO CatalystSqlParser: Parsing command: array<string> 17/08/30 15:12:30 AUDIT CarbonDataRDDFactory$: [pallavi][hduser][Thread-136]Compaction request received for table 30aug_5.uniqdata 17/08/30 15:12:30 INFO HdfsFileLock: pool-24-thread-7 HDFS lock path:hdfs://localhost:54310/user/hive/warehouse/carbon.store/30aug_5/uniqdata/compaction.lock 17/08/30 15:12:30 INFO CarbonDataRDDFactory$: pool-24-thread-7 Acquired the compaction lock for table 30aug_5.uniqdata 17/08/30 15:12:30 INFO DataManagementFunc$: pool-24-thread-7 loads identified for merge is 0 17/08/30 15:12:30 INFO DataManagementFunc$: pool-24-thread-7 loads identified for merge is 1 17/08/30 15:12:30 INFO DataManagementFunc$: pool-24-thread-7 loads identified for merge is 2 17/08/30 15:12:30 INFO DataManagementFunc$: pool-24-thread-7 loads identified for merge is 3 17/08/30 15:12:30 INFO DataManagementFunc$: pool-24-thread-7 loads identified for merge is 4 17/08/30 15:12:30 INFO DataManagementFunc$: pool-24-thread-7 loads identified for merge is 5 17/08/30 15:12:30 INFO DataManagementFunc$: pool-24-thread-7 loads identified for merge is 6 17/08/30 15:12:30 INFO DataManagementFunc$: pool-24-thread-7 loads identified for merge is 7 17/08/30 15:12:30 INFO TableInfo: pool-28-thread-1 Table block size not specified for 30aug_5_uniqdata. Therefore considering the default value 1024 MB 17/08/30 15:12:30 INFO DistributionUtil$: pool-28-thread-1 Requesting total executors: 1 17/08/30 15:12:30 INFO CarbonMergerRDD: No.of Executors required=1 , spark.executor.instances=1, no.of.nodes where data present=1 17/08/30 15:12:30 INFO CarbonMergerRDD: Time taken to wait for executor allocation is =0millis 17/08/30 15:12:30 INFO CarbonMergerRDD: for the node pallavi 17/08/30 15:12:30 INFO CarbonMergerRDD: Task ID is 0no. of blocks is 8 17/08/30 15:12:30 INFO CarbonMergerRDD: Identified no.of.Blocks: 8,parallelism: 4 , no.of.nodes: 1, no.of.tasks: 1 17/08/30 15:12:30 INFO CarbonMergerRDD: Time taken to identify Blocks to scan : 177 17/08/30 15:12:30 INFO CarbonMergerRDD: Node: pallavi, No.Of Blocks: 8 17/08/30 15:12:30 INFO SparkContext: Starting job: collect at Compactor.scala:100 17/08/30 15:12:30 INFO DAGScheduler: Got job 4 (collect at Compactor.scala:100) with 1 output partitions 17/08/30 15:12:30 INFO DAGScheduler: Final stage: ResultStage 4 (collect at Compactor.scala:100) 17/08/30 15:12:30 INFO DAGScheduler: Parents of final stage: List() 17/08/30 15:12:30 INFO DAGScheduler: Missing parents: List() 17/08/30 15:12:30 INFO DAGScheduler: Submitting ResultStage 4 (CarbonMergerRDD[11] at RDD at CarbonRDD.scala:35), which has no missing parents 17/08/30 15:12:30 INFO MemoryStore: Block broadcast_6 stored as values in memory (estimated size 14.2 KB, free 366.2 MB) 17/08/30 15:12:30 INFO MemoryStore: Block broadcast_6_piece0 stored as bytes in memory (estimated size 6.7 KB, free 366.2 MB) 17/08/30 15:12:30 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on 192.168.2.188:41587 (size: 6.7 KB, free: 366.3 MB) 17/08/30 15:12:30 INFO SparkContext: Created broadcast 6 from broadcast at DAGScheduler.scala:996 17/08/30 15:12:30 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 4 (CarbonMergerRDD[11] at RDD at CarbonRDD.scala:35) 17/08/30 15:12:30 INFO TaskSchedulerImpl: Adding task set 4.0 with 1 tasks 17/08/30 15:12:30 INFO TaskSetManager: Starting task 0.0 in stage 4.0 (TID 4, 192.168.2.188, executor 0, partition 0, ANY, 9296 bytes) 17/08/30 15:12:30 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on 192.168.2.188:38477 (size: 6.7 KB, free: 366.3 MB) 17/08/30 15:12:31 INFO TaskSetManager: Finished task 0.0 in stage 4.0 (TID 4) in 266 ms on 192.168.2.188 (executor 0) (1/1) 17/08/30 15:12:31 INFO TaskSchedulerImpl: Removed TaskSet 4.0, whose tasks have all completed, from pool 17/08/30 15:12:31 INFO DAGScheduler: ResultStage 4 (collect at Compactor.scala:100) finished in 0.267 s 17/08/30 15:12:31 INFO DAGScheduler: Job 4 finished: collect at Compactor.scala:100, took 0.276120 s 17/08/30 15:12:31 AUDIT Compactor$: [pallavi][hduser][Thread-143]Compaction request failed for table 30aug_5.uniqdata 17/08/30 15:12:31 ERROR Compactor$: pool-28-thread-1 Compaction request failed for table 30aug_5.uniqdata 17/08/30 15:12:31 ERROR DataManagementFunc$: pool-24-thread-7 Exception in compaction thread java.lang.Exception: Compaction Failure in Merger Rdd. java.util.concurrent.ExecutionException: java.lang.Exception: Compaction Failure in Merger Rdd. at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at org.apache.carbondata.spark.rdd.DataManagementFunc$$anonfun$executeCompaction$1.apply(DataManagementFunc.scala:193) at org.apache.carbondata.spark.rdd.DataManagementFunc$$anonfun$executeCompaction$1.apply(DataManagementFunc.scala:192) at scala.collection.Iterator$class.foreach(Iterator.scala:893) at scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at scala.collection.AbstractIterable.foreach(Iterable.scala:54) at org.apache.carbondata.spark.rdd.DataManagementFunc$.executeCompaction(DataManagementFunc.scala:192) at org.apache.carbondata.spark.rdd.CarbonDataRDDFactory$$anon$2.run(CarbonDataRDDFactory.scala:273) at org.apache.carbondata.spark.rdd.CarbonDataRDDFactory$.startCompactionThreads(CarbonDataRDDFactory.scala:364) at org.apache.carbondata.spark.rdd.CarbonDataRDDFactory$.alterTableForCompaction(CarbonDataRDDFactory.scala:157) at org.apache.spark.sql.execution.command.AlterTableCompaction.processData(carbonTableSchema.scala:175) at org.apache.spark.sql.execution.command.AlterTableCompaction.run(carbonTableSchema.scala:139) at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58) at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56) at org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:74) at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:114) at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:114) at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:135) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:132) at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:113) at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:87) at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:87) at org.apache.spark.sql.Dataset.<init>(Dataset.scala:185) at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:64) at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:592) at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:699) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:220) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:163) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:160) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1.run(SparkExecuteStatementOperation.scala:173) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.Exception: Compaction Failure in Merger Rdd. at org.apache.carbondata.spark.rdd.Compactor$.triggerCompaction(Compactor.scala:143) at org.apache.carbondata.spark.rdd.Compactor.triggerCompaction(Compactor.scala) at org.apache.carbondata.spark.compaction.CompactionCallable.call(CompactionCallable.java:40) at org.apache.carbondata.spark.compaction.CompactionCallable.call(CompactionCallable.java:29) ... 4 more 17/08/30 15:12:31 ERROR CarbonDataRDDFactory$: pool-24-thread-7 Exception in compaction thread java.lang.Exception: Compaction Failure in Merger Rdd. 17/08/30 15:12:31 INFO HdfsFileLock: pool-24-thread-7 Deleted the lock file hdfs://localhost:54310/user/hive/warehouse/carbon.store/30aug_5/uniqdata/compaction.lock 17/08/30 15:12:31 ERROR CarbonDataRDDFactory$: pool-24-thread-7 Exception in start compaction thread. Exception in compaction java.lang.Exception: Compaction Failure in Merger Rdd. 17/08/30 15:12:31 ERROR AbstractDFSCarbonFile: pool-24-thread-7 Exception occurred:File does not exist: hdfs://localhost:54310/user/hive/warehouse/carbon.store/30aug_5/uniqdata/compaction.lock 17/08/30 15:12:31 ERROR HdfsFileLock: pool-24-thread-7 Not able to delete the lock file because it is not existed in location hdfs://localhost:54310/user/hive/warehouse/carbon.store/30aug_5/uniqdata/compaction.lock 17/08/30 15:12:31 ERROR SparkExecuteStatementOperation: Error executing query, currentState RUNNING, java.lang.RuntimeException: Compaction failed. Please check logs for more info. Exception in compaction java.lang.Exception: Compaction Failure in Merger Rdd. at scala.sys.package$.error(package.scala:27) at org.apache.spark.sql.execution.command.AlterTableCompaction.processData(carbonTableSchema.scala:184) at org.apache.spark.sql.execution.command.AlterTableCompaction.run(carbonTableSchema.scala:139) at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58) at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56) at org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:74) at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:114) at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:114) at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:135) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:132) at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:113) at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:87) at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:87) at org.apache.spark.sql.Dataset.<init>(Dataset.scala:185) at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:64) at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:592) at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:699) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:220) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:163) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:160) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1.run(SparkExecuteStatementOperation.scala:173) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) 17/08/30 15:12:31 ERROR SparkExecuteStatementOperation: Error running hive query: org.apache.hive.service.cli.HiveSQLException: java.lang.RuntimeException: Compaction failed. Please check logs for more info. Exception in compaction java.lang.Exception: Compaction Failure in Merger Rdd. at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:258) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:163) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:160) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1.run(SparkExecuteStatementOperation.scala:173) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) > Major and Minor Compaction Failing > ---------------------------------- > > Key: CARBONDATA-1422 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1422 > Project: CarbonData > Issue Type: Bug > Affects Versions: 1.2.0 > Reporter: Pallavi Singh > Fix For: 1.2.0 > > > The major and minor compaction is failing. > The Compaction is failing in the default scenario where the table property > dictionary_include is not specified which is the default behaviour. > Please find the error logs below : > 0: jdbc:hive2://localhost:10000> show segments for table uniqdata; > +--------------------+----------+--------------------------+--------------------------+--+ > | SegmentSequenceId | Status | Load Start Time | Load End > Time | > +--------------------+----------+--------------------------+--------------------------+--+ > | 9 | Success | 2017-08-29 11:17:29.927 | 2017-08-29 > 11:17:30.555 | > | 8 | Success | 2017-08-29 11:17:27.572 | 2017-08-29 > 11:17:28.363 | > | 7 | Success | 2017-08-29 11:17:23.583 | 2017-08-29 > 11:17:25.112 | > | 6 | Success | 2017-08-29 11:17:07.966 | 2017-08-29 > 11:17:09.322 | > | 5 | Success | 2017-08-29 10:38:15.727 | 2017-08-29 > 10:38:16.548 | > | 4 | Success | 2017-08-29 10:37:13.053 | 2017-08-29 > 10:37:13.888 | > | 3 | Success | 2017-08-29 10:36:57.851 | 2017-08-29 > 10:36:59.08 | > | 2 | Success | 2017-08-29 10:36:49.439 | 2017-08-29 > 10:36:50.373 | > | 1 | Success | 2017-08-29 10:36:37.365 | 2017-08-29 > 10:36:38.768 | > | 0 | Success | 2017-08-29 10:36:21.011 | 2017-08-29 > 10:36:26.1 | > +--------------------+----------+--------------------------+--------------------------+--+ > 10 rows selected (0.081 seconds) > 0: jdbc:hive2://localhost:10000> ALTER TABLE uniqdata COMPACT 'MINOR'; > Error: java.lang.RuntimeException: Compaction failed. Please check logs for > more info. Exception in compaction java.lang.Exception: Compaction Failure in > Merger Rdd. (state=,code=0) > 0: jdbc:hive2://localhost:10000> ALTER TABLE uniqdata COMPACT 'MAJOR'; > Error: java.lang.RuntimeException: Compaction failed. Please check logs for > more info. Exception in compaction java.lang.Exception: Compaction Failure in > Merger Rdd. (state=,code=0) > 0: jdbc:hive2://localhost:10000> -- This message was sent by Atlassian JIRA (v6.4.14#64029)