[ https://issues.apache.org/jira/browse/HUDI-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinoth Chandar closed HUDI-1427. -------------------------------- Resolution: Fixed > Throw a FileAlreadyExistsException when set HOODIE_AUTO_COMMIT_PROP to true > --------------------------------------------------------------------------- > > Key: HUDI-1427 > URL: https://issues.apache.org/jira/browse/HUDI-1427 > Project: Apache Hudi > Issue Type: Bug > Components: Spark Integration > Reporter: pengzhiwei > Assignee: pengzhiwei > Priority: Major > Labels: pull-request-available > Fix For: 0.7.0 > > > I have test write hudi with COW & MOR write type. When set > HOODIE_AUTO_COMMIT_PROP to true, a FileAlreadyExistsException throw out: > {code:java} > Exception in thread "main" org.apache.hudi.exception.HoodieIOException: > Failed to create file > /tmp/hudi/tbl_price_cow1/.hoodie/20201202104150.commitException in thread > "main" org.apache.hudi.exception.HoodieIOException: Failed to create file > /tmp/hudi/tbl_price_cow1/.hoodie/20201202104150.commit at > org.apache.hudi.common.table.timeline.HoodieActiveTimeline.createImmutableFileInPath(HoodieActiveTimeline.java:474) > at > org.apache.hudi.common.table.timeline.HoodieActiveTimeline.transitionState(HoodieActiveTimeline.java:350) > at > org.apache.hudi.common.table.timeline.HoodieActiveTimeline.transitionState(HoodieActiveTimeline.java:325) > at > org.apache.hudi.common.table.timeline.HoodieActiveTimeline.saveAsComplete(HoodieActiveTimeline.java:144) > at > org.apache.hudi.client.AbstractHoodieWriteClient.commitStats(AbstractHoodieWriteClient.java:181) > at > org.apache.hudi.client.SparkRDDWriteClient.commit(SparkRDDWriteClient.java:101) > at > org.apache.hudi.HoodieSparkSqlWriter$.commitAndPerformPostOperations(HoodieSparkSqlWriter.scala:413) > at > org.apache.hudi.HoodieSparkSqlWriter$.write(HoodieSparkSqlWriter.scala:210) > at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:125) at > org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:46) > at > org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70) > at > org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68) > at > org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:86) > at > org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:137) > at > org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:133) > at > org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:161) > at > org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) > at > org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:158) at > org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:133) > {code} > It seems that _../.hoodie/20201202104150.commit_ has commit twice and result > in this exception. -- This message was sent by Atlassian Jira (v8.3.4#803005)