[GitHub] [spark] wangyum commented on pull request #38358: [SPARK-40588] FileFormatWriter materializes AQE plan before accessing outputOrdering

[via GitHub]

wangyum commented on PR #38358:
URL: https://github.com/apache/spark/pull/38358#issuecomment-1455371977

   @EnricoMi It seems it will remove the table location if a 
`java.lang.ArithmeticException` is thrown after this change.
   How to reproduce:
   ```scala
   import org.apache.hadoop.fs.{FileSystem, Path}
   
   import org.apache.spark.sql.QueryTest
   import org.apache.spark.sql.catalyst.TableIdentifier
   
   sql("CREATE TABLE IF NOT EXISTS spark32_overwrite(amt1 int) STORED AS ORC")
   sql("CREATE TABLE IF NOT EXISTS spark32_overwrite2(amt1 long) STORED AS ORC")
   sql("INSERT OVERWRITE TABLE spark32_overwrite2 select 6000044164")
   sql("set spark.sql.ansi.enabled=true")
   
   val loc =
     
spark.sessionState.catalog.getTableMetadata(TableIdentifier("spark32_overwrite")).location
   
   val fs = FileSystem.get(loc, spark.sparkContext.hadoopConfiguration)
   println("Location exists: " + fs.exists(new Path(loc)))
   
   try {
     sql("INSERT OVERWRITE TABLE spark32_overwrite select amt1 from " +
       "(select cast(amt1 as int) as amt1 from spark32_overwrite2 distribute by 
amt1)")
   } finally {
     println("Location exists: " + fs.exists(new Path(loc)))
   }
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org