Sean, https://github.com/music-of-the-ainur/almaren-framework/tree/spark-3.2
Just executing "sbt test" will reproduce the error, the same code works for spark 2.3.x, 2.4.x and 3.1.x why doesn't it work for spark 3.2 ? Thank you so much On Mon, Dec 13, 2021 at 12:59 PM Sean Owen <sro...@gmail.com> wrote: > ... but the error is not "because that already exists". See your stack > trace. It's because the definition is recursive. You define temp view > test1, create a second DF from it, and then redefine test1 as that result. > test1 depends on test1. > > On Mon, Dec 13, 2021 at 9:58 AM Daniel de Oliveira Mantovani < > daniel.oliveira.mantov...@gmail.com> wrote: > >> Sean, >> >> The method name is very clear "createOrReplaceTempView" doesn't make any >> sense to throw an exception because this view already exists. Spark 3.2.x >> is breaking back compatibility with no reason or sense. >> >> >> On Mon, Dec 13, 2021 at 12:53 PM Sean Owen <sro...@gmail.com> wrote: >> >>> The error looks 'valid' - you define a temp view in terms of its own >>> previous version, which doesn't quite make sense - somewhere the new >>> definition depends on the old definition. I think it just correctly >>> surfaces as an error now,. >>> >>> On Mon, Dec 13, 2021 at 9:41 AM Daniel de Oliveira Mantovani < >>> daniel.oliveira.mantov...@gmail.com> wrote: >>> >>>> Hello team, >>>> >>>> I've found this issue while I was porting my project from Apache Spark >>>> 3.1.x to 3.2.x. >>>> >>>> >>>> https://stackoverflow.com/questions/69937415/spark-3-2-0-the-different-dataframe-createorreplacetempview-the-same-name-tempvi >>>> >>>> Do we have a bug for that in apache-spark or I need to create one ? >>>> >>>> Thank you so much >>>> >>>> [info] com.github.music.of.the.ainur.almaren.Test *** ABORTED *** >>>> [info] org.apache.spark.sql.AnalysisException: Recursive view >>>> `__TABLE__` detected (cycle: `__TABLE__` -> `__TABLE__`) >>>> [info] at >>>> org.apache.spark.sql.errors.QueryCompilationErrors$.recursiveViewDetectedError(QueryCompilationErrors.scala:2045) >>>> [info] at >>>> org.apache.spark.sql.execution.command.ViewHelper$.checkCyclicViewReference(views.scala:515) >>>> [info] at >>>> org.apache.spark.sql.execution.command.ViewHelper$.$anonfun$checkCyclicViewReference$2(views.scala:522) >>>> [info] at >>>> org.apache.spark.sql.execution.command.ViewHelper$.$anonfun$checkCyclicViewReference$2$adapted(views.scala:522) >>>> [info] at scala.collection.Iterator.foreach(Iterator.scala:941) >>>> [info] at scala.collection.Iterator.foreach$(Iterator.scala:941) >>>> [info] at >>>> scala.collection.AbstractIterator.foreach(Iterator.scala:1429) >>>> [info] at scala.collection.IterableLike.foreach(IterableLike.scala:74) >>>> [info] at >>>> scala.collection.IterableLike.foreach$(IterableLike.scala:73) >>>> [info] at scala.collection.AbstractIterable.foreach(Iterable.scala:56) >>>> >>>> -- >>>> >>>> -- >>>> Daniel Mantovani >>>> >>>> >> >> -- >> >> -- >> Daniel Mantovani >> >> -- -- Daniel Mantovani
