Hi, I am pretty sure that Sean already answered the question. Also i do not think that creating iterative table definitions or data frame definitions are best practice.
Regards, Gourav On Mon, Dec 13, 2021 at 4:00 PM Sean Owen <sro...@gmail.com> wrote: > ... but the error is not "because that already exists". See your stack > trace. It's because the definition is recursive. You define temp view > test1, create a second DF from it, and then redefine test1 as that result. > test1 depends on test1. > > On Mon, Dec 13, 2021 at 9:58 AM Daniel de Oliveira Mantovani < > daniel.oliveira.mantov...@gmail.com> wrote: > >> Sean, >> >> The method name is very clear "createOrReplaceTempView" doesn't make any >> sense to throw an exception because this view already exists. Spark 3.2.x >> is breaking back compatibility with no reason or sense. >> >> >> On Mon, Dec 13, 2021 at 12:53 PM Sean Owen <sro...@gmail.com> wrote: >> >>> The error looks 'valid' - you define a temp view in terms of its own >>> previous version, which doesn't quite make sense - somewhere the new >>> definition depends on the old definition. I think it just correctly >>> surfaces as an error now,. >>> >>> On Mon, Dec 13, 2021 at 9:41 AM Daniel de Oliveira Mantovani < >>> daniel.oliveira.mantov...@gmail.com> wrote: >>> >>>> Hello team, >>>> >>>> I've found this issue while I was porting my project from Apache Spark >>>> 3.1.x to 3.2.x. >>>> >>>> >>>> https://stackoverflow.com/questions/69937415/spark-3-2-0-the-different-dataframe-createorreplacetempview-the-same-name-tempvi >>>> >>>> Do we have a bug for that in apache-spark or I need to create one ? >>>> >>>> Thank you so much >>>> >>>> [info] com.github.music.of.the.ainur.almaren.Test *** ABORTED *** >>>> [info] org.apache.spark.sql.AnalysisException: Recursive view >>>> `__TABLE__` detected (cycle: `__TABLE__` -> `__TABLE__`) >>>> [info] at >>>> org.apache.spark.sql.errors.QueryCompilationErrors$.recursiveViewDetectedError(QueryCompilationErrors.scala:2045) >>>> [info] at >>>> org.apache.spark.sql.execution.command.ViewHelper$.checkCyclicViewReference(views.scala:515) >>>> [info] at >>>> org.apache.spark.sql.execution.command.ViewHelper$.$anonfun$checkCyclicViewReference$2(views.scala:522) >>>> [info] at >>>> org.apache.spark.sql.execution.command.ViewHelper$.$anonfun$checkCyclicViewReference$2$adapted(views.scala:522) >>>> [info] at scala.collection.Iterator.foreach(Iterator.scala:941) >>>> [info] at scala.collection.Iterator.foreach$(Iterator.scala:941) >>>> [info] at >>>> scala.collection.AbstractIterator.foreach(Iterator.scala:1429) >>>> [info] at scala.collection.IterableLike.foreach(IterableLike.scala:74) >>>> [info] at >>>> scala.collection.IterableLike.foreach$(IterableLike.scala:73) >>>> [info] at scala.collection.AbstractIterable.foreach(Iterable.scala:56) >>>> >>>> -- >>>> >>>> -- >>>> Daniel Mantovani >>>> >>>> >> >> -- >> >> -- >> Daniel Mantovani >> >>