If the issue is what you posted in SO, I think the stack trace explains it already. You want to avoid this recursive definition, which in general can't work. I think it's simply explicitly disallowed in all cases now, but, you should not be depending on this anyway - why can't this just be avoided?
On Mon, Dec 13, 2021 at 10:06 AM Daniel de Oliveira Mantovani < daniel.oliveira.mantov...@gmail.com> wrote: > Sean, > > https://github.com/music-of-the-ainur/almaren-framework/tree/spark-3.2 > > Just executing "sbt test" will reproduce the error, the same code works > for spark 2.3.x, 2.4.x and 3.1.x why doesn't it work for spark 3.2 ? > > Thank you so much > > > > On Mon, Dec 13, 2021 at 12:59 PM Sean Owen <sro...@gmail.com> wrote: > >> ... but the error is not "because that already exists". See your stack >> trace. It's because the definition is recursive. You define temp view >> test1, create a second DF from it, and then redefine test1 as that result. >> test1 depends on test1. >> >> On Mon, Dec 13, 2021 at 9:58 AM Daniel de Oliveira Mantovani < >> daniel.oliveira.mantov...@gmail.com> wrote: >> >>> Sean, >>> >>> The method name is very clear "createOrReplaceTempView" doesn't make >>> any sense to throw an exception because this view already exists. Spark >>> 3.2.x is breaking back compatibility with no reason or sense. >>> >>> >>> On Mon, Dec 13, 2021 at 12:53 PM Sean Owen <sro...@gmail.com> wrote: >>> >>>> The error looks 'valid' - you define a temp view in terms of its own >>>> previous version, which doesn't quite make sense - somewhere the new >>>> definition depends on the old definition. I think it just correctly >>>> surfaces as an error now,. >>>> >>>> On Mon, Dec 13, 2021 at 9:41 AM Daniel de Oliveira Mantovani < >>>> daniel.oliveira.mantov...@gmail.com> wrote: >>>> >>>>> Hello team, >>>>> >>>>> I've found this issue while I was porting my project from Apache Spark >>>>> 3.1.x to 3.2.x. >>>>> >>>>> >>>>> https://stackoverflow.com/questions/69937415/spark-3-2-0-the-different-dataframe-createorreplacetempview-the-same-name-tempvi >>>>> >>>>> Do we have a bug for that in apache-spark or I need to create one ? >>>>> >>>>> Thank you so much >>>>> >>>>> [info] com.github.music.of.the.ainur.almaren.Test *** ABORTED *** >>>>> [info] org.apache.spark.sql.AnalysisException: Recursive view >>>>> `__TABLE__` detected (cycle: `__TABLE__` -> `__TABLE__`) >>>>> [info] at >>>>> org.apache.spark.sql.errors.QueryCompilationErrors$.recursiveViewDetectedError(QueryCompilationErrors.scala:2045) >>>>> [info] at >>>>> org.apache.spark.sql.execution.command.ViewHelper$.checkCyclicViewReference(views.scala:515) >>>>> [info] at >>>>> org.apache.spark.sql.execution.command.ViewHelper$.$anonfun$checkCyclicViewReference$2(views.scala:522) >>>>> [info] at >>>>> org.apache.spark.sql.execution.command.ViewHelper$.$anonfun$checkCyclicViewReference$2$adapted(views.scala:522) >>>>> [info] at scala.collection.Iterator.foreach(Iterator.scala:941) >>>>> [info] at scala.collection.Iterator.foreach$(Iterator.scala:941) >>>>> [info] at >>>>> scala.collection.AbstractIterator.foreach(Iterator.scala:1429) >>>>> [info] at >>>>> scala.collection.IterableLike.foreach(IterableLike.scala:74) >>>>> [info] at >>>>> scala.collection.IterableLike.foreach$(IterableLike.scala:73) >>>>> [info] at >>>>> scala.collection.AbstractIterable.foreach(Iterable.scala:56) >>>>> >>>>> -- >>>>> >>>>> -- >>>>> Daniel Mantovani >>>>> >>>>> >>> >>> -- >>> >>> -- >>> Daniel Mantovani >>> >>> > > -- > > -- > Daniel Mantovani > >
