I've reduced the code to reproduce the issue,
val df = spark.sql("SELECT 1")
df.createOrReplaceTempView("__TABLE__")
spark.sql("SELECT * FROM __TABLE__").show
val df2 = spark.sql("SELECT *,2 FROM __TABLE__")
df2.createOrReplaceTempView("__TABLE__") // Exception in Spark 3.2 but
works for Spark 2.4.x and Spark 3.1.x
spark.sql("SELECT * FROM __TABLE__").show
org.apache.spark.sql.AnalysisException: Recursive view `__TABLE__` detected
(cycle: `__TABLE__` -> `__TABLE__`)
at
org.apache.spark.sql.errors.QueryCompilationErrors$.recursiveViewDetectedError(QueryCompilationErrors.scala:2045)
at
org.apache.spark.sql.execution.command.ViewHelper$.checkCyclicViewReference(views.scala:515)
at
org.apache.spark.sql.execution.command.ViewHelper$.$anonfun$checkCyclicViewReference$2(views.scala:522)
at
org.apache.spark.sql.execution.command.ViewHelper$.$anonfun$checkCyclicViewReference$2$adapted(views.scala:522)
On Mon, Dec 13, 2021 at 2:10 PM Sean Owen <sro...@gmail.com> wrote:
> _shrug_ I think this is a bug fix, unless I am missing something here. You
> shouldn't just use __TABLE__ for everything, and I'm not seeing a good
> reason to do that other than it's what you do now.
> I'm not clear if it's coming across that this _can't_ work in the general
> case.
>
> On Mon, Dec 13, 2021 at 11:03 AM Daniel de Oliveira Mantovani <
> daniel.oliveira.mantov...@gmail.com> wrote:
>
>>
>> In this context, I don't want to worry about the name of the temporary
>> table. That's why it is "__TABLE__".
>> The point is that this behavior for Spark 3.2.x it's breaking back
>> compatibility for all previous versions of Apache Spark. In my opinion we
>> should at least have some flag like "spark.sql.legacy.keepSqlRecursive"
>> true/false.
>>
>
--
--
Daniel Mantovani
