I think we're around in circles - you should not do this. You essentially
have "__TABLE__ = SELECT * FROM __TABLE__" and I hope it's clear why that
can't work in general.
At first execution, sure, maybe "old" __TABLE__ refers to "SELECT 1", but
what about the second time? if you stick to that interpretation, it's
actually not executing correctly, though 'works'. If you execute it as is,
it fails for circularity. Both are bad, so it's just disallowed.
Just fix your code?
On Mon, Dec 13, 2021 at 11:27 AM Daniel de Oliveira Mantovani <
daniel.oliveira.mantov...@gmail.com> wrote:
> I've reduced the code to reproduce the issue,
>
> val df = spark.sql("SELECT 1")
> df.createOrReplaceTempView("__TABLE__")
> spark.sql("SELECT * FROM __TABLE__").show
> val df2 = spark.sql("SELECT *,2 FROM __TABLE__")
> df2.createOrReplaceTempView("__TABLE__") // Exception in Spark 3.2 but
> works for Spark 2.4.x and Spark 3.1.x
> spark.sql("SELECT * FROM __TABLE__").show
>
> org.apache.spark.sql.AnalysisException: Recursive view `__TABLE__`
> detected (cycle: `__TABLE__` -> `__TABLE__`)
> at
> org.apache.spark.sql.errors.QueryCompilationErrors$.recursiveViewDetectedError(QueryCompilationErrors.scala:2045)
> at
> org.apache.spark.sql.execution.command.ViewHelper$.checkCyclicViewReference(views.scala:515)
> at
> org.apache.spark.sql.execution.command.ViewHelper$.$anonfun$checkCyclicViewReference$2(views.scala:522)
> at
> org.apache.spark.sql.execution.command.ViewHelper$.$anonfun$checkCyclicViewReference$2$adapted(views.scala:522)
>
> On Mon, Dec 13, 2021 at 2:10 PM Sean Owen <sro...@gmail.com> wrote:
>
>> _shrug_ I think this is a bug fix, unless I am missing something here.
>> You shouldn't just use __TABLE__ for everything, and I'm not seeing a good
>> reason to do that other than it's what you do now.
>> I'm not clear if it's coming across that this _can't_ work in the general
>> case.
>>
>> On Mon, Dec 13, 2021 at 11:03 AM Daniel de Oliveira Mantovani <
>> daniel.oliveira.mantov...@gmail.com> wrote:
>>
>>>
>>> In this context, I don't want to worry about the name of the temporary
>>> table. That's why it is "__TABLE__".
>>> The point is that this behavior for Spark 3.2.x it's breaking back
>>> compatibility for all previous versions of Apache Spark. In my opinion we
>>> should at least have some flag like "spark.sql.legacy.keepSqlRecursive"
>>> true/false.
>>>
>>
>
> --
>
> --
> Daniel Mantovani
>
>
