An issue I encountered while working with Materialized Views in Spark SQL. It appears that there is an inconsistency between the behavior of Materialized Views in Spark SQL and Hive.
When attempting to execute a statement like DROP MATERIALIZED VIEW IF EXISTS test.mv in Spark SQL, I encountered a syntax error indicating that the keyword MATERIALIZED is not recognized. However, the same statement executes successfully in Hive without any errors. pyspark.errors.exceptions.captured.ParseException: [PARSE_SYNTAX_ERROR] Syntax error at or near 'MATERIALIZED'.(line 1, pos 5) == SQL == DROP MATERIALIZED VIEW IF EXISTS test.mv -----^^^ Here are the versions I am using: *Hive: 3.1.1Spark: 3.4* my Spark session: spark = SparkSession.builder \ .appName("test") \ .enableHiveSupport() \ .getOrCreate() Has anyone seen this behaviour or encountered a similar issue or if there are any insights into why this discrepancy exists between Spark SQL and Hive. Thanks Mich Talebzadeh, Technologist | Architect | Data Engineer | Generative AI | FinCrime London United Kingdom view my Linkedin profile https://en.everybodywiki.com/Mich_Talebzadeh Disclaimer: The information provided is correct to the best of my knowledge but of course cannot be guaranteed . It is essential to note that, as with any advice, quote "one test result is worth one-thousand expert opinions (Werner Von Braun)".