[
https://issues.apache.org/jira/browse/SPARK-37648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17635913#comment-17635913
]
Michael F commented on SPARK-37648:
-----------------------------------
Any update here? This continues to be an issue in 3.3.1
> Spark catalog and Delta tables
> ------------------------------
>
> Key: SPARK-37648
> URL: https://issues.apache.org/jira/browse/SPARK-37648
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.1.2
> Environment: Spark version 3.1.2
> Scala version 2.12.10
> Hive version 2.3.7
> Delta version 1.0.0
> Reporter: Hanna Liashchuk
> Priority: Major
>
> I'm using Spark with Delta tables, while tables are created, there are no
> columns in the table.
> Steps to reproduce:
> 1. Start spark-shell
> {code:java}
> spark-shell --conf
> "spark.sql.extensions=io.delta.sql.DeltaSparkSessionExtension" --conf
> "spark.sql.catalog.spark_catalog=org.apache.spark.sql.delta.catalog.DeltaCatalog"
> --conf "spark.sql.legacy.parquet.int96RebaseModeInWrite=LEGACY"{code}
> 2. Create delta table
> {code:java}
> spark.range(10).write.format("delta").option("path",
> "tmp/delta").saveAsTable("delta"){code}
> 3. Make sure table exists
> {code:java}
> spark.catalog.listTables.show{code}
> 4. Find out that columns are not
> {code:java}
> spark.catalog.listColumns("delta").show{code}
> This is critical for Delta integration with different BI tools such as Power
> BI or Tableau, as they are querying spark catalog for the metadata and we are
> getting errors that no columns are found.
> Discussion can be found in Delta repository -
> https://github.com/delta-io/delta/issues/695
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
