dipankarmazumdar commented on issue #431: URL: https://github.com/apache/incubator-xtable/issues/431#issuecomment-2104843042
Hi @rajender07! The error clarifies the problem. It says the `version-hint.text` file was not found in the source table format (Iceberg). Do you see it on S3? This is the metadata file on Iceberg side when used with a `Hadoop` catalog. XTable would need this file to translate into the target Delta format. The important part to understand here is that Iceberg needs a CATALOG to get started with. Your config currently connects Iceberg with a Hive catalog but I don't see any thrift URL or such here. `pyspark --conf "spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions" --conf "spark.sql.catalog.spark_catalog=org.apache.iceberg.spark.SparkSessionCatalog" --conf "spark.sql.catalog.spark_catalog.type=hive"` Can you instead use a Hadoop catalog & configure with something like this: ``` spark.sql.catalog.hadoop_prod = org.apache.iceberg.spark.SparkCatalog spark.sql.catalog.hadoop_prod.type = hadoop spark.sql.catalog.hadoop_prod.warehouse = s3a://your-bucket ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
