XAZAD opened a new issue, #6172:
URL: https://github.com/apache/iceberg/issues/6172
### Apache Iceberg version
0.13.0
### Query engine
Spark
### Please describe the bug 🐞
Method rewriteDataFiles throws
`org.apache.spark.sql.connector.catalog.CatalogNotFoundException: Catalog
'ice' plugin class not found: spark.sql.catalog.ice is not defined.`
My code:
```
val IcebergTable: IcebergTable =
Spark3Util.loadIcebergTable(spark,"ice.someDb.someSchema")
println(IcebergTable.schema.asStruct.fields())
SparkActions
.get(spark)
.rewriteDataFiles(IcebergTable)
.option("target-file-size-bytes", (1024L * 1024L * 1024L).toString)
.option("min-input-files", "2")
.execute()
```
my configuration:
`
{
"hive.metastore.uris": "thrift://*****",
"iceberg.engine.hive.enabled": "true",
"spark.sql.extensions":
"org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions",
"spark.sql.catalog.ice.uri": "thrift://*****",
"spark.sql.catalog.ice": "org.apache.iceberg.spark.SparkCatalog",
"spark.sql.catalog.ice.type": "hive"
}
`
my previous configuration was:
`
{
"spark.sql.catalog.spark_catalog.uri": "***",
"spark.sql.catalog.spark_catalog":
"org.apache.iceberg.spark.SparkSessionCatalog"
"spark.sql.catalog.spark_catalog": "hive"
}
`
error was
`java.lang.RuntimeException:
org.apache.spark.sql.connector.catalog.CatalogNotFoundException: Catalog
'default_iceberg' plugin class not found: spark.sql.catalog.default_iceberg is
not defined`
also while using
org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions I've got an
exception:
"Table %s is not an Iceberg table" in method Spark3Util.loadIcebergTable
changing catalog class solved it.
I'm using spark on Kubernetes, but I don't think it has a big affection to
problem.
Can you please give me an advice where to go further, thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
