dennishuo opened a new pull request, #6428: URL: https://github.com/apache/iceberg/pull/6428
This read-only implementation of the Catalog interface, initially built on top of the [Snowflake JDBC driver](https://docs.snowflake.com/en/user-guide/jdbc.html) for the connection layer, enables engines like Spark using the Iceberg Java SDK to be able to consume [Snowflake-managed Iceberg Tables](https://www.snowflake.com/blog/iceberg-tables-powering-open-standards-with-snowflake-innovations/) via Iceberg Catalog interfaces. Example, assuming a Snowflake account with a database `iot_data` containing a schema `public` and a managed Iceberg table `sensor_test_results`: spark-shell --conf spark.sql.catalog.snowlog=org.apache.iceberg.spark.SparkCatalog \ --conf spark.sql.catalog.snowlog.catalog-impl=org.apache.iceberg.snowflake.SnowflakeCatalog \ --conf spark.sql.catalog.snowlog.uri="jdbc:snowflake://$ACCOUNT.snowflakecomputing.com" \ .... scala> spark.sessionState.catalogManager.setCurrentCatalog("snowlog"); scala> spark.sql("show namespaces in iot_data").show(false); scala> spark.sql("select * from iot_data.public.sensor_test_results limit 10").show(false); Note that the involvement of a JDBC driver is only incidental, and functionality is different from the `JdbcCatalog` - here, Snowflake itself manages manifest/metadata files and table/snapshot metadata, and this catalog layer facilitates the coordination of metadata-file locations and discovery of the latest table snapshot versions without resorting to file-listing or "directory-name"-listing (for listTables or listNamespaces) like the HadoopCatalog. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
