moulimukherjee commented on a change in pull request #416: Adding docs on how to use custom catalog with iceberg URL: https://github.com/apache/incubator-iceberg/pull/416#discussion_r317856977
########## File path: site/docs/custom-catalog.md ########## @@ -0,0 +1,149 @@ +# Custom Catalog Implementation + +It's possible to read an iceberg table either from an hdfs path or from a hive table. It's also possible to use a custom metastore in place of hive. The steps to do that are as follows. + +- [Custom TableOperations](#custom-table-operations-implementation) +- [Custom Catalog](#custom-table-implementation) +- [Custom IcebergSource](#custom-icebergsource) + +### Custom table operations implementation +Extend `BaseMetastoreTableOperations` to provide implementation on how to read and write metadata + +Example: +```java +class CustomTableOperations extends BaseMetastoreTableOperations { + private String dbName; + private String tableName; + private Configuration conf; + private FileIO fileIO; + + protected CustomTableOperations(Configuration conf, String dbName, String tableName) { + this.conf = conf; + this.dbName = dbName; + this.tableName = tableName; + } + + // The doRefresh method should provide implementation on how to get the metadata location + @Override + public void doRefresh() { + + // Example custom service which returns the metadata location given a dbName and tableName + String metadataLocation = CustomService.getMetadataForTable(conf, dbName, tableName); + + // Use existing method to refresh metadata + refreshFromMetadataLocation(metadataLocation); + + } + + // The doCommit method should provide implementation on how to update with metadata location atomically + @Override + public void doCommit(TableMetadata base, TableMetadata metadata) { + // if the metadata is already out of date, reject it + if (base != current()) { Review comment: @aokolnychyi @rdblue I was wondering if it makes sense to have this and the following check as part of the `commit()` method instead, before calling `doCommit`? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org