rymurr commented on pull request #1843: URL: https://github.com/apache/iceberg/pull/1843#issuecomment-734932589
> > I was thinking about this a bit differently, rather than providing a full catalog I was thinking that we just have the various spark catalogs treat file types as namespaces. Then we just have them switch to returning Hadoop tables in the load table method. > > I was thinking this way originally as well. My first attempt at it made for weird interactions between `BaseMetastoreCatalog` and the individual catalogs. And a bunch of dupe code in the various `Catalog` impls. I am still not satisfied w/ this solution as the hadoop delegate catalog is a bit strange but at least it centralises the logic a bit. I re-read what you said earlier @RussellSpitzer and I realized I mis-read it. Your suggestion to just modify the spark catalogs may be easier but it still has a lot of duplication and would require a mechanism to handle `StagedTable`s with `HadoopTables`. I believe the change will end up very similar to my current patch. I can still do the change if you want to see the difference? *Note* regardless of how we implement the change `DROP` still causes a problem in session catalogs, drop never passes through our session catalog. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
