> > Thanks for confirmation. We are using the workaround to create a separate > Hive external table STORED AS PARQUET with the exact location of Delta > table. Our use case is batch-driven and we are running VACUUM with 0 > retention after every batch is completed. Do you see any potential problem > with this workaround, other than during the time when the batch is running > the table can provide some wrong information? >
This is a reasonable workaround to allow other systems to read Delta tables. Another consideration is that if you are running on S3, eventual consistency my increase the amount of time before external readers see a consistent view. Also note, that this prevents you from using time travel. In the near future, I think we should also support generating manifest files that list the data files in the most recent version of the Delta table (see #76 <https://github.com/delta-io/delta/issues/76> for details). This will give support for Presto, though Hive would require some additional modifications on the Hive side (if there are any Hive contributors / committers on this list let me know!). In the longer term, we are talking with authors of other engines to build native support for reading the Delta transaction log (e.g. this announcement from Starburst <https://www.starburstdata.com/technical-blog/starburst-presto-databricks-delta-lake-support/>). Please contact me if you are interested in contributing here!
