We started talking about a proper API/tool for migrating existing tables in [1] but I do think we created any issues. I’ll attach the email thread to the created issue. Thanks!
[1] - https://lists.apache.org/thread.html/0f55e50fe48c15a55b31277b48e70500c642ea50cf8d6a3f12dfafe4@%3Cdev.iceberg.apache.org%3E <https://lists.apache.org/thread.html/0f55e50fe48c15a55b31277b48e70500c642ea50cf8d6a3f12dfafe4@%3Cdev.iceberg.apache.org%3E> > On 3 Jul 2019, at 14:51, 俊杰陈 <cjjnj...@gmail.com> wrote: > > Thanks, Ryan > > OK, I will create one for this and you guys can delete if duplicated. > I wanted to run spark-sql-perf to verify basic functionality and > performance and found that It can't convert from spark/hive table to > iceberg table. > > On Wed, Jul 3, 2019 at 1:07 AM Ryan Blue <rb...@netflix.com.invalid> wrote: >> >> Right now, we don't have a utility to convert tables automatically. I don't >> think there is an open issue for it (though Anton may have opened one). >> >> On Tue, Jul 2, 2019 at 9:43 AM 俊杰陈 <cjjnj...@gmail.com> wrote: >>> >>> Thanks for the explanation. >>> >>> So to migrate a spark table to iceberg table, I can append the >>> SparkDataFiles from listParquetPartition to an iceberg table, right? I >>> don't see an existing open issue for this, is this still wokring in >>> progress? >>> >>> On Mon, Jul 1, 2019 at 7:34 PM Anton Okolnychyi >>> <aokolnyc...@apple.com.invalid> wrote: >>>> >>>> Hey, >>>> >>>> The issue you pointed out is about tracking Iceberg tables in HMS and >>>> levering HMS locks to commit metadata instead of relying on renames. This >>>> allows Iceberg to reliably manage metadata when it is persisted in object >>>> stores. >>>> >>>> At the same time, it is possible to migrate Spark tables to Iceberg >>>> without ETL jobs. You take a look at >>>> org.apache.iceberg.spark.SparkTableUtil. There is a plan to provide a >>>> proper API for migrating tables but it is not done yet. >>>> >>>> Thanks, >>>> Anton >>>> >>>> On 25 Jun 2019, at 16:35, 俊杰陈 <cjjnj...@gmail.com> wrote: >>>> >>>> Hi >>>> >>>> Do we support to read a hive table and convert it to iceberg table? Is >>>> this issue for tracking the feature? >>>> >>>> -- >>>> Thanks & Best Regards >>>> >>>> >>> >>> >>> -- >>> Thanks & Best Regards >> >> >> >> -- >> Ryan Blue >> Software Engineer >> Netflix > > > > -- > Thanks & Best Regards
