Hi Yong Yang, It is supported by Iceberg, and this is exactly how Impala is working. I.e. Impala's Parquet writer writes the data files, then we use Iceberg's API to append them to the table. You can find the relevant code here: https://github.com/apache/impala/blob/822e8373d1f1737865899b80862c2be7b07cc950/fe/src/main/java/org/apache/impala/service/IcebergCatalogOpExecutor.java#L197-L271
For inserting files we use Iceberg's AppendFiles class. For overwriting a table/partitions we use Iceberg's ReplacePartitions class. One important thing that you need to do during writing the Parquet data files is to fill the 'field_id <https://github.com/apache/parquet-format/blob/473a3a7710f992b01af79095757d71e1fc68ef62/src/main/thrift/parquet.thrift#L398>' member for each schema element, corresponding to the Iceberg Column ID. Cheers, Zoltan On Thu, May 27, 2021 at 7:39 AM Peter Vary <pv...@cloudera.com.invalid> wrote: > Hi Yong Yang, > > Your message is ended up in my spam folder claiming that many messages > from @163.com are spam messages, but your question seems legitimate. > > With the Java API you can add Parquet > files to the Iceberg tables where the files conform to the specification. > > For Parquet, take a look here: > http://iceberg.apache.org/spec/#parquet > > For the Java API, take a look at here: https://iceberg.apache.org/api/ > > Thanks, Peter > > On Wed, 19 May 2021, 18:44 yong.sunny, <yong.su...@163.com> wrote: > >> Hi Iceberg Devs, >> >> I am new to the Iceberg. And I have a question about the iceberg >> manifest/manifest list/metadata api. >> I am wondering if the following is supported: >> 1. parquet file is writen by other apps >> 2. use the APIes of iceberg to create manifest file/manifest >> list/metadata(snapshot). The applicant to do that would be loading an >> independent, that is not loaded flink or spark. >> >> Could you please tell me if that is supported, and if that is supported, >> which APIes should I use? >> >> If I post the question in wrong channel, please tell me which one I >> should use. >> >> Thanks and Best regards, >> Yong Yang >> >> >> >> >
