Hi,
I am also trying to achieve something similar. The Merge INTO format has
multiple "when matched/not matched" condition, and usually, you can take
action like "delete" or "update" or "insert", can I do that by "overwrite
part of the destination table with the replacement"? Also, the
recommendation of overwriting, will it use UPSERT, or are you trying to
overwrite everything on the target table?
Till now I have been able to use Iceberg API directly for UPSERT, I believe
there is no way I can do this via dataFrame operations, as a single commit.
Thanks,
Ashish
On Tue, Oct 13, 2020 at 12:24 PM Ryan Blue <rb...@netflix.com.invalid>
wrote:
> Hi Arnaud,
>
> You're right that MERGE INTO isn't supported yet. What I've seen most
> people do is to implement the operation using SQL to select existing data
> and join it with new data, then overwrite part of the destination table
> with the replacement.
>
> On Mon, Oct 12, 2020 at 2:10 PM Arnaud Nauwynck <arnaud.nauwy...@gmail.com>
> wrote:
>
>> Hi Iceberg dev team,
>>
>> I am trying to use iceberg to do "upsert" based on an event table
>> In pure SQL, it is unsupported yet
>>
>> Here is my example with 2 tables: "iceberg_table" should contain updated
>> data, and "table_event" contains event updates.
>>
>> scala> spark.sql(""" MERGE INTO db1.iceberg_table t USING db1.table_event
>> e ON e.id = t.id WHEN MATCHED THEN UPDATE SET t.data = e.data WHEN NOT
>> MATCHED THEN INSERT (id, data) VALUES (id, data) """).show()
>> ..
>> java.lang.UnsupportedOperationException: MERGE INTO TABLE is not
>> supported temporarily.
>>
>> Is there a way to execute it programmatically using Spark Api or Iceberg
>> Api?
>> Any idea when the sql feature might be available ?
>>
>>
>> Thanks in advance
>>
>> Regards,
>> Arnaud Nauwynck
>>
>>
>
> --
> Ryan Blue
> Software Engineer
> Netflix
>
