You would pass it via option, like option(DataSourceWriteOptions.PAYLOAD_CLASS_OPT_KEY(), EmptyHoodieRecordPayload.class.getName())
selvaraj periyasamy <selvaraj.periyasamy1...@gmail.com> 于2020年2月26日周三 上午2:24写道： > OverwriteWithLatestAvroPayload is used for Delta Streamer. Is there a way > for DataSource Writer? > > please correct me , if I am wrong. > > Thanks, > Selva > > > On Mon, Feb 24, 2020 at 1:15 PM Gary Li <yanjia.gary...@gmail.com> wrote: > > > Hi, in this case you need to design your own logic to handle merging. > > Please check OverwriteWithLatestAvroPlayload class. You can write your > own > > one and pass it as DataSourceWriteOptions.PAYLOAD_CLASS_OPT_KEY to Hudi. > > > > On Mon, Feb 24, 2020 at 12:25 PM selvaraj periyasamy < > > selvaraj.periyasamy1...@gmail.com> wrote: > > > > > Hi, I am experimenting Hudi 0.5.0 version for some of the update use > > cases. > > > > > > Our flow is as below > > > > > > RDBMS -> CDC Log -> Hive -> COW table. > > > > > > CCDC log for update would have the value only for the primary key > > columns + > > > updated columns. Remaining column values are null. While upserting > values > > > on CCOW table, would need to update only the column values , which are > > > updated and retain the values for other columns. When I tested, Hudi > > > updates remaining column values as null since log has null values. > > > > > > Is there a way to merge rows for columns which are having values during > > > update? > > > > > > Thanks, > > > Selva > > > > > >
