bhasudha commented on issue #1960: URL: https://github.com/apache/hudi/issues/1960#issuecomment-673910051
@brandon-stanley Based on your description above, you could try this: Instead of skipping the precombine field, you could add the COALESCE(update_date, create_date) as new column before writing to Hudi and pass in that new column as the precombine field. I think you could use withColumn() in Spark to do this. Here duplicates are handled based on the latest value of the precombine field which is the COALESCE() described above. You wouldn't need to worry about Payload class then. Please correct me if I am missing something. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
