Lin Liu created HUDI-8856:
-----------------------------
Summary: CDC queries with start instant archived (in history)
produce incorrect result
Key: HUDI-8856
URL: https://issues.apache.org/jira/browse/HUDI-8856
Project: Apache Hudi
Issue Type: Bug
Reporter: Lin Liu
How to reproduce:
1. Use the CDC query example in the quick start:
https://hudi.apache.org/docs/0.15.0/quick-start-guide
2. Use the following setting to create archived commits, and try more updates
queries:
{code:java}
val opts: Map[String, String] = Map(
"hoodie.archive.automatic" -> "true",
"hoodie.keep.max.commits" -> "3",
"hoodie.keep.min.commits" -> "2",
"hoodie.clean.automatic" -> "clean",
"hoodie.clean.policy" -> "KEEP_LATEST_COMMITS",
"hoodie.clean.trigger.max.commits" -> "1",
"hoodie.clean.commits.retained" -> "3")
val updatesDf = spark.read.format("hudi").load(basePath).withColumn("fare",
col("fare") * 2)
updatesDf.write.format("hudi").
option("hoodie.datasource.write.operation", "upsert").
option("hoodie.datasource.write.partitionpath.field", "city").
option("hoodie.table.cdc.enabled", "true").
option("hoodie.table.name", tableName).
options(opts).
mode(Append).
save(basePath)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
