aditiwari01 edited a comment on issue #2802: URL: https://github.com/apache/hudi/issues/2802#issuecomment-819561416
Hi @n3nash The issue I am facing is in the case when commit gets succeeded. Let me explain the issue with an example: Commit1: insert key1 in partition1 and key2 in partition2 with schema1. Commit2: update key2 in partition2 with schema2 In this case the schema2 is registered successfully after commit2. But the issue is that the `AbstractRealtimeRecordReader` is not reading the latest schema of the table. `AbstractRealtimeRecordReader` tries to read the latest schema for each partition from that last edited log/parquet file. As for the given example, the last edit in partition1 was with schema1. Hence the above code snippet that I have pasted in first comment, takes inn schema1 as writerSchema, causing the issue. (when split belongs to partition1) Also I replace the said logic with taking writerSchema from TableSchemaResolver rather than log/parquet file and it solves the issue for me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
