Re: [PR] Follow up HUDI-6937, fix the RealtimeCompactedRecordReader props inst… [hudi]
danny0405 merged PR #9853: URL: https://github.com/apache/hudi/pull/9853 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Follow up HUDI-6937, fix the RealtimeCompactedRecordReader props inst… [hudi]
danny0405 commented on code in PR #9853: URL: https://github.com/apache/hudi/pull/9853#discussion_r1357639219 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/realtime/RealtimeCompactedRecordReader.java: ## @@ -198,7 +197,7 @@ private Option mergeRecord(HoodieRecord newRecord, A GenericRecord genericRecord = HiveAvroSerializer.rewriteRecordIgnoreResultCheck(oldRecord, getLogScannerReaderSchema()); HoodieRecord record = new HoodieAvroIndexedRecord(genericRecord); Option> mergeResult = HoodieAvroRecordMerger.INSTANCE.merge(record, -genericRecord.getSchema(), newRecord, getLogScannerReaderSchema(), new TypedProperties(payloadProps)); +genericRecord.getSchema(), newRecord, getLogScannerReaderSchema(), payloadProps); Review Comment: `TypedProperties` itself is serializable, it should be a mistake, in HUDI-6937, we found a performance regression of almost 100% for parquet file from insert handle. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Follow up HUDI-6937, fix the RealtimeCompactedRecordReader props inst… [hudi]
codope commented on code in PR #9853: URL: https://github.com/apache/hudi/pull/9853#discussion_r1357636337 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/realtime/RealtimeCompactedRecordReader.java: ## @@ -198,7 +197,7 @@ private Option mergeRecord(HoodieRecord newRecord, A GenericRecord genericRecord = HiveAvroSerializer.rewriteRecordIgnoreResultCheck(oldRecord, getLogScannerReaderSchema()); HoodieRecord record = new HoodieAvroIndexedRecord(genericRecord); Option> mergeResult = HoodieAvroRecordMerger.INSTANCE.merge(record, -genericRecord.getSchema(), newRecord, getLogScannerReaderSchema(), new TypedProperties(payloadProps)); +genericRecord.getSchema(), newRecord, getLogScannerReaderSchema(), payloadProps); Review Comment: any reason why we were wrapping in TypedProperties before? Does it need to be serializable? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Follow up HUDI-6937, fix the RealtimeCompactedRecordReader props inst… [hudi]
hudi-bot commented on PR #9853: URL: https://github.com/apache/hudi/pull/9853#issuecomment-1760596194 ## CI report: * 7882181bc36094e7557e15237991a337d9518b79 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20315) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Follow up HUDI-6937, fix the RealtimeCompactedRecordReader props inst… [hudi]
hudi-bot commented on PR #9853: URL: https://github.com/apache/hudi/pull/9853#issuecomment-1760587858 ## CI report: * 7882181bc36094e7557e15237991a337d9518b79 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
