Re: [PR] [HUDI-7926] dataskipping failure mode should be strict in query test [hudi]
danny0405 merged PR #11502: URL: https://github.com/apache/hudi/pull/11502 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7926] dataskipping failure mode should be strict in query test [hudi]
hudi-bot commented on PR #11502: URL: https://github.com/apache/hudi/pull/11502#issuecomment-2186494481 ## CI report: * 2d80ec8e1a7df051c5e62423b21f29711b684630 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24543) * f85ebcae1fdf20ba678d5e1471285f7e4b8a2bd2 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24544) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7926] dataskipping failure mode should be strict in query test [hudi]
hudi-bot commented on PR #11502: URL: https://github.com/apache/hudi/pull/11502#issuecomment-2186479786 ## CI report: * 2d80ec8e1a7df051c5e62423b21f29711b684630 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24543) * f85ebcae1fdf20ba678d5e1471285f7e4b8a2bd2 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7926] dataskipping failure mode should be strict in query test [hudi]
hudi-bot commented on PR #11502: URL: https://github.com/apache/hudi/pull/11502#issuecomment-2186464201 ## CI report: * 2d80ec8e1a7df051c5e62423b21f29711b684630 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24543) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7926] dataskipping failure mode should be strict in query test [hudi]
hudi-bot commented on PR #11502: URL: https://github.com/apache/hudi/pull/11502#issuecomment-2186379738 ## CI report: * 2d80ec8e1a7df051c5e62423b21f29711b684630 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7926] dataskipping failure mode should be strict in query test [hudi]
KnightChess commented on code in PR #11502:
URL: https://github.com/apache/hudi/pull/11502#discussion_r1650888209
##
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieFileIndex.scala:
##
@@ -472,6 +473,11 @@ object HoodieFileIndex extends Logging {
properties.setProperty(DataSourceReadOptions.FILE_INDEX_LISTING_MODE_OVERRIDE.key,
listingModeOverride)
}
+if (tableConfig != null) {
+ properties.setProperty(RECORDKEY_FIELD.key,
tableConfig.getRecordKeyFields.orElse(Array.empty).mkString(","))
Review Comment:
bucket query index need use it to create correct `KetGenerator`, otherwise
will create `AutoRecordGenWrapperKeyGenerator`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[PR] [HUDI-7926] dataskipping failure mode should be strict in query test [hudi]
KnightChess opened a new pull request, #11502: URL: https://github.com/apache/hudi/pull/11502 ### Change Logs dataskipping failure mode should be strict in test. if use fallback mode default, the query ut is meaningless. There may be other codes that have been introduced into bugs but cannot be measured. - query index use dataskipping strick mode - bucket query index add required parameters ### Impact None ### Risk level (write none, low medium or high below) none ### Documentation Update none ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
