[GitHub] [hudi] hudi-bot commented on pull request #5991: [HUDI-4329] Add separate control for Flink compaction operation sync/async mode

2022-06-27 Thread GitBox
hudi-bot commented on PR #5991: URL: https://github.com/apache/hudi/pull/5991#issuecomment-1168308178 ## CI report: * 48e9a4672ed395aa375d4fc1b26061159eca2d64 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #5988: [HUDI-4332] The wrong current instant may cause file deletion in AppendWriteFunction

2022-06-27 Thread GitBox
hudi-bot commented on PR #5988: URL: https://github.com/apache/hudi/pull/5988#issuecomment-1168308119 ## CI report: * 2ea7ec4d51c4cae7a956926514c80173c34bcdd5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9597

[GitHub] [hudi] hudi-bot commented on pull request #5627: [HUDI-3350][HUDI-3351] Rebase Record combining semantic into `HoodieRecordCombiningEngine`

2022-06-27 Thread GitBox
hudi-bot commented on PR #5627: URL: https://github.com/apache/hudi/pull/5627#issuecomment-1168307542 ## CI report: * 5bb5dfd3ce756d53ba44e410165b81e17ba9d7ba UNKNOWN * 870167d142cad0c0503176c8dc1b0910a996a096 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #5970: [HUDI-4320] Make sure `HoodieStorageConfig.PARQUET_WRITE_LEGACY_FORMAT_ENABLED` could be specified by the writer

2022-06-27 Thread GitBox
hudi-bot commented on PR #5970: URL: https://github.com/apache/hudi/pull/5970#issuecomment-1168303769 ## CI report: * 58d8e2c3b262ffa7263297d141e5c8d5d1690136 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9598

[GitHub] [hudi] hudi-bot commented on pull request #5988: [HUDI-4332] The wrong current instant may cause file deletion in AppendWriteFunction

2022-06-27 Thread GitBox
hudi-bot commented on PR #5988: URL: https://github.com/apache/hudi/pull/5988#issuecomment-1168303806 ## CI report: * 2ea7ec4d51c4cae7a956926514c80173c34bcdd5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9597

[GitHub] [hudi] hudi-bot commented on pull request #5990: [HUDI-4333] fix HoodieFileIndex's listFiles method log print skipping…

2022-06-27 Thread GitBox
hudi-bot commented on PR #5990: URL: https://github.com/apache/hudi/pull/5990#issuecomment-1168303835 ## CI report: * ae0ccb61faceed823788e7be3cf943f53cd97b34 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9600

[GitHub] [hudi] hudi-bot commented on pull request #5608: [HUDI-2150] Rename/Restructure configs for better modularity

2022-06-27 Thread GitBox
hudi-bot commented on PR #5608: URL: https://github.com/apache/hudi/pull/5608#issuecomment-1168303157 ## CI report: * 39e82bdcb6eea6b8c8d657484086cdf3a2c07fbb Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9599

[jira] [Updated] (HUDI-4332) The wrong current instant may cause file deletion in AppendWriteFunction

2022-06-27 Thread Forward Xu (Jira)
Title: Message Title Forward Xu updated an

[jira] [Updated] (HUDI-4329) Add separate control for compaction operation sync/async mode

2022-06-27 Thread ASF GitHub Bot (Jira)
Title: Message Title ASF GitHub Bot update

[GitHub] [hudi] chenshzh opened a new pull request, #5991: [HUDI-4329] Add separate control for Flink compaction operation sync/async mode

2022-06-27 Thread GitBox
chenshzh opened a new pull request, #5991: URL: https://github.com/apache/hudi/pull/5991 Add separate control for Flink compaction operation sync/async mode ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-

[GitHub] [hudi] XuQianJin-Stars commented on pull request #5988: [HUDI-4332] The wrong current instant may cause file deletion in AppendWriteFunction

2022-06-27 Thread GitBox
XuQianJin-Stars commented on PR #5988: URL: https://github.com/apache/hudi/pull/5988#issuecomment-1168272942 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [hudi] hudi-bot commented on pull request #5627: [HUDI-3350][HUDI-3351] Rebase Record combining semantic into `HoodieRecordCombiningEngine`

2022-06-27 Thread GitBox
hudi-bot commented on PR #5627: URL: https://github.com/apache/hudi/pull/5627#issuecomment-1168257656 ## CI report: * Unknown: [CANCELED](TBD) * 5bb5dfd3ce756d53ba44e410165b81e17ba9d7ba UNKNOWN * 870167d142cad0c0503176c8dc1b0910a996a096 Azure: [PENDING](https://dev.azure.c

[GitHub] [hudi] hudi-bot commented on pull request #5627: [HUDI-3350][HUDI-3351] Rebase Record combining semantic into `HoodieRecordCombiningEngine`

2022-06-27 Thread GitBox
hudi-bot commented on PR #5627: URL: https://github.com/apache/hudi/pull/5627#issuecomment-1168254547 ## CI report: * Unknown: [CANCELED](TBD) * 5bb5dfd3ce756d53ba44e410165b81e17ba9d7ba UNKNOWN * 870167d142cad0c0503176c8dc1b0910a996a096 UNKNOWN Bot commands

[GitHub] [hudi] wulei0302 commented on pull request #5627: [HUDI-3350][HUDI-3351] Rebase Record combining semantic into `HoodieRecordCombiningEngine`

2022-06-27 Thread GitBox
wulei0302 commented on PR #5627: URL: https://github.com/apache/hudi/pull/5627#issuecomment-1168226795 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [hudi] hudi-bot commented on pull request #5988: [HUDI-4332] The wrong current instant may cause file deletion in AppendWriteFunction

2022-06-27 Thread GitBox
hudi-bot commented on PR #5988: URL: https://github.com/apache/hudi/pull/5988#issuecomment-1168214053 ## CI report: * 2ea7ec4d51c4cae7a956926514c80173c34bcdd5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9597

[GitHub] [hudi] hudi-bot commented on pull request #5627: [HUDI-3350][HUDI-3351] Rebase Record combining semantic into `HoodieRecordCombiningEngine`

2022-06-27 Thread GitBox
hudi-bot commented on PR #5627: URL: https://github.com/apache/hudi/pull/5627#issuecomment-1168213718 ## CI report: * 5bb5dfd3ce756d53ba44e410165b81e17ba9d7ba UNKNOWN * 427cbc6076f215419616df39deb4dceb54c1b6bc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[jira] [Updated] (HUDI-4333) fix HoodieFileIndex's listFiles method log print skipping percent NaN

2022-06-27 Thread Forward Xu (Jira)
Title: Message Title Forward Xu updated an

[GitHub] [hudi] hudi-bot commented on pull request #5990: [HUDI-4333] fix HoodieFileIndex's listFiles method log print skipping…

2022-06-27 Thread GitBox
hudi-bot commented on PR #5990: URL: https://github.com/apache/hudi/pull/5990#issuecomment-1168186657 ## CI report: * ae0ccb61faceed823788e7be3cf943f53cd97b34 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9600

[GitHub] [hudi] hudi-bot commented on pull request #5608: [HUDI-2150] Rename/Restructure configs for better modularity

2022-06-27 Thread GitBox
hudi-bot commented on PR #5608: URL: https://github.com/apache/hudi/pull/5608#issuecomment-1168186371 ## CI report: * 9d2b850bb1eaee395b456ca818e399cbf1125982 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9580

[GitHub] [hudi] hudi-bot commented on pull request #5990: [HUDI-4333] fix HoodieFileIndex's listFiles method log print skipping…

2022-06-27 Thread GitBox
hudi-bot commented on PR #5990: URL: https://github.com/apache/hudi/pull/5990#issuecomment-1168184640 ## CI report: * ae0ccb61faceed823788e7be3cf943f53cd97b34 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #5608: [HUDI-2150] Rename/Restructure configs for better modularity

2022-06-27 Thread GitBox
hudi-bot commented on PR #5608: URL: https://github.com/apache/hudi/pull/5608#issuecomment-1168184335 ## CI report: * 9d2b850bb1eaee395b456ca818e399cbf1125982 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9580

[GitHub] [hudi] hudi-bot commented on pull request #5639: [HUDI-2516] Upgrade JUnit to 5.8.2

2022-06-27 Thread GitBox
hudi-bot commented on PR #5639: URL: https://github.com/apache/hudi/pull/5639#issuecomment-1168184383 ## CI report: * a885fa78a4c1420ddcf46ad2c634e4f7b6c42b12 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8898

[GitHub] [hudi] hudi-bot commented on pull request #5639: [HUDI-2516] Upgrade JUnit to 5.8.2

2022-06-27 Thread GitBox
hudi-bot commented on PR #5639: URL: https://github.com/apache/hudi/pull/5639#issuecomment-1168182238 ## CI report: * a885fa78a4c1420ddcf46ad2c634e4f7b6c42b12 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8898

[GitHub] [hudi] xiarixiaoyao commented on pull request #5990: [HUDI-4333] fix HoodieFileIndex's listFiles method log print skipping…

2022-06-27 Thread GitBox
xiarixiaoyao commented on PR #5990: URL: https://github.com/apache/hudi/pull/5990#issuecomment-1168175684 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

[jira] [Updated] (HUDI-4333) fix HoodieFileIndex's listFiles method log print skipping percent NaN

2022-06-27 Thread ASF GitHub Bot (Jira)
Title: Message Title ASF GitHub Bot update

[GitHub] [hudi] liujinhui1994 commented on pull request #5639: [HUDI-2516] Upgrade JUnit to 5.8.2

2022-06-27 Thread GitBox
liujinhui1994 commented on PR #5639: URL: https://github.com/apache/hudi/pull/5639#issuecomment-1168172013 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [hudi] XuQianJin-Stars opened a new pull request, #5990: [HUDI-4333] fix HoodieFileIndex's listFiles method log print skipping…

2022-06-27 Thread GitBox
XuQianJin-Stars opened a new pull request, #5990: URL: https://github.com/apache/hudi/pull/5990 … percent NaN ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.*

[jira] [Updated] (HUDI-4333) fix HoodieFileIndex's listFiles method log print skipping percent NaN

2022-06-27 Thread Forward Xu (Jira)
Title: Message Title Forward Xu updated an

[jira] [Updated] (HUDI-4333) fix HoodieFileIndex's listFiles method log print skipping percent NaN

2022-06-27 Thread Forward Xu (Jira)
Title: Message Title Forward Xu updated an

[jira] [Created] (HUDI-4333) fix HoodieFileIndex's listFiles method log print skipping percent NaN

2022-06-27 Thread Forward Xu (Jira)
Title: Message Title Forward Xu created an

[GitHub] [hudi] victorxiang30 commented on pull request #5756: [HUDI-4191] docker failure

2022-06-27 Thread GitBox
victorxiang30 commented on PR #5756: URL: https://github.com/apache/hudi/pull/5756#issuecomment-1168166170 @nsivabalan added description -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [hudi] KnightChess commented on pull request #5982: [HUDI-4325] fix spark sql procedure cause ParseException with semicolon

2022-06-27 Thread GitBox
KnightChess commented on PR #5982: URL: https://github.com/apache/hudi/pull/5982#issuecomment-1168162090 @XuQianJin-Stars @yanghua thanks for review :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[jira] [Updated] (HUDI-4332) The wrong current instant may cause file deletion in AppendWriteFunction

2022-06-27 Thread Danny Chen (Jira)
Title: Message Title Danny Chen updated an

[GitHub] [hudi] hudi-bot commented on pull request #5970: [HUDI-4320] Make sure `HoodieStorageConfig.PARQUET_WRITE_LEGACY_FORMAT_ENABLED` could be specified by the writer

2022-06-27 Thread GitBox
hudi-bot commented on PR #5970: URL: https://github.com/apache/hudi/pull/5970#issuecomment-1168154277 ## CI report: * 6fbc44d2971dee69216b57c77dd6c853d768fb60 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9524

[GitHub] [hudi] boneanxs commented on a diff in pull request #1834: [HUDI-1013] Adding Bulk Insert V2 implementation

2022-06-27 Thread GitBox
boneanxs commented on code in PR #1834: URL: https://github.com/apache/hudi/pull/1834#discussion_r907976925 ## hudi-spark/src/main/java/org/apache/hudi/DataSourceUtils.java: ## @@ -225,35 +220,35 @@ public static void checkRequiredProperties(TypedProperties props, List c }

[GitHub] [hudi] hudi-bot commented on pull request #5988: [HUDI-4332] The current instant may be wrong under some extreme conditions in AppendWriteFunction.

2022-06-27 Thread GitBox
hudi-bot commented on PR #5988: URL: https://github.com/apache/hudi/pull/5988#issuecomment-1168151924 ## CI report: * 2ea7ec4d51c4cae7a956926514c80173c34bcdd5 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9597

[GitHub] [hudi] hudi-bot commented on pull request #5970: [HUDI-4320] Make sure `HoodieStorageConfig.PARQUET_WRITE_LEGACY_FORMAT_ENABLED` could be specified by the writer

2022-06-27 Thread GitBox
hudi-bot commented on PR #5970: URL: https://github.com/apache/hudi/pull/5970#issuecomment-1168151901 ## CI report: * 6fbc44d2971dee69216b57c77dd6c853d768fb60 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9524

[GitHub] [hudi] hudi-bot commented on pull request #5627: [HUDI-3350][HUDI-3351] Rebase Record combining semantic into `HoodieRecordCombiningEngine`

2022-06-27 Thread GitBox
hudi-bot commented on PR #5627: URL: https://github.com/apache/hudi/pull/5627#issuecomment-1168151580 ## CI report: * 5bb5dfd3ce756d53ba44e410165b81e17ba9d7ba UNKNOWN * 5c34cb62dac887b1adb4ea9ff8c3873f3680818b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #5988: [HUDI-4332] The current instant may be wrong under some extreme conditions in AppendWriteFunction.

2022-06-27 Thread GitBox
hudi-bot commented on PR #5988: URL: https://github.com/apache/hudi/pull/5988#issuecomment-1168149449 ## CI report: * 2ea7ec4d51c4cae7a956926514c80173c34bcdd5 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #5627: [HUDI-3350][HUDI-3351] Rebase Record combining semantic into `HoodieRecordCombiningEngine`

2022-06-27 Thread GitBox
hudi-bot commented on PR #5627: URL: https://github.com/apache/hudi/pull/5627#issuecomment-1168149120 ## CI report: * 5bb5dfd3ce756d53ba44e410165b81e17ba9d7ba UNKNOWN * 5c34cb62dac887b1adb4ea9ff8c3873f3680818b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #5627: [HUDI-3350][HUDI-3351] Rebase Record combining semantic into `HoodieRecordCombiningEngine`

2022-06-27 Thread GitBox
hudi-bot commented on PR #5627: URL: https://github.com/apache/hudi/pull/5627#issuecomment-1168146636 ## CI report: * 5bb5dfd3ce756d53ba44e410165b81e17ba9d7ba UNKNOWN * 5c34cb62dac887b1adb4ea9ff8c3873f3680818b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[jira] [Resolved] (HUDI-4325) spark sql procedure cause ParseException with semicolon

2022-06-27 Thread Forward Xu (Jira)
Title: Message Title Forward Xu resolved a

[jira] [Updated] (HUDI-4325) spark sql procedure cause ParseException with semicolon

2022-06-27 Thread Forward Xu (Jira)
Title: Message Title Forward Xu updated an

[jira] [Updated] (HUDI-4325) spark sql procedure cause ParseException with semicolon

2022-06-27 Thread Forward Xu (Jira)
Title: Message Title Forward Xu updated an

[GitHub] [hudi] wulei0302 commented on pull request #5627: [HUDI-3350][HUDI-3351] Rebase Record combining semantic into `HoodieRecordCombiningEngine`

2022-06-27 Thread GitBox
wulei0302 commented on PR #5627: URL: https://github.com/apache/hudi/pull/5627#issuecomment-1168142429 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [hudi] wulei0302 commented on pull request #5627: [HUDI-3350][HUDI-3351] Rebase Record combining semantic into `HoodieRecordCombiningEngine`

2022-06-27 Thread GitBox
wulei0302 commented on PR #5627: URL: https://github.com/apache/hudi/pull/5627#issuecomment-1168141124 > It passes in my local env, I rerun the CI to try -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] neerajpadarthi opened a new issue, #5989: [SUPPORT] Schema Evolution Issue - New columns are not showing up in Spark-SQL.

2022-06-27 Thread GitBox
neerajpadarthi opened a new issue, #5989: URL: https://github.com/apache/hudi/issues/5989 When using Spark-SQL, it is not showing the new fields that are added at the end of the dataset but show up when using spark.read.format(‘org.apache.hudi‘).load(‘‘). I have linked the Glue catalog wit

[jira] [Updated] (HUDI-4332) The current instant may be wrong under some extreme conditions in AppendWriteFunction.

2022-06-27 Thread ASF GitHub Bot (Jira)
Title: Message Title ASF GitHub Bot update

[GitHub] [hudi] BruceKellan opened a new pull request, #5988: [HUDI-4332] The current instant may be wrong under some extreme conditions in AppendWriteFunction.

2022-06-27 Thread GitBox
BruceKellan opened a new pull request, #5988: URL: https://github.com/apache/hudi/pull/5988 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the pur

[jira] [Created] (HUDI-4332) The current instant may be wrong under some extreme conditions in AppendWriteFunction.

2022-06-27 Thread Ying Lin (Jira)
Title: Message Title Ying Lin created an i

[jira] [Assigned] (HUDI-4318) IndexOutOfBoundException when recordKey has List values for Bucket index table

2022-06-27 Thread Yuwei Xiao (Jira)
Title: Message Title Yuwei Xiao assigned a

[hudi] branch master updated (b14ed47f21 -> 09dc001430)

2022-06-27 Thread forwardxu
This is an automated email from the ASF dual-hosted git repository. forwardxu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from b14ed47f21 [HUDI-3506] Add call procedure for CommitsCommand (#5974) add 09dc001430 [HUDI-4325] fix spark sql pr

[GitHub] [hudi] XuQianJin-Stars merged pull request #5982: [HUDI-4325] fix spark sql procedure cause ParseException with semicolon

2022-06-27 Thread GitBox
XuQianJin-Stars merged PR #5982: URL: https://github.com/apache/hudi/pull/5982 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.

[GitHub] [hudi] XuQianJin-Stars merged pull request #5974: [HUDI-3506] Add call procedure for CommitsCommand

2022-06-27 Thread GitBox
XuQianJin-Stars merged PR #5974: URL: https://github.com/apache/hudi/pull/5974 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.

[hudi] branch master updated: [HUDI-3506] Add call procedure for CommitsCommand (#5974)

2022-06-27 Thread forwardxu
This is an automated email from the ASF dual-hosted git repository. forwardxu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new b14ed47f21 [HUDI-3506] Add call procedure for C

[GitHub] [hudi] melin commented on pull request #5943: [HUDI-4186] [WIP] Support Hudi with Spark 3.3.0

2022-06-27 Thread GitBox
melin commented on PR #5943: URL: https://github.com/apache/hudi/pull/5943#issuecomment-116835 Upgrade Parquet version to 1.12.3. [PARQUET-1968](https://issues.apache.org/jira/browse/PARQUET-1968) can simplify current In predicate filter pushdown. -- This is an automated message fr

[GitHub] [hudi] hudi-bot commented on pull request #5987: [HUDI-4331] Allow loading external config file from class loader

2022-06-27 Thread GitBox
hudi-bot commented on PR #5987: URL: https://github.com/apache/hudi/pull/5987#issuecomment-1168108594 ## CI report: * 7f26e9d2ab6d19cc842a9a69e486751c6b7400e6 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9593

[GitHub] [hudi] hudi-bot commented on pull request #5987: [HUDI-4331] Allow loading external config file from class loader

2022-06-27 Thread GitBox
hudi-bot commented on PR #5987: URL: https://github.com/apache/hudi/pull/5987#issuecomment-1168035562 ## CI report: * 7f26e9d2ab6d19cc842a9a69e486751c6b7400e6 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9593

[GitHub] [hudi] hudi-bot commented on pull request #5987: [HUDI-4331] Allow loading external config file from class loader

2022-06-27 Thread GitBox
hudi-bot commented on PR #5987: URL: https://github.com/apache/hudi/pull/5987#issuecomment-1168033142 ## CI report: * 7f26e9d2ab6d19cc842a9a69e486751c6b7400e6 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] zhedoubushishi commented on issue #5548: [SUPPORT] Hudi global configuration on EMR

2022-06-27 Thread GitBox
zhedoubushishi commented on issue #5548: URL: https://github.com/apache/hudi/issues/5548#issuecomment-1168021451 Thanks for reporting this issue. I submitted a PR https://github.com/apache/hudi/pull/5987 to fix this issue. With this PR, similar to EMR, you can add `hudi-defaults.conf` to `s

[GitHub] [hudi] zhedoubushishi commented on issue #5291: [SUPPORT] How to use hudi-defaults.conf with Glue

2022-06-27 Thread GitBox
zhedoubushishi commented on issue #5291: URL: https://github.com/apache/hudi/issues/5291#issuecomment-1168019390 @moustafaalaa With https://github.com/apache/hudi/pull/5987, similar to EMR, you can add `hudi-defaults.conf` to `spark.yarn.dist.files` so that Hudi can load this file. T

[jira] [Updated] (HUDI-4331) Allow loading external config file from class loader

2022-06-27 Thread ASF GitHub Bot (Jira)
Title: Message Title ASF GitHub Bot update

[GitHub] [hudi] zhedoubushishi opened a new pull request, #5987: [HUDI-4331] Allow loading external config file from class loader

2022-06-27 Thread GitBox
zhedoubushishi opened a new pull request, #5987: URL: https://github.com/apache/hudi/pull/5987 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the

[jira] [Created] (HUDI-4331) Allow loading external config file from class loader

2022-06-27 Thread Wenning Ding (Jira)
Title: Message Title Wenning Ding created

[GitHub] [hudi] bhasudha opened a new pull request, #5986: [DOCS] Fix Reliable ingestion from AWS S3 blog for configs

2022-06-27 Thread GitBox
bhasudha opened a new pull request, #5986: URL: https://github.com/apache/hudi/pull/5986 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purpos

[jira] [Created] (HUDI-4330) NPE when trying to upsert into a dataset with no Meta Fields

2022-06-27 Thread Alexey Kudinkin (Jira)
Title: Message Title Alexey Kudinkin creat

[GitHub] [hudi] hudi-bot commented on pull request #5978: [HUDI-4284] Implement bloom lookup tree as red-black tree

2022-06-27 Thread GitBox
hudi-bot commented on PR #5978: URL: https://github.com/apache/hudi/pull/5978#issuecomment-1167833711 ## CI report: * 34f0288415460a1938af792d7dff9bba36525884 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9590

[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #5828: [HUDI-4219] Merge Into when update expression "col=s.col+2" on precombine cause exception

2022-06-27 Thread GitBox
alexeykudinkin commented on code in PR #5828: URL: https://github.com/apache/hudi/pull/5828#discussion_r907755195 ## hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/MergeIntoHoodieTableCommand.scala: ## @@ -213,6 +221,26 @@ case class MergeIntoH

[GitHub] [hudi] phillycoder opened a new issue, #5985: [SUPPORT] Hudi upsert fails with java.lang.ClassCastException: optional binary xx (STRING) is not a group

2022-06-27 Thread GitBox
phillycoder opened a new issue, #5985: URL: https://github.com/apache/hudi/issues/5985 **Describe the problem you faced** Getting `java.lang.ClassCastException: optional binary xx (STRING)` exception when a record get updated. Also this issue specifically happening when a field is

[GitHub] [hudi] rubenssoto opened a new issue, #5984: [SUPPORT] Error on GlobalSortPartitioner using 0.9.0

2022-06-27 Thread GitBox
rubenssoto opened a new issue, #5984: URL: https://github.com/apache/hudi/issues/5984 Hello, We are trying to process a table and write to Hudi, the data size from source is 274GB but the data sink is only 300mb. We are using EMR on EKS: 30 pods 32gb each pod 6000mb overh

[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #5722: [HUDI-4170] Make user can use hoodie.datasource.read.paths to read necessary files

2022-06-27 Thread GitBox
alexeykudinkin commented on code in PR #5722: URL: https://github.com/apache/hudi/pull/5722#discussion_r906326302 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/MergeOnReadSnapshotRelation.scala: ## @@ -104,24 +104,8 @@ class MergeOnReadSnapshotRelatio

[GitHub] [hudi] hudi-bot commented on pull request #5982: [HUDI-4325] fix spark sql procedure cause ParseException with semicolon

2022-06-27 Thread GitBox
hudi-bot commented on PR #5982: URL: https://github.com/apache/hudi/pull/5982#issuecomment-1167716383 ## CI report: * ed65b16fc70131ae5fddb0f256fd52ffe99edcd5 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9588

[GitHub] [hudi] alexeykudinkin commented on issue #5808: [SUPPORT] Data skipping using Column Stats Bloom does not seem to work at all

2022-06-27 Thread GitBox
alexeykudinkin commented on issue #5808: URL: https://github.com/apache/hudi/issues/5808#issuecomment-1167715468 @Zhangshunyu awesome! Glad that it got resolved for you -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [hudi] hudi-bot commented on pull request #5945: [HUDI-4308] READ_OPTIMIZED read mode will temporary loss of data when compaction

2022-06-27 Thread GitBox
hudi-bot commented on PR #5945: URL: https://github.com/apache/hudi/pull/5945#issuecomment-1167692976 ## CI report: * 89f6c56da7eb96dca262b40009c73ee4a8ad4971 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9589

[GitHub] [hudi] kazdy commented on issue #5904: [SUPPORT] Spark saveAsTable() not working as expected

2022-06-27 Thread GitBox
kazdy commented on issue #5904: URL: https://github.com/apache/hudi/issues/5904#issuecomment-1167635783 As a workaround, I could :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [hudi] xushiyan commented on pull request #5627: [HUDI-3350][HUDI-3351] Rebase Record combining semantic into `HoodieRecordCombiningEngine`

2022-06-27 Thread GitBox
xushiyan commented on PR #5627: URL: https://github.com/apache/hudi/pull/5627#issuecomment-1167610974 @wulei0302 seems like CI caught some bug ``` 2022-06-27T15:52:42.0821073Z [ERROR] testSchemaEvolutionForTableType{String}[1] Time elapsed: 14.556 s <<< ERROR! 2022-06-27T15:52

[GitHub] [hudi] hudi-bot commented on pull request #5627: [HUDI-3350][HUDI-3351] Rebase Record combining semantic into `HoodieRecordCombiningEngine`

2022-06-27 Thread GitBox
hudi-bot commented on PR #5627: URL: https://github.com/apache/hudi/pull/5627#issuecomment-1167546331 ## CI report: * 5bb5dfd3ce756d53ba44e410165b81e17ba9d7ba UNKNOWN * 5c34cb62dac887b1adb4ea9ff8c3873f3680818b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #5937: [HUDI-4298] When reading the mor table with QUERY_TYPE_SNAPSHOT,Unabl…

2022-06-27 Thread GitBox
hudi-bot commented on PR #5937: URL: https://github.com/apache/hudi/pull/5937#issuecomment-1167529577 ## CI report: * 5b84e8069a40f65207e461307c5e3a02b0919f65 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9585

[GitHub] [hudi] minihippo commented on issue #5904: [SUPPORT] Spark saveAsTable() not working as expected

2022-06-27 Thread GitBox
minihippo commented on issue #5904: URL: https://github.com/apache/hudi/issues/5904#issuecomment-1167498998 @kazdy Could you use `save()` api as workaround? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hudi] minihippo commented on issue #5612: Exception while upserting records - An error occurred while calling o155.pyWriteDynamicFrame. Expected instance of group converter but got "org.apac

2022-06-27 Thread GitBox
minihippo commented on issue #5612: URL: https://github.com/apache/hudi/issues/5612#issuecomment-1167496584 @gtwuser Can I close this issue, and we talk about the unsolved problem in the other issue which already using Spark 3.1? -- This is an automated message from the Apache Git Service

[GitHub] [hudi] minihippo commented on issue #5701: [SUPPORT]hudi how to upsert a non null array data to a existing column with array of nulls,optional binary. java.lang.ClassCastException: optional b

2022-06-27 Thread GitBox
minihippo commented on issue #5701: URL: https://github.com/apache/hudi/issues/5701#issuecomment-1167494245 @gtwuser For the first bulk insert, are values of `NWDepStatus` empty? If it is, u could try to only write other columns into the table -- This is an automated message from the

[GitHub] [hudi] minihippo commented on issue #5915: [SUPPORT] Schema Evolution - Changing field type from Date to String is not supported

2022-06-27 Thread GitBox
minihippo commented on issue #5915: URL: https://github.com/apache/hudi/issues/5915#issuecomment-1167479309 @Reimus Has your problem been solved? if it is, feel free to close the issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] minihippo commented on issue #5916: [SUPPORT] `show fsview latest` throwing IllegalStateException...pending compactions for merge_on_read table

2022-06-27 Thread GitBox
minihippo commented on issue #5916: URL: https://github.com/apache/hudi/issues/5916#issuecomment-1167472691 @amit-ranjan-de Cloud u find 20220615161228.compaction and 20220615172826.compaction in the timeline? -- This is an automated message from the Apache Git Service. To respond to the

[jira] [Closed] (HUDI-4295) Test TestHoodieActiveTimeline.testCreateNewInstantTime is flaky

2022-06-27 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-4295. - Resolution: Fixed > Test TestHoodieActiveTimeline.testCreateNewInstantTime is flaky >

[jira] [Updated] (HUDI-4296) Test TestHoodieSparkSqlWriter.testSchemaEvolutionForTableType is flaky

2022-06-27 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4296: -- Status: Patch Available (was: In Progress) > Test TestHoodieSparkSqlWriter.testSchemaEvolutionForTableT

[jira] [Closed] (HUDI-4296) Test TestHoodieSparkSqlWriter.testSchemaEvolutionForTableType is flaky

2022-06-27 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-4296. - Resolution: Fixed > Test TestHoodieSparkSqlWriter.testSchemaEvolutionForTableType is flaky > -

[jira] [Closed] (HUDI-4291) Test TestCleanPlanExecutor.testKeepLatestFileVersions is flaky

2022-06-27 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-4291. - Resolution: Fixed > Test TestCleanPlanExecutor.testKeepLatestFileVersions is flaky > -

[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…

2022-06-27 Thread GitBox
hudi-bot commented on PR #5884: URL: https://github.com/apache/hudi/pull/5884#issuecomment-1167395342 ## CI report: * 8cbe40dfa825fbf8dca54cf7fda4b0893441e14e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9584

[GitHub] [hudi] hudi-bot commented on pull request #5974: [HUDI-3506] Add call procedure for CommitsCommand

2022-06-27 Thread GitBox
hudi-bot commented on PR #5974: URL: https://github.com/apache/hudi/pull/5974#issuecomment-1167379643 ## CI report: * 104cce4132b75da702d0fc94c21d325a399d38c8 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9582

[jira] [Created] (HUDI-4329) Add separate control for compaction operation sync/async mode

2022-06-27 Thread Shizhi Chen (Jira)
Shizhi Chen created HUDI-4329: - Summary: Add separate control for compaction operation sync/async mode Key: HUDI-4329 URL: https://issues.apache.org/jira/browse/HUDI-4329 Project: Apache Hudi Is

[GitHub] [hudi] bkosuru commented on issue #5983: [SUPPORT] No partition pruning for Incremental Query

2022-06-27 Thread GitBox
bkosuru commented on issue #5983: URL: https://github.com/apache/hudi/issues/5983#issuecomment-1167320212 Same issue in 0.8.0 version -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[jira] [Updated] (HUDI-4277) Support computed column for flink HoodieTableSource

2022-06-27 Thread Shizhi Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shizhi Chen updated HUDI-4277: -- Issue Type: Improvement (was: Bug) > Support computed column for flink HoodieTableSource > ---

[GitHub] [hudi] danny0405 commented on a diff in pull request #5937: [HUDI-4298] When reading the mor table with QUERY_TYPE_SNAPSHOT,Unabl…

2022-06-27 Thread GitBox
danny0405 commented on code in PR #5937: URL: https://github.com/apache/hudi/pull/5937#discussion_r907354859 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/utils/TestWriteBase.java: ## @@ -353,6 +361,21 @@ private void checkWrittenDataMor(File baseFile,

[jira] [Updated] (HUDI-4287) Optimize Flink checkpoint meta mechanism to fix mistaken pending instants

2022-06-27 Thread Shizhi Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shizhi Chen updated HUDI-4287: -- Status: In Progress (was: Open) > Optimize Flink checkpoint meta mechanism to fix mistaken pending inst

[jira] [Commented] (HUDI-4287) Optimize Flink checkpoint meta mechanism to fix mistaken pending instants

2022-06-27 Thread Shizhi Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17559150#comment-17559150 ] Shizhi Chen commented on HUDI-4287: --- [~danny0405] details added. > Optimize Flink check

[GitHub] [hudi] danny0405 commented on a diff in pull request #5937: [HUDI-4298] When reading the mor table with QUERY_TYPE_SNAPSHOT,Unabl…

2022-06-27 Thread GitBox
danny0405 commented on code in PR #5937: URL: https://github.com/apache/hudi/pull/5937#discussion_r907354859 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/utils/TestWriteBase.java: ## @@ -353,6 +361,21 @@ private void checkWrittenDataMor(File baseFile,

[jira] [Created] (HUDI-4328) New partitions will not be consumed in snapshot read

2022-06-27 Thread lei w (Jira)
lei w created HUDI-4328: --- Summary: New partitions will not be consumed in snapshot read Key: HUDI-4328 URL: https://issues.apache.org/jira/browse/HUDI-4328 Project: Apache Hudi Issue Type: Bug

[jira] [Updated] (HUDI-4287) Optimize Flink checkpoint meta mechanism to fix mistaken pending instants

2022-06-27 Thread Shizhi Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shizhi Chen updated HUDI-4287: -- Description: *Problem reveiw* CkpMetadata is introduced into flink module to reduce timeline burden, bu

[jira] [Updated] (HUDI-4287) Optimize Flink checkpoint meta mechanism to fix mistaken pending instants

2022-06-27 Thread Shizhi Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shizhi Chen updated HUDI-4287: -- Description: *Problem reveiw* CkpMetadata is introduced into flink module to reduce timeline burden, bu

[GitHub] [hudi] hudi-bot commented on pull request #5608: [HUDI-2150] Rename/Restructure configs for better modularity

2022-06-27 Thread GitBox
hudi-bot commented on PR #5608: URL: https://github.com/apache/hudi/pull/5608#issuecomment-1167307449 ## CI report: * 9d2b850bb1eaee395b456ca818e399cbf1125982 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9580

  1   2   >