[GitHub] [hudi] hudi-bot commented on pull request #6989: [HUDI-5000] Support schema evolution for Hive/presto

2022-10-28 Thread GitBox
hudi-bot commented on PR #6989: URL: https://github.com/apache/hudi/pull/6989#issuecomment-1294568865 ## CI report: * ce351e5cffcc7a9bcbdee2ea07bd07390f70b93c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1256

[GitHub] [hudi] hudi-bot commented on pull request #7011: [HUDI-5102] source operator(monitor and reader) support user uid

2022-10-28 Thread GitBox
hudi-bot commented on PR #7011: URL: https://github.com/apache/hudi/pull/7011#issuecomment-1294569125 ## CI report: * 7ee57b09c114d8c625c1592e28178e8b40aca4e1 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=126

[GitHub] [hudi] hudi-bot commented on pull request #7036: [HUDI-5076] Fixing non serializable path used in engineContext with metadata table intialization

2022-10-28 Thread GitBox
hudi-bot commented on PR #7036: URL: https://github.com/apache/hudi/pull/7036#issuecomment-1294569285 ## CI report: * 59bfa926b165365cdd27cedd42ed1c35c687da26 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1247

[GitHub] [hudi] hudi-bot commented on pull request #7068: [HUDI-5096] boolean param is broken in HiveSyncTool

2022-10-28 Thread GitBox
hudi-bot commented on PR #7068: URL: https://github.com/apache/hudi/pull/7068#issuecomment-1294569583 ## CI report: * d64d4f129371549e68315c6e313b1a9b11164759 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1263

[GitHub] [hudi] hudi-bot commented on pull request #6989: [HUDI-5000] Support schema evolution for Hive/presto

2022-10-28 Thread GitBox
hudi-bot commented on PR #6989: URL: https://github.com/apache/hudi/pull/6989#issuecomment-1294580441 ## CI report: * ce351e5cffcc7a9bcbdee2ea07bd07390f70b93c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1256

[GitHub] [hudi] hudi-bot commented on pull request #7011: [HUDI-5102] source operator(monitor and reader) support user uid

2022-10-28 Thread GitBox
hudi-bot commented on PR #7011: URL: https://github.com/apache/hudi/pull/7011#issuecomment-1294580705 ## CI report: * c15524e44895edccc80a6fba5b93563b842e32e8 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1263

[GitHub] [hudi] hudi-bot commented on pull request #7068: [HUDI-5096] boolean param is broken in HiveSyncTool

2022-10-28 Thread GitBox
hudi-bot commented on PR #7068: URL: https://github.com/apache/hudi/pull/7068#issuecomment-1294581156 ## CI report: * d64d4f129371549e68315c6e313b1a9b11164759 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1263

[GitHub] [hudi] danny0405 commented on a diff in pull request #7042: [HUDI-5082] Improve the cdc log file name format

2022-10-28 Thread GitBox
danny0405 commented on code in PR #7042: URL: https://github.com/apache/hudi/pull/7042#discussion_r1007744473 ## hudi-common/src/main/java/org/apache/hudi/common/fs/FSUtils.java: ## @@ -79,7 +80,7 @@ public class FSUtils { private static final Logger LOG = LogManager.getLogge

[GitHub] [hudi] hudi-bot commented on pull request #6989: [HUDI-5000] Support schema evolution for Hive/presto

2022-10-28 Thread GitBox
hudi-bot commented on PR #6989: URL: https://github.com/apache/hudi/pull/6989#issuecomment-1294591239 ## CI report: * d76cb8da4634364b540faf91b66c2fc9644476c4 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=126

[GitHub] [hudi] hudi-bot commented on pull request #7068: [HUDI-5096] boolean param is broken in HiveSyncTool

2022-10-28 Thread GitBox
hudi-bot commented on PR #7068: URL: https://github.com/apache/hudi/pull/7068#issuecomment-1294591624 ## CI report: * d64d4f129371549e68315c6e313b1a9b11164759 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1263

[GitHub] [hudi] xicm commented on a diff in pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true

2022-10-28 Thread GitBox
xicm commented on code in PR #6986: URL: https://github.com/apache/hudi/pull/6986#discussion_r1007788563 ## hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java: ## @@ -874,21 +887,30 @@ private static Object rewritePrimaryTypeWithDiffSchemaType(Object oldValue,

[GitHub] [hudi] xicm commented on a diff in pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true

2022-10-28 Thread GitBox
xicm commented on code in PR #6986: URL: https://github.com/apache/hudi/pull/6986#discussion_r1007788563 ## hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java: ## @@ -874,21 +887,30 @@ private static Object rewritePrimaryTypeWithDiffSchemaType(Object oldValue,

[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning

2022-10-28 Thread GitBox
hudi-bot commented on PR #7041: URL: https://github.com/apache/hudi/pull/7041#issuecomment-1294711267 ## CI report: * acd416d779132b9fd7a7b1fe58eaaeebcf1b821f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1249

[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning

2022-10-28 Thread GitBox
hudi-bot commented on PR #7041: URL: https://github.com/apache/hudi/pull/7041#issuecomment-1294717454 ## CI report: * acd416d779132b9fd7a7b1fe58eaaeebcf1b821f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1249

[GitHub] [hudi] danny0405 commented on pull request #7011: [HUDI-5102] source operator(monitor and reader) support user uid

2022-10-28 Thread GitBox
danny0405 commented on PR #7011: URL: https://github.com/apache/hudi/pull/7011#issuecomment-1294734269 Hi, can you rebase with the latest master and force push with the branch ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] parisni opened a new pull request, #7082: Turn metadata table enabled for readers

2022-10-28 Thread GitBox
parisni opened a new pull request, #7082: URL: https://github.com/apache/hudi/pull/7082 currently MDT is enabled by default for writers however reader don't benefit from it by default. this makes the need to add .option(`hoodie.metadata.enabled`,`true`) in a given source code when migr

[GitHub] [hudi] lvhu-goodluck opened a new pull request, #7083: Fix typos in Spark client related classes

2022-10-28 Thread GitBox
lvhu-goodluck opened a new pull request, #7083: URL: https://github.com/apache/hudi/pull/7083 ### What is the purpose of the pull request This PR fixes a few typos in Spark client related classes. ### Risk level (write none, low medium or high below) none ### Contributor's

[GitHub] [hudi] lvhu-goodluck commented on pull request #7083: Fix typos in Spark client related classes

2022-10-28 Thread GitBox
lvhu-goodluck commented on PR #7083: URL: https://github.com/apache/hudi/pull/7083#issuecomment-1294760171 This pull request is a trivial rework / code cleanup without any test coverage. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [hudi] Zouxxyy opened a new pull request, #7084: [HUDI-5057][FOLLOWUP] Fix msck repair hudi table

2022-10-28 Thread GitBox
Zouxxyy opened a new pull request, #7084: URL: https://github.com/apache/hudi/pull/7084 ### Change Logs use hoodieCatalogTable.tableConfig to get configs ### Impact none ### Risk level (write none, low medium or high below) low ### Documentation Updat

[jira] [Assigned] (HUDI-5095) Flink: Stores a special watermark(flag) to identify the current progress of writing data

2022-10-28 Thread Forward Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Forward Xu reassigned HUDI-5095: Assignee: yuemeng (was: Forward Xu) > Flink: Stores a special watermark(flag) to identify the curr

[GitHub] [hudi] Zouxxyy commented on pull request #7084: [HUDI-5057][FOLLOWUP] Fix msck repair hudi table

2022-10-28 Thread GitBox
Zouxxyy commented on PR #7084: URL: https://github.com/apache/hudi/pull/7084#issuecomment-1294768535 @YannByron Can you help review it~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [hudi] YannByron commented on a diff in pull request #7042: [HUDI-5082] Improve the cdc log file name format

2022-10-28 Thread GitBox
YannByron commented on code in PR #7042: URL: https://github.com/apache/hudi/pull/7042#discussion_r1007877496 ## hudi-common/src/main/java/org/apache/hudi/common/fs/FSUtils.java: ## @@ -79,7 +80,7 @@ public class FSUtils { private static final Logger LOG = LogManager.getLogge

[GitHub] [hudi] 1032851561 commented on pull request #7011: [HUDI-5102] source operator(monitor and reader) support user uid

2022-10-28 Thread GitBox
1032851561 commented on PR #7011: URL: https://github.com/apache/hudi/pull/7011#issuecomment-1294789721 > -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

[GitHub] [hudi] 1032851561 closed pull request #7011: [HUDI-5102] source operator(monitor and reader) support user uid

2022-10-28 Thread GitBox
1032851561 closed pull request #7011: [HUDI-5102] source operator(monitor and reader) support user uid URL: https://github.com/apache/hudi/pull/7011 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [hudi] xicm commented on a diff in pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true

2022-10-28 Thread GitBox
xicm commented on code in PR #6986: URL: https://github.com/apache/hudi/pull/6986#discussion_r1007788563 ## hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java: ## @@ -874,21 +887,30 @@ private static Object rewritePrimaryTypeWithDiffSchemaType(Object oldValue,

[GitHub] [hudi] 1032851561 commented on pull request #7011: [HUDI-5102] source operator(monitor and reader) support user uid

2022-10-28 Thread GitBox
1032851561 commented on PR #7011: URL: https://github.com/apache/hudi/pull/7011#issuecomment-1294795054 > Hi, can you rebase with the latest master and force push with the branch ? I have create new branch "fix_op_uid" . Do I pull request again or directly push branch? -- T

[GitHub] [hudi] hudi-bot commented on pull request #6448: [HUDI-4647] Keep the hive sync settings in spark sql consistent

2022-10-28 Thread GitBox
hudi-bot commented on PR #6448: URL: https://github.com/apache/hudi/pull/6448#issuecomment-1294795668 ## CI report: * 745b577fa1a032be4290b2b5cc08095f613ec87e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1263

[GitHub] [hudi] hudi-bot commented on pull request #7082: Turn metadata table enabled for readers

2022-10-28 Thread GitBox
hudi-bot commented on PR #7082: URL: https://github.com/apache/hudi/pull/7082#issuecomment-1294804256 ## CI report: * f587991b736626ff301eec1b80383ac6436e1f6b UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7083: Fix typos in Spark client related classes

2022-10-28 Thread GitBox
hudi-bot commented on PR #7083: URL: https://github.com/apache/hudi/pull/7083#issuecomment-1294804326 ## CI report: * 954a0e36c2f6ab9f7a845cb0892a83220efe087a UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7084: [HUDI-5057][FOLLOWUP] Fix msck repair hudi table

2022-10-28 Thread GitBox
hudi-bot commented on PR #7084: URL: https://github.com/apache/hudi/pull/7084#issuecomment-1294804399 ## CI report: * 39614fb44f235a894e3e6329432f958c73b0b6d5 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] 1032851561 opened a new pull request, #7085: [HUDI-5102] source operator(monitor and reader) support user uid

2022-10-28 Thread GitBox
1032851561 opened a new pull request, #7085: URL: https://github.com/apache/hudi/pull/7085 source operator(monitor and reader) support user uid ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute

[GitHub] [hudi] hudi-bot commented on pull request #7011: [HUDI-5102] source operator(monitor and reader) support user uid

2022-10-28 Thread GitBox
hudi-bot commented on PR #7011: URL: https://github.com/apache/hudi/pull/7011#issuecomment-1294809832 ## CI report: * c15524e44895edccc80a6fba5b93563b842e32e8 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1263

[GitHub] [hudi] hudi-bot commented on pull request #7082: Turn metadata table enabled for readers

2022-10-28 Thread GitBox
hudi-bot commented on PR #7082: URL: https://github.com/apache/hudi/pull/7082#issuecomment-1294810086 ## CI report: * f587991b736626ff301eec1b80383ac6436e1f6b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1264

[GitHub] [hudi] hudi-bot commented on pull request #7083: Fix typos in Spark client related classes

2022-10-28 Thread GitBox
hudi-bot commented on PR #7083: URL: https://github.com/apache/hudi/pull/7083#issuecomment-1294810125 ## CI report: * 954a0e36c2f6ab9f7a845cb0892a83220efe087a Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1264

[GitHub] [hudi] hudi-bot commented on pull request #7084: [HUDI-5057][FOLLOWUP] Fix msck repair hudi table

2022-10-28 Thread GitBox
hudi-bot commented on PR #7084: URL: https://github.com/apache/hudi/pull/7084#issuecomment-1294810173 ## CI report: * 39614fb44f235a894e3e6329432f958c73b0b6d5 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1264

[GitHub] [hudi] hudi-bot commented on pull request #7085: [HUDI-5102] source operator(monitor and reader) support user uid

2022-10-28 Thread GitBox
hudi-bot commented on PR #7085: URL: https://github.com/apache/hudi/pull/7085#issuecomment-1294810209 ## CI report: * c15524e44895edccc80a6fba5b93563b842e32e8 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] dongkelun commented on pull request #6448: [HUDI-4647] Keep the hive sync settings in spark sql consistent

2022-10-28 Thread GitBox
dongkelun commented on PR #6448: URL: https://github.com/apache/hudi/pull/6448#issuecomment-1294812311 The job running on agent Azure Pipelines 8 ran longer than the maximum time of 150 minutes -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning

2022-10-28 Thread GitBox
hudi-bot commented on PR #7041: URL: https://github.com/apache/hudi/pull/7041#issuecomment-1294816115 ## CI report: * acd416d779132b9fd7a7b1fe58eaaeebcf1b821f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1249

[GitHub] [hudi] hudi-bot commented on pull request #7085: [HUDI-5102] source operator(monitor and reader) support user uid

2022-10-28 Thread GitBox
hudi-bot commented on PR #7085: URL: https://github.com/apache/hudi/pull/7085#issuecomment-1294816387 ## CI report: * c15524e44895edccc80a6fba5b93563b842e32e8 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1264

[GitHub] [hudi] xushiyan commented on pull request #6079: [HUDI-3287] Remove hudi-spark dependencies from hudi-kafka-connect-bundle

2022-10-28 Thread GitBox
xushiyan commented on PR #6079: URL: https://github.com/apache/hudi/pull/6079#issuecomment-1294824072 verified locally with the bundle jar and it's working ok -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [hudi] xushiyan merged pull request #6079: [HUDI-3287] Remove hudi-spark dependencies from hudi-kafka-connect-bundle

2022-10-28 Thread GitBox
xushiyan merged PR #6079: URL: https://github.com/apache/hudi/pull/6079 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.

[hudi] branch master updated (ae541ba0a5 -> c6e5fd0271)

2022-10-28 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from ae541ba0a5 [HUDI-5072] Extract `ExecutionStrategy#transform` duplicate code (#7030) add c6e5fd0271 [HUDI-3287] R

[GitHub] [hudi] Aload commented on issue #6618: Caused by: org.apache.http.NoHttpResponseException: xxxxxx:34812 failed to respond[SUPPORT]

2022-10-28 Thread GitBox
Aload commented on issue #6618: URL: https://github.com/apache/hudi/issues/6618#issuecomment-1294829064 > @danny0405 @yuzhaojing : have you guys seen the exception before. any pointers. yes I had the same problem when I was in 0.11 -- This is an automated message from the Apache Gi

[GitHub] [hudi] eric9204 commented on issue #6966: [SUPPORT]HoodieWriteHandle: Error writing record HoodieRecord{key=HoodieKey { recordKey=id308723 partitionPath=202210141643}, currentLocation='null',

2022-10-28 Thread GitBox
eric9204 commented on issue #6966: URL: https://github.com/apache/hudi/issues/6966#issuecomment-1294832847 @fengjian428 This case can reproduce above issue ``` import java.io.EOFException import org.apache.spark.SparkConf import org.apache.spark.sql.SparkSession import org

[jira] [Updated] (HUDI-4982) Make bundle combination testing covered in CI

2022-10-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4982: - Epic Link: HUDI-3303 (was: HUDI-3529) > Make bundle combination testing covered in CI > -

[jira] [Updated] (HUDI-2673) Add integration/e2e test for kafka-connect functionality

2022-10-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2673: - Sprint: Hudi-Sprint-Apr-19, Hudi-Sprint-Apr-25, 2022/05/02, 2022/05/16, 2022/11/01 (was: Hudi-Sprint-Apr-

[jira] [Closed] (HUDI-3287) Remove unnecessary deps in hudi-kafka-connect

2022-10-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-3287. Resolution: Fixed > Remove unnecessary deps in hudi-kafka-connect >

[GitHub] [hudi] chenshzh commented on a diff in pull request #7017: [HUDI-5066] Support flink hoodie source metaclient cache

2022-10-28 Thread GitBox
chenshzh commented on code in PR #7017: URL: https://github.com/apache/hudi/pull/7017#discussion_r1007944652 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableSource.java: ## @@ -162,7 +163,7 @@ public HoodieTableSource( this.limit = limit =

[GitHub] [hudi] hudi-bot commented on pull request #6448: [HUDI-4647] Keep the hive sync settings in spark sql consistent

2022-10-28 Thread GitBox
hudi-bot commented on PR #6448: URL: https://github.com/apache/hudi/pull/6448#issuecomment-1294871652 ## CI report: * 745b577fa1a032be4290b2b5cc08095f613ec87e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1263

[GitHub] [hudi] hudi-bot commented on pull request #6989: [HUDI-5000] Support schema evolution for Hive/presto

2022-10-28 Thread GitBox
hudi-bot commented on PR #6989: URL: https://github.com/apache/hudi/pull/6989#issuecomment-1294872409 ## CI report: * 552ad3ddcc716636f2a783125ff5c140c56e6326 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1263

[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning

2022-10-28 Thread GitBox
hudi-bot commented on PR #7041: URL: https://github.com/apache/hudi/pull/7041#issuecomment-1294872589 ## CI report: * 00f351db8eab5af32063e4a41c4e451ca9838ee4 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=126

[GitHub] [hudi] hudi-bot commented on pull request #7068: [HUDI-5096] boolean param is broken in HiveSyncTool

2022-10-28 Thread GitBox
hudi-bot commented on PR #7068: URL: https://github.com/apache/hudi/pull/7068#issuecomment-1294872701 ## CI report: * 5f5752509e76de38a11d6c9af1efbddaacdb020b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1264

[GitHub] [hudi] xiarixiaoyao commented on pull request #6989: [HUDI-5000] Support schema evolution for Hive/presto

2022-10-28 Thread GitBox
xiarixiaoyao commented on PR #6989: URL: https://github.com/apache/hudi/pull/6989#issuecomment-1294876856 @fsilent Thank you for your contribution, pls raise another pr to update docs -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [hudi] hudi-bot commented on pull request #6448: [HUDI-4647] Keep the hive sync settings in spark sql consistent

2022-10-28 Thread GitBox
hudi-bot commented on PR #6448: URL: https://github.com/apache/hudi/pull/6448#issuecomment-1294877050 ## CI report: * 745b577fa1a032be4290b2b5cc08095f613ec87e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1263

[GitHub] [hudi] xiarixiaoyao merged pull request #6989: [HUDI-5000] Support schema evolution for Hive/presto

2022-10-28 Thread GitBox
xiarixiaoyao merged PR #6989: URL: https://github.com/apache/hudi/pull/6989 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apa

[hudi] branch master updated (c6e5fd0271 -> 48d23ff233)

2022-10-28 Thread mengtao
This is an automated email from the ASF dual-hosted git repository. mengtao pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from c6e5fd0271 [HUDI-3287] Remove hudi-spark dependencies from hudi-kafka-connect-bundle (#6079) add 48d23ff233 [HUDI

[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning

2022-10-28 Thread GitBox
hudi-bot commented on PR #7041: URL: https://github.com/apache/hudi/pull/7041#issuecomment-1294883514 ## CI report: * 00f351db8eab5af32063e4a41c4e451ca9838ee4 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=126

[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning

2022-10-28 Thread GitBox
hudi-bot commented on PR #7041: URL: https://github.com/apache/hudi/pull/7041#issuecomment-1294888771 ## CI report: * 74e7a278867b56d1b6a6bbee29fb477469246487 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=126

[jira] [Created] (HUDI-5103) Does not work with Azure Data lake Gen2

2022-10-28 Thread Dheeraj Panangat (Jira)
Dheeraj Panangat created HUDI-5103: -- Summary: Does not work with Azure Data lake Gen2 Key: HUDI-5103 URL: https://issues.apache.org/jira/browse/HUDI-5103 Project: Apache Hudi Issue Type: Bug

[GitHub] [hudi] hudi-bot commented on pull request #7082: Turn metadata table enabled for readers

2022-10-28 Thread GitBox
hudi-bot commented on PR #7082: URL: https://github.com/apache/hudi/pull/7082#issuecomment-1294946525 ## CI report: * f587991b736626ff301eec1b80383ac6436e1f6b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1264

[GitHub] [hudi] xushiyan commented on pull request #6933: [HUDI-4954] Shade avro and parquet-avro in all bundles

2022-10-28 Thread GitBox
xushiyan commented on PR #6933: URL: https://github.com/apache/hudi/pull/6933#issuecomment-1294988852 both HoodieSparkSqlWriter and DeltaSync register avro schema via `org.apache.spark.SparkConf#registerAvroSchemas`, which expects unshaded avro schema class, hence resulted in class casting

[GitHub] [hudi] xicm commented on pull request #7068: [HUDI-5096] boolean param is broken in HiveSyncTool

2022-10-28 Thread GitBox
xicm commented on PR #7068: URL: https://github.com/apache/hudi/pull/7068#issuecomment-1294996051 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

[GitHub] [hudi] nochimow closed issue #4299: [SUPPORT] Upsert performance decreased after 3 years of data loading

2022-10-28 Thread GitBox
nochimow closed issue #4299: [SUPPORT] Upsert performance decreased after 3 years of data loading URL: https://github.com/apache/hudi/issues/4299 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] hudi-bot commented on pull request #7068: [HUDI-5096] boolean param is broken in HiveSyncTool

2022-10-28 Thread GitBox
hudi-bot commented on PR #7068: URL: https://github.com/apache/hudi/pull/7068#issuecomment-1295027763 ## CI report: * 5f5752509e76de38a11d6c9af1efbddaacdb020b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1264

[GitHub] [hudi] hudi-bot commented on pull request #7083: Fix typos in Spark client related classes

2022-10-28 Thread GitBox
hudi-bot commented on PR #7083: URL: https://github.com/apache/hudi/pull/7083#issuecomment-1295027903 ## CI report: * 954a0e36c2f6ab9f7a845cb0892a83220efe087a Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1264

[GitHub] [hudi] nochimow commented on issue #6811: [SUPPORT] Slow upsert performance

2022-10-28 Thread GitBox
nochimow commented on issue #6811: URL: https://github.com/apache/hudi/issues/6811#issuecomment-1295029331 @ctlgdanielli -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] hudi-bot commented on pull request #7084: [HUDI-5057][FOLLOWUP] Fix msck repair hudi table

2022-10-28 Thread GitBox
hudi-bot commented on PR #7084: URL: https://github.com/apache/hudi/pull/7084#issuecomment-1295049006 ## CI report: * 39614fb44f235a894e3e6329432f958c73b0b6d5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1264

[GitHub] [hudi] xushiyan merged pull request #6930: [HUDI-4716] Avoid parquet-hadoop-bundle in hudi-hadoop-mr

2022-10-28 Thread GitBox
xushiyan merged PR #6930: URL: https://github.com/apache/hudi/pull/6930 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.

[GitHub] [hudi] xushiyan commented on pull request #6930: [HUDI-4716] Avoid parquet-hadoop-bundle in hudi-hadoop-mr

2022-10-28 Thread GitBox
xushiyan commented on PR #6930: URL: https://github.com/apache/hudi/pull/6930#issuecomment-1295080530 tested basic COW query with hadoop-mr-bundle (included parquet-avro 1.10.1) and query worked with hive 2.3.9. so the compatibility is verified -- This is an automated message from the Apa

[hudi] branch master updated: [HUDI-4716] Avoid parquet-hadoop-bundle in hudi-hadoop-mr (#6930)

2022-10-28 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new adf53806e8 [HUDI-4716] Avoid parquet-hadoop-bund

[jira] [Closed] (HUDI-4716) Avoid bundle parquet in hadoop-mr

2022-10-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-4716. Resolution: Fixed > Avoid bundle parquet in hadoop-mr > - > >

[GitHub] [hudi] hudi-bot commented on pull request #7085: [HUDI-5102] source operator(monitor and reader) support user uid

2022-10-28 Thread GitBox
hudi-bot commented on PR #7085: URL: https://github.com/apache/hudi/pull/7085#issuecomment-1295116182 ## CI report: * c15524e44895edccc80a6fba5b93563b842e32e8 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1264

[GitHub] [hudi] jonvex opened a new pull request, #7086: [HUDI-4624] Implement Closable for S3EventsSource

2022-10-28 Thread GitBox
jonvex opened a new pull request, #7086: URL: https://github.com/apache/hudi/pull/7086 ### Change Logs We are not freeing resources properly in S3EventsSource. The SQS queue is now shutdown in close() ### Impact _Describe any public API or user-facing feature change or a

[jira] [Updated] (HUDI-4624) Make sure all DeltaStreamer Sources are Closeable

2022-10-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4624: - Labels: pull-request-available (was: ) > Make sure all DeltaStreamer Sources are Closeable >

[jira] [Updated] (HUDI-5074) Auto convert table names to lower case due to hive sync issues

2022-10-28 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5074: -- Status: Patch Available (was: In Progress) > Auto convert table names to lower case due to hive

[jira] [Updated] (HUDI-4624) Make sure all DeltaStreamer Sources are Closeable

2022-10-28 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-4624: -- Status: Patch Available (was: In Progress) > Make sure all DeltaStreamer Sources are Closeable

[jira] [Updated] (HUDI-4624) Make sure all DeltaStreamer Sources are Closeable

2022-10-28 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-4624: -- Status: In Progress (was: Open) > Make sure all DeltaStreamer Sources are Closeable > -

[GitHub] [hudi] hudi-bot commented on pull request #6448: [HUDI-4647] Keep the hive sync settings in spark sql consistent

2022-10-28 Thread GitBox
hudi-bot commented on PR #6448: URL: https://github.com/apache/hudi/pull/6448#issuecomment-1295201356 ## CI report: * c51f59c9afa364b39e9a74ff74f2bb5e8fad4e42 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1264

[GitHub] [hudi] hudi-bot commented on pull request #7086: [HUDI-4624] Implement Closable for S3EventsSource

2022-10-28 Thread GitBox
hudi-bot commented on PR #7086: URL: https://github.com/apache/hudi/pull/7086#issuecomment-1295202521 ## CI report: * 7afa3cff8e8157d2962afd482163ac7eca5b021d UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #6926: [HUDI-3676] Enhance tests for trigger clean every Nth commit

2022-10-28 Thread GitBox
hudi-bot commented on PR #6926: URL: https://github.com/apache/hudi/pull/6926#issuecomment-1295207937 ## CI report: * 76bc62cf70c15699c11989eb9169bf41c3483974 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1261

[GitHub] [hudi] hudi-bot commented on pull request #7086: [HUDI-4624] Implement Closable for S3EventsSource

2022-10-28 Thread GitBox
hudi-bot commented on PR #7086: URL: https://github.com/apache/hudi/pull/7086#issuecomment-1295209085 ## CI report: * 7afa3cff8e8157d2962afd482163ac7eca5b021d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1265

[GitHub] [hudi] hudi-bot commented on pull request #6926: [HUDI-3676] Enhance tests for trigger clean every Nth commit

2022-10-28 Thread GitBox
hudi-bot commented on PR #6926: URL: https://github.com/apache/hudi/pull/6926#issuecomment-1295220612 ## CI report: * 76bc62cf70c15699c11989eb9169bf41c3483974 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1261

[hudi] branch master updated: [HUDI-5035] Remove usage of deprecated HoodieTimer constructor (#6952)

2022-10-28 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 6be2057376 [HUDI-5035] Remove usage of deprecated H

[GitHub] [hudi] yihua merged pull request #6952: [HUDI-5035] Remove usage of deprecated HoodieTimer constructor

2022-10-28 Thread GitBox
yihua merged PR #6952: URL: https://github.com/apache/hudi/pull/6952 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning

2022-10-28 Thread GitBox
hudi-bot commented on PR #7041: URL: https://github.com/apache/hudi/pull/7041#issuecomment-1295283476 ## CI report: * a78cf3612346492a082bf211940c46c9e82752a1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1265

[GitHub] [hudi] pratyakshsharma opened a new pull request, #7087: [minor]: Fix creation of timeline instants in cleaner test cases

2022-10-28 Thread GitBox
pratyakshsharma opened a new pull request, #7087: URL: https://github.com/apache/hudi/pull/7087 ### Change Logs Fix creation of timeline instants to have proper increasing order in the below test cases in TestCleanPlanExecutor - - testKeepLatestFileVersionsWithBootstrapFileClean

[GitHub] [hudi] hudi-bot commented on pull request #7087: [minor]: Fix creation of timeline instants in cleaner test cases

2022-10-28 Thread GitBox
hudi-bot commented on PR #7087: URL: https://github.com/apache/hudi/pull/7087#issuecomment-1295299669 ## CI report: * c3eb4a4a4581adb693ba8d2baa54e7c490fafd2d UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7087: [minor]: Fix creation of timeline instants in cleaner test cases

2022-10-28 Thread GitBox
hudi-bot commented on PR #7087: URL: https://github.com/apache/hudi/pull/7087#issuecomment-1295304383 ## CI report: * c3eb4a4a4581adb693ba8d2baa54e7c490fafd2d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1265

[GitHub] [hudi] xushiyan commented on a diff in pull request #7071: [HUDI-4982] Upgrade Bundle Testing

2022-10-28 Thread GitBox
xushiyan commented on code in PR #7071: URL: https://github.com/apache/hudi/pull/7071#discussion_r1008346010 ## packaging/bundle-validation/Dockerfile-base: ## @@ -47,3 +49,44 @@ RUN wget https://archive.apache.org/dist/spark/spark-$SPARK_VERSION/spark-$SPARK && tar -xf $

[GitHub] [hudi] xushiyan commented on a diff in pull request #7068: [HUDI-5096] boolean param is broken in HiveSyncTool

2022-10-28 Thread GitBox
xushiyan commented on code in PR #7068: URL: https://github.com/apache/hudi/pull/7068#discussion_r1008352259 ## hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncConfig.java: ## @@ -108,45 +108,45 @@ public static class HiveSyncConfigParams { + "in

[GitHub] [hudi] xushiyan commented on a diff in pull request #7068: [HUDI-5096] boolean param is broken in HiveSyncTool

2022-10-28 Thread GitBox
xushiyan commented on code in PR #7068: URL: https://github.com/apache/hudi/pull/7068#discussion_r1008352259 ## hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncConfig.java: ## @@ -108,45 +108,45 @@ public static class HiveSyncConfigParams { + "in

[GitHub] [hudi] hudi-bot commented on pull request #7068: [HUDI-5096] boolean param is broken in HiveSyncTool

2022-10-28 Thread GitBox
hudi-bot commented on PR #7068: URL: https://github.com/apache/hudi/pull/7068#issuecomment-1295354912 ## CI report: * 5f5752509e76de38a11d6c9af1efbddaacdb020b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1264

[GitHub] [hudi] nsivabalan commented on pull request #5416: [HUDI-3963] Use Lock-Free Message Queue Disruptor Improving Hoodie Writing Efficiency

2022-10-28 Thread GitBox
nsivabalan commented on PR #5416: URL: https://github.com/apache/hudi/pull/5416#issuecomment-1295355705 will review by tomorrow. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[GitHub] [hudi] hudi-bot commented on pull request #7086: [HUDI-4624] Implement Closable for S3EventsSource

2022-10-28 Thread GitBox
hudi-bot commented on PR #7086: URL: https://github.com/apache/hudi/pull/7086#issuecomment-1295429133 ## CI report: * 7afa3cff8e8157d2962afd482163ac7eca5b021d Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1265

[GitHub] [hudi] hudi-bot commented on pull request #6926: [HUDI-3676] Enhance tests for trigger clean every Nth commit

2022-10-28 Thread GitBox
hudi-bot commented on PR #6926: URL: https://github.com/apache/hudi/pull/6926#issuecomment-1295495489 ## CI report: * 620cd686b307a74af7f0b8b35279661c25fad24d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1265

[GitHub] [hudi] yihua opened a new pull request, #7088: [HUDI-] Add the feature flag back to disable HoodieFileIndex and fall back to HoodieROTablePathFilter

2022-10-28 Thread GitBox
yihua opened a new pull request, #7088: URL: https://github.com/apache/hudi/pull/7088 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any performance i

[GitHub] [hudi] hudi-bot commented on pull request #7087: [minor]: Fix creation of timeline instants in cleaner test cases

2022-10-28 Thread GitBox
hudi-bot commented on PR #7087: URL: https://github.com/apache/hudi/pull/7087#issuecomment-1295612716 ## CI report: * c3eb4a4a4581adb693ba8d2baa54e7c490fafd2d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1265

[GitHub] [hudi] yihua commented on a diff in pull request #7088: [HUDI-] Add the feature flag back to disable HoodieFileIndex and fall back to HoodieROTablePathFilter

2022-10-28 Thread GitBox
yihua commented on code in PR #7088: URL: https://github.com/apache/hudi/pull/7088#discussion_r1008567676 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala: ## @@ -91,7 +90,7 @@ object DataSourceReadOptions { val ENABLE_HOODIE

[jira] [Created] (HUDI-5104) Add feature flag to disable HoodieFileIndex and fall back to HoodieROTablePathFilter

2022-10-28 Thread Ethan Guo (Jira)
Ethan Guo created HUDI-5104: --- Summary: Add feature flag to disable HoodieFileIndex and fall back to HoodieROTablePathFilter Key: HUDI-5104 URL: https://issues.apache.org/jira/browse/HUDI-5104 Project: Apach

[jira] [Assigned] (HUDI-5104) Add feature flag to disable HoodieFileIndex and fall back to HoodieROTablePathFilter

2022-10-28 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo reassigned HUDI-5104: --- Assignee: Ethan Guo > Add feature flag to disable HoodieFileIndex and fall back to > HoodieROTablePa

  1   2   >