[GitHub] [hudi] hudi-bot commented on pull request #5933: [HUDI-4293] Implement Create/Drop/Show/Refresh Index Command for Secondary Index

2022-06-21 Thread GitBox
hudi-bot commented on PR #5933: URL: https://github.com/apache/hudi/pull/5933#issuecomment-1162722354 ## CI report: * 756c7b8850df26dc1cfd28c2b269b646e96ff683 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #5890: [HUDI-4273] Support inline schedule clustering for Flink stream

2022-06-21 Thread GitBox
hudi-bot commented on PR #5890: URL: https://github.com/apache/hudi/pull/5890#issuecomment-1162718613 ## CI report: * 08e1fa6f7820b82180d3c0352c1f92f2b4fe2c6a UNKNOWN * 2ccf74dd4a15eb7b5e9e1aa52476a41f1da01c3e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[jira] [Updated] (HUDI-4293) Implement Create/Drop/Show/Refresh Index Command for Secondary Index

2022-06-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4293: - Labels: pull-request-available (was: ) > Implement Create/Drop/Show/Refresh Index Command for Sec

[GitHub] [hudi] huberylee opened a new pull request, #5933: [HUDI-4293] Implement Create/Drop/Show/Refresh Index Command for Secondary Index

2022-06-21 Thread GitBox
huberylee opened a new pull request, #5933: URL: https://github.com/apache/hudi/pull/5933 ## What is the purpose of the pull request Implementing create/drop/show/refresh index command for secondary index based on related index syntax, and persisting index metadata into hoodie.proper

[jira] [Created] (HUDI-4297) Test TestHoodieDeltaStreamerWithMultiWriter.testUpsertsContinuousModeWithMultipleWritersWithoutConflicts is flaky

2022-06-21 Thread Danny Chen (Jira)
Danny Chen created HUDI-4297: Summary: Test TestHoodieDeltaStreamerWithMultiWriter.testUpsertsContinuousModeWithMultipleWritersWithoutConflicts is flaky Key: HUDI-4297 URL: https://issues.apache.org/jira/browse/HUDI-

[jira] [Created] (HUDI-4296) Test TestHoodieSparkSqlWriter.testSchemaEvolutionForTableType is flaky

2022-06-21 Thread Danny Chen (Jira)
Danny Chen created HUDI-4296: Summary: Test TestHoodieSparkSqlWriter.testSchemaEvolutionForTableType is flaky Key: HUDI-4296 URL: https://issues.apache.org/jira/browse/HUDI-4296 Project: Apache Hudi

[jira] [Created] (HUDI-4295) Test TestHoodieActiveTimeline.testCreateNewInstantTime is flaky

2022-06-21 Thread Danny Chen (Jira)
Danny Chen created HUDI-4295: Summary: Test TestHoodieActiveTimeline.testCreateNewInstantTime is flaky Key: HUDI-4295 URL: https://issues.apache.org/jira/browse/HUDI-4295 Project: Apache Hudi Is

[GitHub] [hudi] h7kanna commented on pull request #5815: [HUDI-4213] Infer keygen clazz for Spark SQL

2022-06-21 Thread GitBox
h7kanna commented on PR #5815: URL: https://github.com/apache/hudi/pull/5815#issuecomment-1162708950 Yes.. after further debugging.. it seems this commit is not the problem, hudi-defaults.conf is not being picked up with 0.11.1 in EMR cluster mode anymore. I have to find why. -- This

[GitHub] [hudi] danny0405 commented on issue #5931: [SUPPORT] Spark sql can't read hudi table.

2022-06-21 Thread GitBox
danny0405 commented on issue #5931: URL: https://github.com/apache/hudi/issues/5931#issuecomment-1162704975 Seems some class conflicts. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [hudi] danny0405 commented on pull request #5815: [HUDI-4213] Infer keygen clazz for Spark SQL

2022-06-21 Thread GitBox
danny0405 commented on PR #5815: URL: https://github.com/apache/hudi/pull/5815#issuecomment-1162701924 > I think this broke existing tables with CustomKeyGen or something related to partition extraction in 0.11.1. Not able to update from 0.10.1 to 0.11.1 as the partitions are resulting in '

[GitHub] [hudi] WangCHX commented on issue #5861: [SUPPORT] Hudi spark datasource error after migrate from 0.8 to 0.11

2022-06-21 Thread GitBox
WangCHX commented on issue #5861: URL: https://github.com/apache/hudi/issues/5861#issuecomment-1162687420 we are using `DeltaStreamer` to create the table, do we also need to add that configuration? -- This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [hudi] h7kanna commented on pull request #5815: [HUDI-4213] Infer keygen clazz for Spark SQL

2022-06-21 Thread GitBox
h7kanna commented on PR #5815: URL: https://github.com/apache/hudi/pull/5815#issuecomment-1162660467 I think this broke existing tables with CustomKeyGen or something related. Not able to update from 0.10.1 to 0.11.1 as the partitions are resulting in 'default' instead of date partitions

[GitHub] [hudi] hudi-bot commented on pull request #5890: [HUDI-4273] Support inline schedule clustering for Flink stream

2022-06-21 Thread GitBox
hudi-bot commented on PR #5890: URL: https://github.com/apache/hudi/pull/5890#issuecomment-1162638431 ## CI report: * 08e1fa6f7820b82180d3c0352c1f92f2b4fe2c6a UNKNOWN * bad27b281256bc02dea359383176ffc72d590c60 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] WangCHX commented on issue #5861: [SUPPORT] Hudi spark datasource error after migrate from 0.8 to 0.11

2022-06-21 Thread GitBox
WangCHX commented on issue #5861: URL: https://github.com/apache/hudi/issues/5861#issuecomment-1162636410 @boneanxs will `--conf 'spark.sql.extensions=org.apache.spark.sql.hudi.HoodieSparkSessionExtension'` fix the issue when inserting into existing table? -- This is an automated me

[GitHub] [hudi] leoyy0316 opened a new issue, #5932: [SUPPORT]

2022-06-21 Thread GitBox
leoyy0316 opened a new issue, #5932: URL: https://github.com/apache/hudi/issues/5932 can not delete data when use spark scala code 1.use sparksql to create table create table temp_db.hudi_delete_leo ( id int, name string, ts bigint ) using hudi tblproperties (

[jira] [Assigned] (HUDI-3508) Support FileSystemView command based on Call Produce Command

2022-06-21 Thread Forward Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Forward Xu reassigned HUDI-3508: Assignee: jimmyz (was: Forward Xu) > Support FileSystemView command based on Call Produce Command

[jira] [Updated] (HUDI-3508) Support FileSystemView command based on Call Produce Command

2022-06-21 Thread Forward Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Forward Xu updated HUDI-3508: - Fix Version/s: 0.12.0 > Support FileSystemView command based on Call Produce Command > ---

[GitHub] [hudi] kk17 commented on issue #5861: [SUPPORT] Hudi spark datasource error after migrate from 0.8 to 0.11

2022-06-21 Thread GitBox
kk17 commented on issue #5861: URL: https://github.com/apache/hudi/issues/5861#issuecomment-1162628043 ok now. this document https://hudi.apache.org/docs/table_management/ need to be updated. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [hudi] BruceKellan closed issue #4993: [SUPPORT] Flink Streaming read about dynamic day partition

2022-06-21 Thread GitBox
BruceKellan closed issue #4993: [SUPPORT] Flink Streaming read about dynamic day partition URL: https://github.com/apache/hudi/issues/4993 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [hudi] BruceKellan commented on issue #4993: [SUPPORT] Flink Streaming read about dynamic day partition

2022-06-21 Thread GitBox
BruceKellan commented on issue #4993: URL: https://github.com/apache/hudi/issues/4993#issuecomment-1162618490 I have some idea to optimize and I will open a new issue. Thanks your reply. -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [hudi] BruceKellan commented on issue #4993: [SUPPORT] Flink Streaming read about dynamic day partition

2022-06-21 Thread GitBox
BruceKellan commented on issue #4993: URL: https://github.com/apache/hudi/issues/4993#issuecomment-1162613059 @xushiyan When I start the flink application in day `2022-03-08`, the partition `day=2022=03-09/type=Login` does not exist. The next day (2022-03-09), the partition `day=2

[GitHub] [hudi] hudi-bot commented on pull request #5890: [HUDI-4273] Support inline schedule clustering for Flink stream

2022-06-21 Thread GitBox
hudi-bot commented on PR #5890: URL: https://github.com/apache/hudi/pull/5890#issuecomment-1162612364 ## CI report: * 08e1fa6f7820b82180d3c0352c1f92f2b4fe2c6a UNKNOWN * bad27b281256bc02dea359383176ffc72d590c60 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] boneanxs commented on issue #5861: [SUPPORT] Hudi spark datasource error after migrate from 0.8 to 0.11

2022-06-21 Thread GitBox
boneanxs commented on issue #5861: URL: https://github.com/apache/hudi/issues/5861#issuecomment-1162607127 The `primarykey` must be specified when you creating hudi table, by default it is uuid, this error means the table has no column names `uuid`, you can change the sql to ```sql

[GitHub] [hudi] WangCHX commented on issue #5861: [SUPPORT] Hudi spark datasource error after migrate from 0.8 to 0.11

2022-06-21 Thread GitBox
WangCHX commented on issue #5861: URL: https://github.com/apache/hudi/issues/5861#issuecomment-1162606883 i did try add that. same issue though. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[jira] [Assigned] (HUDI-4294) Introduce build action to actually perform index data generation

2022-06-21 Thread shibei (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shibei reassigned HUDI-4294: Assignee: shibei > Introduce build action to actually perform index data generation > -

[jira] [Created] (HUDI-4294) Introduce build action to actually perform index data generation

2022-06-21 Thread shibei (Jira)
shibei created HUDI-4294: Summary: Introduce build action to actually perform index data generation Key: HUDI-4294 URL: https://issues.apache.org/jira/browse/HUDI-4294 Project: Apache Hudi Issue Typ

[jira] [Updated] (HUDI-4293) Implement Create/Drop/Show/Refresh Index Command for Secondary Index

2022-06-21 Thread shibei (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shibei updated HUDI-4293: - Status: In Progress (was: Open) > Implement Create/Drop/Show/Refresh Index Command for Secondary Index >

[jira] [Closed] (HUDI-4271) Throw NoSuchElementException: FileID xx of partition path xx does not exist. when execute HoodieMergeHandle.getLatestBaseFile but FileID is exist in path.

2022-06-21 Thread HunterHunter (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HunterHunter closed HUDI-4271. -- Fix Version/s: 0.12.0 Resolution: Fixed > Throw NoSuchElementException: FileID xx of partition pa

[jira] [Assigned] (HUDI-4293) Implement Create/Drop/Show/Refresh Index Command for Secondary Index

2022-06-21 Thread shibei (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shibei reassigned HUDI-4293: Assignee: shibei > Implement Create/Drop/Show/Refresh Index Command for Secondary Index > -

[jira] [Created] (HUDI-4293) Implement Create/Drop/Show/Refresh Index Command for Secondary Index

2022-06-21 Thread shibei (Jira)
shibei created HUDI-4293: Summary: Implement Create/Drop/Show/Refresh Index Command for Secondary Index Key: HUDI-4293 URL: https://issues.apache.org/jira/browse/HUDI-4293 Project: Apache Hudi Issue

[GitHub] [hudi] kk17 commented on issue #5861: [SUPPORT] Hudi spark datasource error after migrate from 0.8 to 0.11

2022-06-21 Thread GitBox
kk17 commented on issue #5861: URL: https://github.com/apache/hudi/issues/5861#issuecomment-1162584170 @boneanxs no. by after i added, got another error: ``` 22/06/22 03:02:41 ERROR SparkSQLDriver: Failed in [create table if not exists hudi_table2( id int, name string, p

[GitHub] [hudi] danny0405 commented on a diff in pull request #5890: [HUDI-4273] Support inline schedule clustering for Flink stream

2022-06-21 Thread GitBox
danny0405 commented on code in PR #5890: URL: https://github.com/apache/hudi/pull/5890#discussion_r903237307 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/ITTestDataStreamWrite.java: ## @@ -234,6 +234,7 @@ private void testWriteToHoodie( Pipelines

[GitHub] [hudi] danny0405 commented on a diff in pull request #5890: [HUDI-4273] Support inline schedule clustering for Flink stream

2022-06-21 Thread GitBox
danny0405 commented on code in PR #5890: URL: https://github.com/apache/hudi/pull/5890#discussion_r903236727 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/util/ClusteringUtil.java: ## @@ -0,0 +1,77 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

[jira] [Closed] (HUDI-4286) Add release notes for 0.11.1

2022-06-21 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo closed HUDI-4286. --- Resolution: Fixed > Add release notes for 0.11.1 > > > Key: HUDI-

[GitHub] [hudi] boneanxs commented on issue #5861: [SUPPORT] Hudi spark datasource error after migrate from 0.8 to 0.11

2022-06-21 Thread GitBox
boneanxs commented on issue #5861: URL: https://github.com/apache/hudi/issues/5861#issuecomment-1162569814 @WangCHX @kk17 Have you add ```bash #spark2.4/spark3.1 --conf 'spark.sql.extensions=org.apache.spark.sql.hudi.HoodieSparkSessionExtension' #spark3.2 --conf 'spark.s

[GitHub] [hudi] danny0405 commented on pull request #5930: [MINOR] Disable flaky clean test for bootstrap file only

2022-06-21 Thread GitBox
danny0405 commented on PR #5930: URL: https://github.com/apache/hudi/pull/5930#issuecomment-1162568174 The other 2 tests failed in the CI are also flaky, we might need to figure out the reasons. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [hudi] danny0405 commented on a diff in pull request #5918: [minor] following 4270, add unit tests for the keys lost case

2022-06-21 Thread GitBox
danny0405 commented on code in PR #5918: URL: https://github.com/apache/hudi/pull/5918#discussion_r903229933 ## hudi-common/src/test/java/org/apache/hudi/common/table/timeline/TestHoodieActiveTimeline.java: ## @@ -516,7 +516,7 @@ public void testCreateNewInstantTime() throws Exc

[GitHub] [hudi] danny0405 commented on pull request #5918: [minor] following 4270, add unit tests for the keys lost case

2022-06-21 Thread GitBox
danny0405 commented on PR #5918: URL: https://github.com/apache/hudi/pull/5918#issuecomment-1162563172 The patch only modifies hudi-flink module and it test passes, the failed test case is flaky which has no relation with this patch, so i would just merge it then. -- This is an automated

[jira] [Commented] (HUDI-4279) Strength the remote fs view lagging check when latest commit refresh is enabled

2022-06-21 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17557191#comment-17557191 ] Danny Chen commented on HUDI-4279: -- Fixed via master branch: c9590790f8dc114ee827b5843c6b

[jira] [Resolved] (HUDI-4279) Strength the remote fs view lagging check when latest commit refresh is enabled

2022-06-21 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-4279. -- > Strength the remote fs view lagging check when latest commit refresh is > enabled > -

[GitHub] [hudi] danny0405 closed pull request #5928: [DO NOT MERGE] Test 4279

2022-06-21 Thread GitBox
danny0405 closed pull request #5928: [DO NOT MERGE] Test 4279 URL: https://github.com/apache/hudi/pull/5928 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-ma

[hudi] branch master updated: [HUDI-4279] Strength the remote fs view lagging check when latest commit refresh is enabled (#5917)

2022-06-21 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new c9590790f8 [HUDI-4279] Strength the remote fs v

[GitHub] [hudi] danny0405 merged pull request #5917: [HUDI-4279] Strength the remote fs view lagging check when latest com…

2022-06-21 Thread GitBox
danny0405 merged PR #5917: URL: https://github.com/apache/hudi/pull/5917 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache

[GitHub] [hudi] danny0405 commented on pull request #5917: [HUDI-4279] Strength the remote fs view lagging check when latest com…

2022-06-21 Thread GitBox
danny0405 commented on PR #5917: URL: https://github.com/apache/hudi/pull/5917#issuecomment-1162560782 The failed case is flaky, which is not caused by this patch, so i would merge it then. Link of the track issue: https://issues.apache.org/jira/browse/HUDI-4291 -- This is an automated

[GitHub] [hudi] loukey-lj commented on pull request #5877: [HUDI-4260] Change KEYGEN_CLASS_NAME without default value

2022-06-21 Thread GitBox
loukey-lj commented on PR #5877: URL: https://github.com/apache/hudi/pull/5877#issuecomment-1162546373 > Thanks, i would suggest to keep it as it is in order to be null-safe, or we should check all the code paths that use the option for null checking. For example, such validatio

[GitHub] [hudi] hudi-bot commented on pull request #5929: [HUDI-3508] Add call procedure for FileSystemViewCommand

2022-06-21 Thread GitBox
hudi-bot commented on PR #5929: URL: https://github.com/apache/hudi/pull/5929#issuecomment-1162528837 ## CI report: * f7c1a435f01f5a432ea35073d1d91bee137bfbfb UNKNOWN * f24c3cbd6cf6cdb2776aadcf58deec0c19afeec9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] vinothchandar commented on pull request #5436: [RFC-51] [HUDI-3478] Change Data Capture RFC

2022-06-21 Thread GitBox
vinothchandar commented on PR #5436: URL: https://github.com/apache/hudi/pull/5436#issuecomment-1162525774 Let me re-review the RFC tonight @YannByron . On @danny0405 's point above, as long as we can hide the implementation details (CDC log blocks or not) from the user and support t

[GitHub] [hudi] WangCHX commented on issue #5931: [SUPPORT] Spark sql can't read hudi table.

2022-06-21 Thread GitBox
WangCHX commented on issue #5931: URL: https://github.com/apache/hudi/issues/5931#issuecomment-1162486108 could be related: https://github.com/apache/hudi/issues/5861 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hudi] WangCHX commented on issue #5861: [SUPPORT] Hudi spark datasource error after migrate from 0.8 to 0.11

2022-06-21 Thread GitBox
WangCHX commented on issue #5861: URL: https://github.com/apache/hudi/issues/5861#issuecomment-1162485824 oh nice. I think I am also affected. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] hudi-bot commented on pull request #5929: [HUDI-3508] Add call procedure for FileSystemViewCommand

2022-06-21 Thread GitBox
hudi-bot commented on PR #5929: URL: https://github.com/apache/hudi/pull/5929#issuecomment-1162467172 ## CI report: * f7c1a435f01f5a432ea35073d1d91bee137bfbfb UNKNOWN * 57da73a083ebe2786fc945488d350731a4c9a0a2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #5929: [HUDI-3508] Add call procedure for FileSystemViewCommand

2022-06-21 Thread GitBox
hudi-bot commented on PR #5929: URL: https://github.com/apache/hudi/pull/5929#issuecomment-1162464970 ## CI report: * f7c1a435f01f5a432ea35073d1d91bee137bfbfb UNKNOWN * 57da73a083ebe2786fc945488d350731a4c9a0a2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] WangCHX commented on issue #5931: [SUPPORT] Spark sql can't read hudi table.

2022-06-21 Thread GitBox
WangCHX commented on issue #5931: URL: https://github.com/apache/hudi/issues/5931#issuecomment-1162443802 and what is the code control to add `location` property to the hudi table? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [hudi] WangCHX opened a new issue, #5931: [SUPPORT] Spark sql can't read hudi table.

2022-06-21 Thread GitBox
WangCHX opened a new issue, #5931: URL: https://github.com/apache/hudi/issues/5931 **_Tips before filing an issue_** - Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)? - Join the mailing list to engage in conversations and get faster support at dev-subscr..

[GitHub] [hudi] bschell commented on pull request #5366: [HUDI-1176] Upgrade hudi to log4j2

2022-06-21 Thread GitBox
bschell commented on PR #5366: URL: https://github.com/apache/hudi/pull/5366#issuecomment-1162402000 @codope Nice catch, I changed that exclusion. I don't see any other usages of log4j 1 so I think we are good there. I also think we shouldn't run into upstream dependency issues as we now su

[GitHub] [hudi] hudi-bot commented on pull request #5366: [HUDI-1176] Upgrade hudi to log4j2

2022-06-21 Thread GitBox
hudi-bot commented on PR #5366: URL: https://github.com/apache/hudi/pull/5366#issuecomment-1162302209 ## CI report: * 3c738a422cfad82c1ac06a43ce59f8749f89d1f9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9441

[GitHub] [hudi] hudi-bot commented on pull request #5877: [HUDI-4260] Change KEYGEN_CLASS_NAME without default value

2022-06-21 Thread GitBox
hudi-bot commented on PR #5877: URL: https://github.com/apache/hudi/pull/5877#issuecomment-1162161492 ## CI report: * a6e8a3375933b7b0d65553fd09b2299906d8eff0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9440

[GitHub] [hudi] hudi-bot commented on pull request #5366: [HUDI-1176] Upgrade hudi to log4j2

2022-06-21 Thread GitBox
hudi-bot commented on PR #5366: URL: https://github.com/apache/hudi/pull/5366#issuecomment-1162160445 ## CI report: * 11d182c5ab7fe56d8d3782ae9b6a5b6064fb224e Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9346

[GitHub] [hudi] hudi-bot commented on pull request #5366: [HUDI-1176] Upgrade hudi to log4j2

2022-06-21 Thread GitBox
hudi-bot commented on PR #5366: URL: https://github.com/apache/hudi/pull/5366#issuecomment-1162153952 ## CI report: * 11d182c5ab7fe56d8d3782ae9b6a5b6064fb224e Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9346

[GitHub] [hudi] hudi-bot commented on pull request #5786: [HUDI-2955] Support Hadoop 3.x Hive 3.x and Spark 3.2.x default (rebase)

2022-06-21 Thread GitBox
hudi-bot commented on PR #5786: URL: https://github.com/apache/hudi/pull/5786#issuecomment-1162148453 ## CI report: * d0eccf905d4778c46160e08fb48d4087bfd3d5d3 UNKNOWN * 0519143412014cf44a61571635eea4beb5688638 UNKNOWN * cdb27bea041845d0056ab013f8f637dbbe0cc739 UNKNOWN * b7

[GitHub] [hudi] hudi-bot commented on pull request #5877: [HUDI-4260] Change KEYGEN_CLASS_NAME without default value

2022-06-21 Thread GitBox
hudi-bot commented on PR #5877: URL: https://github.com/apache/hudi/pull/5877#issuecomment-1162033615 ## CI report: * 2e33dca6f6a65d190f60db20737303b549c8b3b3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9402

[GitHub] [hudi] tfontana1 closed issue #5821: [SUPPORT] Failed to upsert for commit time - PySpark Unable to Write to Hudi with ABFS

2022-06-21 Thread GitBox
tfontana1 closed issue #5821: [SUPPORT] Failed to upsert for commit time - PySpark Unable to Write to Hudi with ABFS URL: https://github.com/apache/hudi/issues/5821 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] tfontana1 commented on issue #5821: [SUPPORT] Failed to upsert for commit time - PySpark Unable to Write to Hudi with ABFS

2022-06-21 Thread GitBox
tfontana1 commented on issue #5821: URL: https://github.com/apache/hudi/issues/5821#issuecomment-1162023459 Thank you, I will close this. I ended up using Hudi 0.11.0 but the issue was with my operator and drivers having mismatched jar versions. -- This is an automated message from the A

[GitHub] [hudi] hudi-bot commented on pull request #5877: [HUDI-4260] Change KEYGEN_CLASS_NAME without default value

2022-06-21 Thread GitBox
hudi-bot commented on PR #5877: URL: https://github.com/apache/hudi/pull/5877#issuecomment-1161987302 ## CI report: * 2e33dca6f6a65d190f60db20737303b549c8b3b3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9402

[GitHub] [hudi] hudi-bot commented on pull request #5786: [HUDI-2955] Support Hadoop 3.x Hive 3.x and Spark 3.2.x default (rebase)

2022-06-21 Thread GitBox
hudi-bot commented on PR #5786: URL: https://github.com/apache/hudi/pull/5786#issuecomment-1161973036 ## CI report: * d0eccf905d4778c46160e08fb48d4087bfd3d5d3 UNKNOWN * 0519143412014cf44a61571635eea4beb5688638 UNKNOWN * cdb27bea041845d0056ab013f8f637dbbe0cc739 UNKNOWN * b7

[GitHub] [hudi] hudi-bot commented on pull request #5930: [MINOR] Disable flaky clean test for bootstrap file only

2022-06-21 Thread GitBox
hudi-bot commented on PR #5930: URL: https://github.com/apache/hudi/pull/5930#issuecomment-1161968077 ## CI report: * 4d2995f685bc6e7b0dbf3375ad43bf332063604c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9438

[GitHub] [hudi] hudi-bot commented on pull request #5929: [HUDI-3508] Add call procedure for FileSystemViewCommand

2022-06-21 Thread GitBox
hudi-bot commented on PR #5929: URL: https://github.com/apache/hudi/pull/5929#issuecomment-1161920984 ## CI report: * f7c1a435f01f5a432ea35073d1d91bee137bfbfb UNKNOWN * 57da73a083ebe2786fc945488d350731a4c9a0a2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[jira] [Commented] (HUDI-4291) Test TestCleanPlanExecutor.testKeepLatestFileVersions is flaky

2022-06-21 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17556971#comment-17556971 ] Sagar Sumit commented on HUDI-4291: --- For now, I am diabling the test [https://github.com

[GitHub] [hudi] hudi-bot commented on pull request #5786: [HUDI-2955] Support Hadoop 3.x Hive 3.x and Spark 3.2.x default (rebase)

2022-06-21 Thread GitBox
hudi-bot commented on PR #5786: URL: https://github.com/apache/hudi/pull/5786#issuecomment-1161910232 ## CI report: * d0eccf905d4778c46160e08fb48d4087bfd3d5d3 UNKNOWN * 0519143412014cf44a61571635eea4beb5688638 UNKNOWN * cdb27bea041845d0056ab013f8f637dbbe0cc739 UNKNOWN * b7

[GitHub] [hudi] hudi-bot commented on pull request #5930: [MINOR] Disable flaky clean test for bootstrap file only

2022-06-21 Thread GitBox
hudi-bot commented on PR #5930: URL: https://github.com/apache/hudi/pull/5930#issuecomment-1161905310 ## CI report: * 4d2995f685bc6e7b0dbf3375ad43bf332063604c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9438

[GitHub] [hudi] hudi-bot commented on pull request #5929: [HUDI-3508] Add call procedure for FileSystemViewCommand

2022-06-21 Thread GitBox
hudi-bot commented on PR #5929: URL: https://github.com/apache/hudi/pull/5929#issuecomment-1161905264 ## CI report: * f7c1a435f01f5a432ea35073d1d91bee137bfbfb UNKNOWN * 57da73a083ebe2786fc945488d350731a4c9a0a2 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #5930: [MINOR] Disable flaky clean test for bootstrap file only

2022-06-21 Thread GitBox
hudi-bot commented on PR #5930: URL: https://github.com/apache/hudi/pull/5930#issuecomment-1161900032 ## CI report: * 4d2995f685bc6e7b0dbf3375ad43bf332063604c UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #5929: [HUDI-3508] Add call procedure for FileSystemViewCommand

2022-06-21 Thread GitBox
hudi-bot commented on PR #5929: URL: https://github.com/apache/hudi/pull/5929#issuecomment-1161899972 ## CI report: * f7c1a435f01f5a432ea35073d1d91bee137bfbfb UNKNOWN * 57da73a083ebe2786fc945488d350731a4c9a0a2 UNKNOWN Bot commands @hudi-bot supports the following

[GitHub] [hudi] hudi-bot commented on pull request #5929: [HUDI-3508] Add call procedure for FileSystemViewCommand

2022-06-21 Thread GitBox
hudi-bot commented on PR #5929: URL: https://github.com/apache/hudi/pull/5929#issuecomment-1161891784 ## CI report: * f7c1a435f01f5a432ea35073d1d91bee137bfbfb UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] codope opened a new pull request, #5930: [MINOR] Disable flaky clean test for bootstrap file only

2022-06-21 Thread GitBox
codope opened a new pull request, #5930: URL: https://github.com/apache/hudi/pull/5930 ## What is the purpose of the pull request Disables a flaky test to unblock CI. Root cause to be tracked down in HUDI-4291. ## Brief change log *(for example:)* - *Modify Annotati

[GitHub] [hudi] rahil-c commented on a diff in pull request #5786: [HUDI-2955] Support Hadoop 3.x Hive 3.x and Spark 3.2.x default (rebase)

2022-06-21 Thread GitBox
rahil-c commented on code in PR #5786: URL: https://github.com/apache/hudi/pull/5786#discussion_r902164267 ## pom.xml: ## @@ -1669,15 +1762,17 @@ ${fasterxml.spark3.version} true -true +false hudi-spark-datasou

[GitHub] [hudi] rahil-c commented on a diff in pull request #5786: [HUDI-2955] Support Hadoop 3.x Hive 3.x and Spark 3.2.x default (rebase)

2022-06-21 Thread GitBox
rahil-c commented on code in PR #5786: URL: https://github.com/apache/hudi/pull/5786#discussion_r892857789 ## hudi-client/hudi-client-common/src/test/java/org/apache/hudi/io/storage/TestHoodieReaderWriterBase.java: ## @@ -217,15 +221,24 @@ protected void verifyComplexRecords(It

[GitHub] [hudi] rahil-c commented on a diff in pull request #5786: [HUDI-2955] Support Hadoop 3.x Hive 3.x and Spark 3.2.x default (rebase)

2022-06-21 Thread GitBox
rahil-c commented on code in PR #5786: URL: https://github.com/apache/hudi/pull/5786#discussion_r892847919 ## docker/hoodie/hadoop/hive_base/startup.sh: ## @@ -22,5 +22,4 @@ hadoop fs -chmod g+w /tmp hadoop fs -chmod g+w /user/hive/warehouse cd $HIVE_HOME/bin -export AUX

[jira] [Updated] (HUDI-3508) Support FileSystemView command based on Call Produce Command

2022-06-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3508: - Labels: pull-request-available (was: ) > Support FileSystemView command based on Call Produce Com

[GitHub] [hudi] microbearz opened a new pull request, #5929: [HUDI-3508] Add call procedure for FileSystemViewCommand

2022-06-21 Thread GitBox
microbearz opened a new pull request, #5929: URL: https://github.com/apache/hudi/pull/5929 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purp

[GitHub] [hudi] hudi-bot commented on pull request #5890: [HUDI-4273] Support inline schedule clustering for Flink stream

2022-06-21 Thread GitBox
hudi-bot commented on PR #5890: URL: https://github.com/apache/hudi/pull/5890#issuecomment-1161820156 ## CI report: * 08e1fa6f7820b82180d3c0352c1f92f2b4fe2c6a UNKNOWN * bad27b281256bc02dea359383176ffc72d590c60 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #5928: [DO NOT MERGE] Test 4279

2022-06-21 Thread GitBox
hudi-bot commented on PR #5928: URL: https://github.com/apache/hudi/pull/5928#issuecomment-1161797677 ## CI report: * f8b58b8aeda4df524bab4f131758efcda32de9d0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9435

[GitHub] [hudi] garyli1019 commented on a diff in pull request #5677: [HUDI-4152] Flink offline compaction support compacting multi compaction plan at once

2022-06-21 Thread GitBox
garyli1019 commented on code in PR #5677: URL: https://github.com/apache/hudi/pull/5677#discussion_r902614478 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/compact/HoodieFlinkCompactor.java: ## @@ -218,74 +228,112 @@ private void compact() throws Excepti

[jira] [Commented] (HUDI-4291) Test TestCleanPlanExecutor.testKeepLatestFileVersions is flaky

2022-06-21 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17556902#comment-17556902 ] Sagar Sumit commented on HUDI-4291: --- >From my local runs, I noticed that it only happens

[GitHub] [hudi] hudi-bot commented on pull request #5890: [HUDI-4273] Support inline schedule clustering for Flink stream

2022-06-21 Thread GitBox
hudi-bot commented on PR #5890: URL: https://github.com/apache/hudi/pull/5890#issuecomment-1161736209 ## CI report: * 08e1fa6f7820b82180d3c0352c1f92f2b4fe2c6a UNKNOWN * e2195d27bcd63c9b9a419aa09560d61ea9d9d75d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #5928: [DO NOT MERGE] Test 4279

2022-06-21 Thread GitBox
hudi-bot commented on PR #5928: URL: https://github.com/apache/hudi/pull/5928#issuecomment-1161730541 ## CI report: * f8b58b8aeda4df524bab4f131758efcda32de9d0 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9435

[GitHub] [hudi] hudi-bot commented on pull request #5928: [DO NOT MERGE] Test 4279

2022-06-21 Thread GitBox
hudi-bot commented on PR #5928: URL: https://github.com/apache/hudi/pull/5928#issuecomment-1161725274 ## CI report: * f8b58b8aeda4df524bab4f131758efcda32de9d0 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #5890: [HUDI-4273] Support inline schedule clustering for Flink stream

2022-06-21 Thread GitBox
hudi-bot commented on PR #5890: URL: https://github.com/apache/hudi/pull/5890#issuecomment-1161725105 ## CI report: * 08e1fa6f7820b82180d3c0352c1f92f2b4fe2c6a UNKNOWN * 22b639b81ea33060ed656270ed5365d941d25dbe Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #5927: [HUDI-4292] Update the RFC-46 doc because the Record Merge API is changed from CombineEngine to HoodieMerge

2022-06-21 Thread GitBox
hudi-bot commented on PR #5927: URL: https://github.com/apache/hudi/pull/5927#issuecomment-1161719403 ## CI report: * a53bc7ba85daf815570eb0cfbb4eaebff83b3671 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9433

[GitHub] [hudi] santoshsb commented on issue #5452: Schema Evolution: Missing column for previous records when new entry does not have the same while upsert.

2022-06-21 Thread GitBox
santoshsb commented on issue #5452: URL: https://github.com/apache/hudi/issues/5452#issuecomment-1161712629 @xiarixiaoyao as mentioned earlier we didn't solve the nested column case, we are currently trying to finalize a fixed schema and while reading in the data with spark use this schema

[GitHub] [hudi] danny0405 opened a new pull request, #5928: [DO NOT MERGE] Test 4279

2022-06-21 Thread GitBox
danny0405 opened a new pull request, #5928: URL: https://github.com/apache/hudi/pull/5928 …mit refresh is enabled Signed-off-by: LinMingQiang <1356469...@qq.com> ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/con

[GitHub] [hudi] yuzhaojing commented on a diff in pull request #5890: [HUDI-4273] Support inline schedule clustering for Flink stream

2022-06-21 Thread GitBox
yuzhaojing commented on code in PR #5890: URL: https://github.com/apache/hudi/pull/5890#discussion_r902557641 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/clustering/ClusteringOperator.java: ## @@ -100,15 +105,25 @@ public class ClusteringOperator exten

[GitHub] [hudi] yuzhaojing commented on a diff in pull request #5890: [HUDI-4273] Support inline schedule clustering for Flink stream

2022-06-21 Thread GitBox
yuzhaojing commented on code in PR #5890: URL: https://github.com/apache/hudi/pull/5890#discussion_r902557363 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/clustering/HoodieFlinkClusteringJob.java: ## @@ -45,148 +49,290 @@ import org.slf4j.Logger; impo

[GitHub] [hudi] hudi-bot commented on pull request #5917: [HUDI-4279] Strength the remote fs view lagging check when latest com…

2022-06-21 Thread GitBox
hudi-bot commented on PR #5917: URL: https://github.com/apache/hudi/pull/5917#issuecomment-1161674341 ## CI report: * f8b58b8aeda4df524bab4f131758efcda32de9d0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9413

[GitHub] [hudi] danny0405 closed pull request #5917: [HUDI-4279] Strength the remote fs view lagging check when latest com…

2022-06-21 Thread GitBox
danny0405 closed pull request #5917: [HUDI-4279] Strength the remote fs view lagging check when latest com… URL: https://github.com/apache/hudi/pull/5917 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hudi] hudi-bot commented on pull request #5917: [HUDI-4279] Strength the remote fs view lagging check when latest com…

2022-06-21 Thread GitBox
hudi-bot commented on PR #5917: URL: https://github.com/apache/hudi/pull/5917#issuecomment-1161669549 ## CI report: * f8b58b8aeda4df524bab4f131758efcda32de9d0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9413

[GitHub] [hudi] hudi-bot commented on pull request #5890: [HUDI-4273] Support inline schedule clustering for Flink stream

2022-06-21 Thread GitBox
hudi-bot commented on PR #5890: URL: https://github.com/apache/hudi/pull/5890#issuecomment-1161669429 ## CI report: * 08e1fa6f7820b82180d3c0352c1f92f2b4fe2c6a UNKNOWN * 22b639b81ea33060ed656270ed5365d941d25dbe Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] danny0405 commented on pull request #5917: [HUDI-4279] Strength the remote fs view lagging check when latest com…

2022-06-21 Thread GitBox
danny0405 commented on PR #5917: URL: https://github.com/apache/hudi/pull/5917#issuecomment-1161668959 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [hudi] hudi-bot commented on pull request #5890: [HUDI-4273] Support inline schedule clustering for Flink stream

2022-06-21 Thread GitBox
hudi-bot commented on PR #5890: URL: https://github.com/apache/hudi/pull/5890#issuecomment-1161659566 ## CI report: * 08e1fa6f7820b82180d3c0352c1f92f2b4fe2c6a UNKNOWN * 22b639b81ea33060ed656270ed5365d941d25dbe Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #5722: [HUDI-4170] Make user can use hoodie.datasource.read.paths to read necessary files

2022-06-21 Thread GitBox
hudi-bot commented on PR #5722: URL: https://github.com/apache/hudi/pull/5722#issuecomment-1161659176 ## CI report: * 25abb3b5243131584a4bb143faa2fc4b3a5ec0f3 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9021

[GitHub] [hudi] hudi-bot commented on pull request #5890: [HUDI-4273] Support inline schedule clustering for Flink stream

2022-06-21 Thread GitBox
hudi-bot commented on PR #5890: URL: https://github.com/apache/hudi/pull/5890#issuecomment-1161654535 ## CI report: * 08e1fa6f7820b82180d3c0352c1f92f2b4fe2c6a UNKNOWN * 22b639b81ea33060ed656270ed5365d941d25dbe Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

  1   2   3   >