[jira] [Resolved] (HUDI-1820) Remove legacy code for Flink writer

2021-10-14 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-1820. -- Resolution: Invalid The legacy code already been removed. > Remove legacy code for Flink writer > -

[jira] [Assigned] (HUDI-2559) Ensure unique timestamps are generated for commit times with concurrent writers

2021-10-14 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-2559: - Assignee: sivabalan narayanan > Ensure unique timestamps are generated for commit

[jira] [Created] (HUDI-2559) Ensure unique timestamps are generated for commit times with concurrent writers

2021-10-14 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-2559: - Summary: Ensure unique timestamps are generated for commit times with concurrent writers Key: HUDI-2559 URL: https://issues.apache.org/jira/browse/HUDI-2559

[jira] [Updated] (HUDI-2558) Clustering w/ sort columns with null values fails

2021-10-14 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2558: -- Fix Version/s: 0.10.0 > Clustering w/ sort columns with null values fails >

[jira] [Assigned] (HUDI-2558) Clustering w/ sort columns with null values fails

2021-10-14 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-2558: - Assignee: sivabalan narayanan > Clustering w/ sort columns with null values fails

[jira] [Created] (HUDI-2558) Clustering w/ sort columns with null values fails

2021-10-14 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-2558: - Summary: Clustering w/ sort columns with null values fails Key: HUDI-2558 URL: https://issues.apache.org/jira/browse/HUDI-2558 Project: Apache Hudi

[jira] [Updated] (HUDI-2558) Clustering w/ sort columns with null values fails

2021-10-14 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2558: -- Labels: sev:high user-support-issues (was: ) > Clustering w/ sort columns with null val

[GitHub] [hudi] hudi-bot edited a comment on pull request #3794: [HUDI-2553] Metadata table compaction trigger max delta commits default config (re-enable)

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3794: URL: https://github.com/apache/hudi/pull/3794#issuecomment-942793927 ## CI report: * 31852dac3234f80b094392197a34ac5704f2e784 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] manojpec commented on pull request #3794: [HUDI-2553] Metadata table compaction trigger max delta commits default config (re-enable)

2021-10-14 Thread GitBox
manojpec commented on pull request #3794: URL: https://github.com/apache/hudi/pull/3794#issuecomment-943505633 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[jira] [Updated] (HUDI-2557) Shade javax.servlet for flink bundle jar

2021-10-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2557: - Labels: pull-request-available (was: ) > Shade javax.servlet for flink bundle jar > -

[GitHub] [hudi] yiduwangkai commented on pull request #3798: [HUDI-2557] Shade javax.servlet for flink bundle jar

2021-10-14 Thread GitBox
yiduwangkai commented on pull request #3798: URL: https://github.com/apache/hudi/pull/3798#issuecomment-943494883 @danny0405 sorry, i donot know how i can avoid this problem that i submitting code that someone else has already submitted -- This is an automated message from the Apache Gi

[GitHub] [hudi] hudi-bot edited a comment on pull request #3799: [HUDI-2491] hoodie.datasource.hive_sync.mode=hms mode is supported in…

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3799: URL: https://github.com/apache/hudi/pull/3799#issuecomment-943176004 ## CI report: * f44907a941b5b61e642abb5783f70fe8830fe6a6 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] hudi-bot edited a comment on pull request #3799: [HUDI-2491] hoodie.datasource.hive_sync.mode=hms mode is supported in…

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3799: URL: https://github.com/apache/hudi/pull/3799#issuecomment-943176004 ## CI report: * f44907a941b5b61e642abb5783f70fe8830fe6a6 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] leesf commented on a change in pull request #3754: [HUDI-2482] support 'drop partition' sql

2021-10-14 Thread GitBox
leesf commented on a change in pull request #3754: URL: https://github.com/apache/hudi/pull/3754#discussion_r729045601 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/HoodieSqlUtils.scala ## @@ -92,7 +93,45 @@ object HoodieSqlUtils exten

[GitHub] [hudi] leesf commented on a change in pull request #3754: [HUDI-2482] support 'drop partition' sql

2021-10-14 Thread GitBox
leesf commented on a change in pull request #3754: URL: https://github.com/apache/hudi/pull/3754#discussion_r729045075 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/HoodieSqlUtils.scala ## @@ -92,7 +93,45 @@ object HoodieSqlUtils exten

[GitHub] [hudi] hudi-bot edited a comment on pull request #3802: [HUDI-1500] Support replace commit in DeltaSync with commit metadata preserved

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3802: URL: https://github.com/apache/hudi/pull/3802#issuecomment-943342747 ## CI report: * a6459139223bd70e665424b1ae2b1b9a8f08b5c0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] codope commented on a change in pull request #3799: [HUDI-2491] hoodie.datasource.hive_sync.mode=hms mode is supported in…

2021-10-14 Thread GitBox
codope commented on a change in pull request #3799: URL: https://github.com/apache/hudi/pull/3799#discussion_r728979206 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/BootstrapExecutor.java ## @@ -159,10 +159,14 @@ public void execute() thro

[GitHub] [hudi] hudi-bot edited a comment on pull request #3802: [HUDI-1500] Support replace commit in DeltaSync with commit metadata preserved

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3802: URL: https://github.com/apache/hudi/pull/3802#issuecomment-943342747 ## CI report: * a6459139223bd70e665424b1ae2b1b9a8f08b5c0 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] hudi-bot commented on pull request #3802: [HUDI-1500] Support replace commit in DeltaSync with commit metadata preserved

2021-10-14 Thread GitBox
hudi-bot commented on pull request #3802: URL: https://github.com/apache/hudi/pull/3802#issuecomment-943342747 ## CI report: * a6459139223bd70e665424b1ae2b1b9a8f08b5c0 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[jira] [Commented] (HUDI-2549) Exceptions when using second writer into Hudi table managed by DeltaStreamer

2021-10-14 Thread Dave Hagman (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17428798#comment-17428798 ] Dave Hagman commented on HUDI-2549: --- OK so you only ran 1 iteration of each (1 commit fr

[jira] [Updated] (HUDI-1500) Support incrementally reading clustering commit via Spark Datasource/DeltaStreamer

2021-10-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1500: - Labels: pull-request-available (was: ) > Support incrementally reading clustering commit via Spa

[GitHub] [hudi] codope opened a new pull request #3802: [HUDI-1500] Support replace commit in DeltaSync with commit metadata preserved

2021-10-14 Thread GitBox
codope opened a new pull request #3802: URL: https://github.com/apache/hudi/pull/3802 ## What is the purpose of the pull request This PR fixes [HUDI-1500](https://issues.apache.org/jira/browse/HUDI-1500) for deltastreamer. For Spark datasource, it was fixed by #3139 ## Brief

[GitHub] [hudi] hudi-bot edited a comment on pull request #3203: [HUDI-2086] Refactor hive mor_incremental_view

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3203: URL: https://github.com/apache/hudi/pull/3203#issuecomment-872092745 ## CI report: * 0fa6297ce58eb877fd5c4eba59fef20ad9335d26 UNKNOWN * c4af04dab3dab31ef05ba6007000738a2dfb81ce Azure: [SUCCESS](https://dev.azure.com/apache-hudi-

[GitHub] [hudi] nsivabalan commented on pull request #3776: [HUDI-2543]: Added guides section

2021-10-14 Thread GitBox
nsivabalan commented on pull request #3776: URL: https://github.com/apache/hudi/pull/3776#issuecomment-943312764 One nit. When I mouse over "Guides", I see Tuning as first entry and Trouble shooting as 2nd entry. while on the left pane order is different. Can we fix that. https://user-i

[jira] [Closed] (HUDI-2484) Hive sync not working in HMS mode with DeltaStreamer

2021-10-14 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-2484. - Resolution: Fixed > Hive sync not working in HMS mode with DeltaStreamer > ---

[jira] [Resolved] (HUDI-2525) Test prometheus metrics with hudi (both spark ds and deltastreamer)

2021-10-14 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit resolved HUDI-2525. --- Resolution: Fixed > Test prometheus metrics with hudi (both spark ds and deltastreamer) >

[GitHub] [hudi] hudi-bot edited a comment on pull request #3203: [HUDI-2086] Refactor hive mor_incremental_view

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3203: URL: https://github.com/apache/hudi/pull/3203#issuecomment-872092745 ## CI report: * 0fa6297ce58eb877fd5c4eba59fef20ad9335d26 UNKNOWN * 1a80559bd98829552acffdaf20d3ea0384d1d936 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-

[GitHub] [hudi] hudi-bot edited a comment on pull request #3203: [HUDI-2086] Refactor hive mor_incremental_view

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3203: URL: https://github.com/apache/hudi/pull/3203#issuecomment-872092745 ## CI report: * 0fa6297ce58eb877fd5c4eba59fef20ad9335d26 UNKNOWN * 1a80559bd98829552acffdaf20d3ea0384d1d936 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-

[jira] [Resolved] (HUDI-2556) Tweak some default config options for flink

2021-10-14 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-2556. -- Resolution: Fixed Fixed via master branch: 2c370cbae084a41162fedbcc0b1e66558629dcbe > Tweak some defaul

[jira] [Updated] (HUDI-2556) Tweak some default config options for flink

2021-10-14 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-2556: - Component/s: Flink Integration > Tweak some default config options for flink > ---

[hudi] branch master updated (f897e6d -> 2c370cb)

2021-10-14 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from f897e6d [HUDI-2551] Support DefaultHoodieRecordPayload for flink (#3792) add 2c370cb [HUDI-2556] Tweak some

[GitHub] [hudi] danny0405 merged pull request #3800: [HUDI-2556] Tweak some default config options for flink

2021-10-14 Thread GitBox
danny0405 merged pull request #3800: URL: https://github.com/apache/hudi/pull/3800 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubsc

[GitHub] [hudi] danny0405 commented on a change in pull request #3203: [HUDI-2086] Refactor hive mor_incremental_view

2021-10-14 Thread GitBox
danny0405 commented on a change in pull request #3203: URL: https://github.com/apache/hudi/pull/3203#discussion_r728895613 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/RealtimeFileStatus.java ## @@ -0,0 +1,90 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] [hudi] hudi-bot edited a comment on pull request #3800: [HUDI-2556] Tweak some default config options for flink

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3800: URL: https://github.com/apache/hudi/pull/3800#issuecomment-943179326 ## CI report: * b676a7d441b059d4c22918e700a84b8fe51e240b UNKNOWN * 56a96d75ecea14a1f0367ccb339a45f4c8813dfa Azure: [SUCCESS](https://dev.azure.com/apache-hudi-

[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #3203: [HUDI-2086] Refactor hive mor_incremental_view

2021-10-14 Thread GitBox
xiarixiaoyao commented on a change in pull request #3203: URL: https://github.com/apache/hudi/pull/3203#discussion_r728890684 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieRealtimeInputFormatUtils.java ## @@ -161,6 +162,51 @@ return rtSplit

[GitHub] [hudi] hudi-bot edited a comment on pull request #3771: [HUDI-2402] Add Kerberos configuration options to Hive Sync

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3771: URL: https://github.com/apache/hudi/pull/3771#issuecomment-939200284 ## CI report: * 9e64e88d819b6b6bf5ccc5811ea5f4714138fc9e UNKNOWN * c2bc8115f70b89dfc31f27645f98cfbff8d79c0f Azure: [FAILURE](https://dev.azure.com/apache-hudi-

[GitHub] [hudi] hudi-bot edited a comment on pull request #3800: [HUDI-2556] Tweak some default config options for flink

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3800: URL: https://github.com/apache/hudi/pull/3800#issuecomment-943179326 ## CI report: * 75ee4b1f600b8231384fb65986e974c1edf26590 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot edited a comment on pull request #3203: [HUDI-2086] Refactor hive mor_incremental_view

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3203: URL: https://github.com/apache/hudi/pull/3203#issuecomment-872092745 ## CI report: * 0fa6297ce58eb877fd5c4eba59fef20ad9335d26 UNKNOWN * 1a80559bd98829552acffdaf20d3ea0384d1d936 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-

[GitHub] [hudi] hudi-bot edited a comment on pull request #3799: [HUDI-2491] hoodie.datasource.hive_sync.mode=hms mode is supported in…

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3799: URL: https://github.com/apache/hudi/pull/3799#issuecomment-943176004 ## CI report: * f44907a941b5b61e642abb5783f70fe8830fe6a6 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] gaoshihang opened a new issue #3801: [SUPPORT]Does Flink-cdc support Schema-Evolution?

2021-10-14 Thread GitBox
gaoshihang opened a new issue #3801: URL: https://github.com/apache/hudi/issues/3801 @danny0405 Hi~please ask a question, I use kafka(debezium) as source table, and hudi as target table, create a cdc application. How can I do schema evolution use flink-cdc? -- This is an autom

[GitHub] [hudi] danny0405 commented on pull request #3798: 3797 java.lang.NoSuchMethodError: io.javalin.core.CachedRequestWrappe…

2021-10-14 Thread GitBox
danny0405 commented on pull request #3798: URL: https://github.com/apache/hudi/pull/3798#issuecomment-943210670 Changes the title and commit message to "[HUDI-2557] Shade javax.servlet for flink bundle jar" -- This is an automated message from the Apache Git Service. To respond to the me

[jira] [Created] (HUDI-2557) Shade javax.servlet for flink bundle jar

2021-10-14 Thread Danny Chen (Jira)
Danny Chen created HUDI-2557: Summary: Shade javax.servlet for flink bundle jar Key: HUDI-2557 URL: https://issues.apache.org/jira/browse/HUDI-2557 Project: Apache Hudi Issue Type: Test

[GitHub] [hudi] danny0405 commented on a change in pull request #3203: [HUDI-2086] Refactor hive mor_incremental_view

2021-10-14 Thread GitBox
danny0405 commented on a change in pull request #3203: URL: https://github.com/apache/hudi/pull/3203#discussion_r728830538 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieRealtimeInputFormatUtils.java ## @@ -161,6 +162,51 @@ return rtSplits.t

[GitHub] [hudi] hudi-bot edited a comment on pull request #3800: [HUDI-2556] Tweak some default config options for flink

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3800: URL: https://github.com/apache/hudi/pull/3800#issuecomment-943179326 ## CI report: * 75ee4b1f600b8231384fb65986e974c1edf26590 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] danny0405 commented on issue #3796: [SUPPORT] Flink write to hudi,after running for a period of time,throw a NoClassDefFoundError

2021-10-14 Thread GitBox
danny0405 commented on issue #3796: URL: https://github.com/apache/hudi/issues/3796#issuecomment-943201590 Yes, the bundle jar does not package the hadoop jar. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [hudi] hudi-bot edited a comment on pull request #3800: [HUDI-2556] Tweak some default config options for flink

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3800: URL: https://github.com/apache/hudi/pull/3800#issuecomment-943179326 ## CI report: * 75ee4b1f600b8231384fb65986e974c1edf26590 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot edited a comment on pull request #3800: [HUDI-2556] Tweak some default config options for flink

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3800: URL: https://github.com/apache/hudi/pull/3800#issuecomment-943179326 ## CI report: * 75ee4b1f600b8231384fb65986e974c1edf26590 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] hudi-bot edited a comment on pull request #3771: [HUDI-2402] Add Kerberos configuration options to Hive Sync

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3771: URL: https://github.com/apache/hudi/pull/3771#issuecomment-939200284 ## CI report: * b4808aaf973608255c97e1eb1f46ff04d9bb4bee Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] hudi-bot edited a comment on pull request #3800: [HUDI-2556] Tweak some default config options for flink

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3800: URL: https://github.com/apache/hudi/pull/3800#issuecomment-943179326 ## CI report: * 75ee4b1f600b8231384fb65986e974c1edf26590 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] hudi-bot edited a comment on pull request #3771: [HUDI-2402] Add Kerberos configuration options to Hive Sync

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3771: URL: https://github.com/apache/hudi/pull/3771#issuecomment-939200284 ## CI report: * b4808aaf973608255c97e1eb1f46ff04d9bb4bee Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] hudi-bot edited a comment on pull request #3203: [HUDI-2086] Refactor hive mor_incremental_view

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3203: URL: https://github.com/apache/hudi/pull/3203#issuecomment-872092745 ## CI report: * 0fa6297ce58eb877fd5c4eba59fef20ad9335d26 UNKNOWN * 6d67b68e19a43f8668e5773d27ca9c33a8de0a37 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-

[GitHub] [hudi] test-wangxiaoyu commented on a change in pull request #3771: [HUDI-2402] Add Kerberos configuration options to Hive Sync

2021-10-14 Thread GitBox
test-wangxiaoyu commented on a change in pull request #3771: URL: https://github.com/apache/hudi/pull/3771#discussion_r728802039 ## File path: hudi-spark-datasource/hudi-spark-common/src/main/java/org/apache/hudi/DataSourceUtils.java ## @@ -307,6 +307,10 @@ public static HiveS

[GitHub] [hudi] hudi-bot commented on pull request #3800: [HUDI-2556] Tweak some default config options for flink

2021-10-14 Thread GitBox
hudi-bot commented on pull request #3800: URL: https://github.com/apache/hudi/pull/3800#issuecomment-943179326 ## CI report: * 75ee4b1f600b8231384fb65986e974c1edf26590 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[GitHub] [hudi] hudi-bot edited a comment on pull request #3799: [HUDI-2491] hoodie.datasource.hive_sync.mode=hms mode is supported in…

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3799: URL: https://github.com/apache/hudi/pull/3799#issuecomment-943176004 ## CI report: * f44907a941b5b61e642abb5783f70fe8830fe6a6 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] hudi-bot edited a comment on pull request #3771: [HUDI-2402] Add Kerberos configuration options to Hive Sync

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3771: URL: https://github.com/apache/hudi/pull/3771#issuecomment-939200284 ## CI report: * b4808aaf973608255c97e1eb1f46ff04d9bb4bee Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] test-wangxiaoyu commented on a change in pull request #3771: [HUDI-2402] Add Kerberos configuration options to Hive Sync

2021-10-14 Thread GitBox
test-wangxiaoyu commented on a change in pull request #3771: URL: https://github.com/apache/hudi/pull/3771#discussion_r728800351 ## File path: hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java ## @@ -77,6 +77,10 @@ public HiveSyncTool(HiveSyncConfig

[GitHub] [hudi] hudi-bot edited a comment on pull request #3203: [HUDI-2086] Refactor hive mor_incremental_view

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3203: URL: https://github.com/apache/hudi/pull/3203#issuecomment-872092745 ## CI report: * 0fa6297ce58eb877fd5c4eba59fef20ad9335d26 UNKNOWN * 6d67b68e19a43f8668e5773d27ca9c33a8de0a37 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-

[GitHub] [hudi] hudi-bot commented on pull request #3799: [HUDI-2491] hoodie.datasource.hive_sync.mode=hms mode is supported in…

2021-10-14 Thread GitBox
hudi-bot commented on pull request #3799: URL: https://github.com/apache/hudi/pull/3799#issuecomment-943176004 ## CI report: * f44907a941b5b61e642abb5783f70fe8830fe6a6 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[GitHub] [hudi] hudi-bot edited a comment on pull request #3798: 3797 java.lang.NoSuchMethodError: io.javalin.core.CachedRequestWrappe…

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3798: URL: https://github.com/apache/hudi/pull/3798#issuecomment-943124632 ## CI report: * e8b0555ed45956734721eda7529f04a3e739a0d2 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[jira] [Updated] (HUDI-2556) Tweak some default config options for flink

2021-10-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2556: - Labels: pull-request-available (was: ) > Tweak some default config options for flink > --

[GitHub] [hudi] danny0405 opened a new pull request #3800: [HUDI-2556] Tweak some default config options for flink

2021-10-14 Thread GitBox
danny0405 opened a new pull request #3800: URL: https://github.com/apache/hudi/pull/3800 * rename write.insert.drop.duplicates to write.precombine and set it as true for COW table * set index.global.enabled default as true * set compaction.target_io default as 500GB ## *Tips*

[GitHub] [hudi] xiarixiaoyao commented on pull request #3203: [HUDI-2086] Refactor hive mor_incremental_view

2021-10-14 Thread GitBox
xiarixiaoyao commented on pull request #3203: URL: https://github.com/apache/hudi/pull/3203#issuecomment-943163714 @danny0405 @nsivabalan thanks you very much for your patience to review those code。 already rebase the code and addressed all comments -- This is an automated message from

[GitHub] [hudi] parisni commented on issue #2498: [SUPPORT] Hudi MERGE_ON_READ load to dataframe fails for the versions [0.6.0],[0.7.0] and runs for [0.5.3]

2021-10-14 Thread GitBox
parisni commented on issue #2498: URL: https://github.com/apache/hudi/issues/2498#issuecomment-943156528 @nsivabalan Also I have to mention this is OSS spark 2.4.4 with metastore overwrite with aws glue to connect spark to glue: https://github.com/awslabs/aws-glue-libs This might be rel

[GitHub] [hudi] fuyun2024 commented on pull request #3799: [HUDI-2491] hoodie.datasource.hive_sync.mode=hms mode is supported in…

2021-10-14 Thread GitBox
fuyun2024 commented on pull request #3799: URL: https://github.com/apache/hudi/pull/3799#issuecomment-943152937 @codepe Are you free to take a look at it for me ? This is my new commit . -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [hudi] t822876884 commented on issue #3796: [SUPPORT] Flink write to hudi,after running for a period of time,throw a NoClassDefFoundError

2021-10-14 Thread GitBox
t822876884 commented on issue #3796: URL: https://github.com/apache/hudi/issues/3796#issuecomment-943149014 ![image](https://user-images.githubusercontent.com/17778114/137283507-2c6d7894-589f-4b79-8157-ddd0397a573c.png) the jar hudi-flink-bundle_2.12-0.9.0.jar from mvnrepository.com has

[GitHub] [hudi] fuyun2024 opened a new pull request #3799: [HUDI-2491] hoodie.datasource.hive_sync.mode=hms mode is supported in…

2021-10-14 Thread GitBox
fuyun2024 opened a new pull request #3799: URL: https://github.com/apache/hudi/pull/3799 … spark writer option ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.

[GitHub] [hudi] codope commented on a change in pull request #3771: [HUDI-2402] Add Kerberos configuration options to Hive Sync

2021-10-14 Thread GitBox
codope commented on a change in pull request #3771: URL: https://github.com/apache/hudi/pull/3771#discussion_r728752802 ## File path: hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java ## @@ -77,6 +77,10 @@ public HiveSyncTool(HiveSyncConfig cfg, Hiv

[GitHub] [hudi] t822876884 commented on issue #3796: [SUPPORT] Flink write to hudi,after running for a period of time,throw a NoClassDefFoundError

2021-10-14 Thread GitBox
t822876884 commented on issue #3796: URL: https://github.com/apache/hudi/issues/3796#issuecomment-943128499 > Seems you do not set up the `HADOOP_CLASSPATH` corrently, how do you submit your job ? i have a machine to submit the job by command "/home/flink/flink-1.12.2/bin/flink run

[GitHub] [hudi] fuyun2024 closed pull request #3722: HUDI-2491 hoodie.datasource.hive_sync.mode=hms mode is supported in s…

2021-10-14 Thread GitBox
fuyun2024 closed pull request #3722: URL: https://github.com/apache/hudi/pull/3722 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubsc

[GitHub] [hudi] hudi-bot edited a comment on pull request #3798: 3797 java.lang.NoSuchMethodError: io.javalin.core.CachedRequestWrappe…

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3798: URL: https://github.com/apache/hudi/pull/3798#issuecomment-943124632 ## CI report: * e8b0555ed45956734721eda7529f04a3e739a0d2 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] hudi-bot edited a comment on pull request #3203: [HUDI-2086] Refactor hive mor_incremental_view

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3203: URL: https://github.com/apache/hudi/pull/3203#issuecomment-872092745 ## CI report: * 0fa6297ce58eb877fd5c4eba59fef20ad9335d26 UNKNOWN * b5649459e5e8a5ebf5a140418c16951294a29689 Azure: [FAILURE](https://dev.azure.com/apache-hudi-

[GitHub] [hudi] hudi-bot commented on pull request #3798: 3797 java.lang.NoSuchMethodError: io.javalin.core.CachedRequestWrappe…

2021-10-14 Thread GitBox
hudi-bot commented on pull request #3798: URL: https://github.com/apache/hudi/pull/3798#issuecomment-943124632 ## CI report: * e8b0555ed45956734721eda7529f04a3e739a0d2 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[GitHub] [hudi] t822876884 removed a comment on issue #3796: [SUPPORT] Flink write to hudi,after running for a period of time,throw a NoClassDefFoundError

2021-10-14 Thread GitBox
t822876884 removed a comment on issue #3796: URL: https://github.com/apache/hudi/issues/3796#issuecomment-943123939 i have a machine to submit the job by command "/home/flink/flink-1.12.2/bin/flink run -c com.xxx.streaming.bdg.exec.YarnDataExecutorSQL -m yarn-cluster -d -yjm 2048 -ytm 512

[GitHub] [hudi] t822876884 commented on issue #3796: [SUPPORT] Flink write to hudi,after running for a period of time,throw a NoClassDefFoundError

2021-10-14 Thread GitBox
t822876884 commented on issue #3796: URL: https://github.com/apache/hudi/issues/3796#issuecomment-943123939 i have a machine to submit the job by command "/home/flink/flink-1.12.2/bin/flink run -c com.xxx.streaming.bdg.exec.YarnDataExecutorSQL -m yarn-cluster -d -yjm 2048 -ytm 5120 -p 4 -

[GitHub] [hudi] hudi-bot edited a comment on pull request #3614: [HUDI-2370] Supports data encryption

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3614: URL: https://github.com/apache/hudi/pull/3614#issuecomment-914114290 ## CI report: * 8328d5b6a6859e6b9b0b043f703ec84ed2607151 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #3203: [HUDI-2086] Refactor hive mor_incremental_view

2021-10-14 Thread GitBox
xiarixiaoyao commented on a change in pull request #3203: URL: https://github.com/apache/hudi/pull/3203#discussion_r728743555 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/realtime/HoodieParquetRealtimeInputFormat.java ## @@ -119,6 +307,11 @@ void addProjec

[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #3203: [HUDI-2086] Refactor hive mor_incremental_view

2021-10-14 Thread GitBox
xiarixiaoyao commented on a change in pull request #3203: URL: https://github.com/apache/hudi/pull/3203#discussion_r728743157 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/realtime/HoodieEmptyRecordReader.java ## @@ -0,0 +1,68 @@ +/* + * Licensed to the Apa

[GitHub] [hudi] hudi-bot edited a comment on pull request #3762: [HUDI-1294] Adding inline read and seek based read(batch get) for hfile log blocks in metadata table

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3762: URL: https://github.com/apache/hudi/pull/3762#issuecomment-938271221 ## CI report: * 5fb7a2afa196fd75ada005d26a0fb9fce5472545 UNKNOWN * ce6740e42070d594075d47b0efdb3f598fd32e3f Azure: [SUCCESS](https://dev.azure.com/apache-hudi-

[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #3203: [HUDI-2086] Refactor hive mor_incremental_view

2021-10-14 Thread GitBox
xiarixiaoyao commented on a change in pull request #3203: URL: https://github.com/apache/hudi/pull/3203#discussion_r728741908 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/realtime/HoodieParquetRealtimeInputFormat.java ## @@ -66,6 +90,170 @@ return Hoo

[jira] [Created] (HUDI-2556) Tweak some default config options for flink

2021-10-14 Thread Danny Chen (Jira)
Danny Chen created HUDI-2556: Summary: Tweak some default config options for flink Key: HUDI-2556 URL: https://issues.apache.org/jira/browse/HUDI-2556 Project: Apache Hudi Issue Type: Task

[jira] [Commented] (HUDI-2478) Handle failure mid-way during init buckets

2021-10-14 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17428670#comment-17428670 ] Prashant Wason commented on HUDI-2478: -- Since buckets are initialized only once, the

[GitHub] [hudi] yiduwangkai opened a new pull request #3798: 3797 java.lang.NoSuchMethodError: io.javalin.core.CachedRequestWrappe…

2021-10-14 Thread GitBox
yiduwangkai opened a new pull request #3798: URL: https://github.com/apache/hudi/pull/3798 …r.getContentLengthLong() ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull re

[GitHub] [hudi] yiduwangkai opened a new issue #3797: [SUPPORT]

2021-10-14 Thread GitBox
yiduwangkai opened a new issue #3797: URL: https://github.com/apache/hudi/issues/3797 * Hudi version : 0.10 * Hive version : 2.3.8 * Hadoop version : 2.10.1 * Flink version : 1.13.1 i use flink to fetch data from mysql and sink to hudi, there are some errors `http-request

[GitHub] [hudi] danny0405 commented on issue #3796: [SUPPORT] Flink write to hudi,after running for a period of time,throw a NoClassDefFoundError

2021-10-14 Thread GitBox
danny0405 commented on issue #3796: URL: https://github.com/apache/hudi/issues/3796#issuecomment-943098268 Seems you do not set up the `HADOOP_CLASSPATH` corrently, how do you submit your job ? -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [hudi] danny0405 commented on a change in pull request #3203: [HUDI-2086] Refactor hive mor_incremental_view

2021-10-14 Thread GitBox
danny0405 commented on a change in pull request #3203: URL: https://github.com/apache/hudi/pull/3203#discussion_r728695624 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieRealtimeInputFormatUtils.java ## @@ -69,17 +70,18 @@ private static final

[GitHub] [hudi] hudi-bot edited a comment on pull request #3614: [HUDI-2370] Supports data encryption

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3614: URL: https://github.com/apache/hudi/pull/3614#issuecomment-914114290 ## CI report: * 8328d5b6a6859e6b9b0b043f703ec84ed2607151 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] manojpec removed a comment on pull request #3794: [HUDI-2553] Metadata table compaction trigger max delta commits default config (re-enable)

2021-10-14 Thread GitBox
manojpec removed a comment on pull request #3794: URL: https://github.com/apache/hudi/pull/3794#issuecomment-943082717 @hudi-bot azure run -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [hudi] liujinhui1994 removed a comment on pull request #3614: [HUDI-2370] Supports data encryption

2021-10-14 Thread GitBox
liujinhui1994 removed a comment on pull request #3614: URL: https://github.com/apache/hudi/pull/3614#issuecomment-941975593 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [hudi] liujinhui1994 commented on pull request #3614: [HUDI-2370] Supports data encryption

2021-10-14 Thread GitBox
liujinhui1994 commented on pull request #3614: URL: https://github.com/apache/hudi/pull/3614#issuecomment-943089066 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [hudi] hudi-bot edited a comment on pull request #3203: [HUDI-2086] Refactor hive mor_incremental_view

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3203: URL: https://github.com/apache/hudi/pull/3203#issuecomment-872092745 ## CI report: * 0fa6297ce58eb877fd5c4eba59fef20ad9335d26 UNKNOWN * b5649459e5e8a5ebf5a140418c16951294a29689 Azure: [FAILURE](https://dev.azure.com/apache-hudi-

[GitHub] [hudi] manojpec commented on pull request #3794: [HUDI-2553] Metadata table compaction trigger max delta commits default config (re-enable)

2021-10-14 Thread GitBox
manojpec commented on pull request #3794: URL: https://github.com/apache/hudi/pull/3794#issuecomment-943082717 @hudi-bot azure run -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [hudi] hudi-bot edited a comment on pull request #3762: [HUDI-1294] Adding inline read and seek based read(batch get) for hfile log blocks in metadata table

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3762: URL: https://github.com/apache/hudi/pull/3762#issuecomment-938271221 ## CI report: * 5fb7a2afa196fd75ada005d26a0fb9fce5472545 UNKNOWN * f27df7aa4e3438bfe003fc054823a88538144c61 Azure: [FAILURE](https://dev.azure.com/apache-hudi-

[GitHub] [hudi] hudi-bot edited a comment on pull request #3762: [HUDI-1294] Adding inline read and seek based read(batch get) for hfile log blocks in metadata table

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3762: URL: https://github.com/apache/hudi/pull/3762#issuecomment-938271221 ## CI report: * 5fb7a2afa196fd75ada005d26a0fb9fce5472545 UNKNOWN * f27df7aa4e3438bfe003fc054823a88538144c61 Azure: [FAILURE](https://dev.azure.com/apache-hudi-

[GitHub] [hudi] hudi-bot edited a comment on pull request #3519: [DO NOT MERGE] 0.9.0 release patch for flink

2021-10-14 Thread GitBox
hudi-bot edited a comment on pull request #3519: URL: https://github.com/apache/hudi/pull/3519#issuecomment-903204631 ## CI report: * 1d3142cd55878ba81a358bf0b4d194779585bada Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res