[GitHub] [hudi] lw309637554 commented on a change in pull request #2773: [HUDI-1764] Add Hudi-CLI support for clustering

2021-04-07 Thread GitBox
lw309637554 commented on a change in pull request #2773: URL: https://github.com/apache/hudi/pull/2773#discussion_r608501941 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/functional/TestHoodieDeltaStreamer.java ## @@ -1013,26 +1014,22 @@ public void test

[GitHub] [hudi] ztcheck removed a comment on issue #2680: [SUPPORT]Hive sync error by using run_sync_tool.sh

2021-04-07 Thread GitBox
ztcheck removed a comment on issue #2680: URL: https://github.com/apache/hudi/issues/2680#issuecomment-814771640 > @ztcheck Are you using the `hudi-hive-sync-bundle` to execute the script `run_sync_tool .sh` ? > > Can you please provide what `HUDI_HIVE_UBER_JAR` is in your env ? Y

[GitHub] [hudi] ztcheck commented on issue #2680: [SUPPORT]Hive sync error by using run_sync_tool.sh

2021-04-07 Thread GitBox
ztcheck commented on issue #2680: URL: https://github.com/apache/hudi/issues/2680#issuecomment-814771640 > @ztcheck Are you using the `hudi-hive-sync-bundle` to execute the script `run_sync_tool .sh` ? > > Can you please provide what `HUDI_HIVE_UBER_JAR` is in your env ? Yes,`hudi

[GitHub] [hudi] simon824 opened a new pull request #2782: [MINOR] Optimized code

2021-04-07 Thread GitBox
simon824 opened a new pull request #2782: URL: https://github.com/apache/hudi/pull/2782 ## What is the purpose of the pull request Optimized some code ## Brief change log ## Verify this pull request ## Committer checklist - [ ] Has a corresponding JIRA

[GitHub] [hudi] codecov-io edited a comment on pull request #2773: [HUDI-1764] Add Hudi-CLI support for clustering

2021-04-07 Thread GitBox
codecov-io edited a comment on pull request #2773: URL: https://github.com/apache/hudi/pull/2773#issuecomment-813928206 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2773?src=pr&el=h1) Report > Merging [#2773](https://codecov.io/gh/apache/hudi/pull/2773?src=pr&el=desc) (fc2f340) in

[GitHub] [hudi] hddong commented on pull request #2325: [HUDI-699]Fix CompactionCommand and add unit test for CompactionCommand

2021-04-07 Thread GitBox
hddong commented on pull request #2325: URL: https://github.com/apache/hudi/pull/2325#issuecomment-814734415 @wangxianghu: It's OK now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [hudi] aditiwari01 commented on a change in pull request #2765: [HUDI-1716]: Resolving default values for schema from dataframe

2021-04-07 Thread GitBox
aditiwari01 commented on a change in pull request #2765: URL: https://github.com/apache/hudi/pull/2765#discussion_r608436934 ## File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionUtils.scala ## @@ -49,7 +51,48 @@ object AvroConversionUtils {

[GitHub] [hudi] aditiwari01 commented on a change in pull request #2765: [HUDI-1716]: Resolving default values for schema from dataframe

2021-04-07 Thread GitBox
aditiwari01 commented on a change in pull request #2765: URL: https://github.com/apache/hudi/pull/2765#discussion_r608434413 ## File path: hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/TestAvroConversionUtils.scala ## @@ -0,0 +1,85 @@ +/* + * Licensed to the

[GitHub] [hudi] aditiwari01 commented on a change in pull request #2765: [HUDI-1716]: Resolving default values for schema from dataframe

2021-04-07 Thread GitBox
aditiwari01 commented on a change in pull request #2765: URL: https://github.com/apache/hudi/pull/2765#discussion_r608433997 ## File path: hudi-utilities/src/test/resources/delta-streamer-config/source-jdbc.avsc ## @@ -26,34 +26,42 @@ }, { "name": "TIMESTAMP", -

[GitHub] [hudi] jintaoguan commented on a change in pull request #2773: [HUDI-1764] Add Hudi-CLI support for clustering

2021-04-07 Thread GitBox
jintaoguan commented on a change in pull request #2773: URL: https://github.com/apache/hudi/pull/2773#discussion_r608427661 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieClusteringJob.java ## @@ -156,15 +155,15 @@ private int doCluster(JavaSparkCon

[GitHub] [hudi] jintaoguan commented on a change in pull request #2773: [HUDI-1764] Add Hudi-CLI support for clustering

2021-04-07 Thread GitBox
jintaoguan commented on a change in pull request #2773: URL: https://github.com/apache/hudi/pull/2773#discussion_r608427661 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieClusteringJob.java ## @@ -156,15 +155,15 @@ private int doCluster(JavaSparkCon

[GitHub] [hudi] jintaoguan commented on a change in pull request #2773: [HUDI-1764] Add Hudi-CLI support for clustering

2021-04-07 Thread GitBox
jintaoguan commented on a change in pull request #2773: URL: https://github.com/apache/hudi/pull/2773#discussion_r608409284 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/SparkMain.java ## @@ -159,6 +161,31 @@ public static void main(String[] args) throws Exc

[hudi] branch master updated: [HUDI-1751] DeltaStreamer print many unnecessary warn log (#2754)

2021-04-07 Thread nagarwal
This is an automated email from the ASF dual-hosted git repository. nagarwal pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new dadd081 [HUDI-1751] DeltaStreamer print many un

[GitHub] [hudi] n3nash merged pull request #2754: [HUDI-1751] Remove irrelevant properties from passing to kafkaConsumer which in turn prints lot of warn logs

2021-04-07 Thread GitBox
n3nash merged pull request #2754: URL: https://github.com/apache/hudi/pull/2754 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please con

[GitHub] [hudi] n3nash commented on issue #2696: Metadata and runtime exceptions in Hudi 0.7.0 on AWS Glue

2021-04-07 Thread GitBox
n3nash commented on issue #2696: URL: https://github.com/apache/hudi/issues/2696#issuecomment-814684654 @kimberlyamandalu This is unknown territory for me as well. Let me loop in some AWS experts here. @umehrot2 Do you have any idea what the timeouts may be related to ? To summarize this t

[GitHub] [hudi] n3nash commented on issue #2771: [SUPPORT] Log files are not compacted

2021-04-07 Thread GitBox
n3nash commented on issue #2771: URL: https://github.com/apache/hudi/issues/2771#issuecomment-814682374 @stackfun I see that you have some conflicting configs ``` "hoodie.datasource.compaction.async.enable": True, "hoodie.index.type": "SIMPLE", "hood

[GitHub] [hudi] jintaoguan commented on a change in pull request #2773: [HUDI-1764] Hudi-CLI support for clustering

2021-04-07 Thread GitBox
jintaoguan commented on a change in pull request #2773: URL: https://github.com/apache/hudi/pull/2773#discussion_r608409284 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/SparkMain.java ## @@ -159,6 +161,31 @@ public static void main(String[] args) throws Exc

[GitHub] [hudi] n3nash commented on issue #2774: [SUPPORT] NoSuchMethodError on exposing metrics

2021-04-07 Thread GitBox
n3nash commented on issue #2774: URL: https://github.com/apache/hudi/issues/2774#issuecomment-814678338 @vobukh This looks like a problem due to the version differences between `io.dropwizard` in [prometheus-java client](https://github.com/prometheus/client_java/blob/master/simplec

[GitHub] [hudi] jintaoguan commented on a change in pull request #2773: [HUDI-1764] Hudi-CLI support for clustering

2021-04-07 Thread GitBox
jintaoguan commented on a change in pull request #2773: URL: https://github.com/apache/hudi/pull/2773#discussion_r608409284 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/SparkMain.java ## @@ -159,6 +161,31 @@ public static void main(String[] args) throws Exc

[GitHub] [hudi] codecov-io edited a comment on pull request #2283: [HUDI-1415] Read Hoodie Table As Spark DataSource Table

2021-04-07 Thread GitBox
codecov-io edited a comment on pull request #2283: URL: https://github.com/apache/hudi/pull/2283#issuecomment-734137301 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2283?src=pr&el=h1) Report > Merging [#2283](https://codecov.io/gh/apache/hudi/pull/2283?src=pr&el=desc) (ba70819) in

[GitHub] [hudi] codecov-io edited a comment on pull request #2283: [HUDI-1415] Read Hoodie Table As Spark DataSource Table

2021-04-07 Thread GitBox
codecov-io edited a comment on pull request #2283: URL: https://github.com/apache/hudi/pull/2283#issuecomment-734137301 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2283?src=pr&el=h1) Report > Merging [#2283](https://codecov.io/gh/apache/hudi/pull/2283?src=pr&el=desc) (ba70819) in

[jira] [Updated] (HUDI-1772) HoodieFileGroupId compareTo logical error(fileId self compare)

2021-04-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1772: - Labels: pull-request-available (was: ) > HoodieFileGroupId compareTo logical error(fileId self co

[GitHub] [hudi] codecov-io commented on pull request #2780: [HUDI-1772] HoodieFileGroupId compareTo logical error(fileId self compare)

2021-04-07 Thread GitBox
codecov-io commented on pull request #2780: URL: https://github.com/apache/hudi/pull/2780#issuecomment-814672175 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2780?src=pr&el=h1) Report > Merging [#2780](https://codecov.io/gh/apache/hudi/pull/2780?src=pr&el=desc) (8257ade) into [ma

[GitHub] [hudi] lw309637554 commented on a change in pull request #2773: [HUDI-1764] Hudi-CLI support for clustering

2021-04-07 Thread GitBox
lw309637554 commented on a change in pull request #2773: URL: https://github.com/apache/hudi/pull/2773#discussion_r608401545 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/SparkMain.java ## @@ -159,6 +161,31 @@ public static void main(String[] args) throws Ex

[GitHub] [hudi] codecov-io edited a comment on pull request #2645: [HUDI-1659] Basic Implementation Of Spark Sql Support

2021-04-07 Thread GitBox
codecov-io edited a comment on pull request #2645: URL: https://github.com/apache/hudi/pull/2645#issuecomment-792430670 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2645?src=pr&el=h1) Report > Merging [#2645](https://codecov.io/gh/apache/hudi/pull/2645?src=pr&el=desc) (3e93ac6) in

[GitHub] [hudi] lw309637554 commented on a change in pull request #2773: [HUDI-1764] Hudi-CLI support for clustering

2021-04-07 Thread GitBox
lw309637554 commented on a change in pull request #2773: URL: https://github.com/apache/hudi/pull/2773#discussion_r608399250 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieClusteringJob.java ## @@ -156,15 +155,15 @@ private int doCluster(JavaSparkCo

[GitHub] [hudi] lw309637554 commented on a change in pull request #2765: [HUDI-1716]: Resolving default values for schema from dataframe

2021-04-07 Thread GitBox
lw309637554 commented on a change in pull request #2765: URL: https://github.com/apache/hudi/pull/2765#discussion_r608395924 ## File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionUtils.scala ## @@ -49,7 +51,48 @@ object AvroConversionUtils {

[GitHub] [hudi] lw309637554 commented on a change in pull request #2765: [HUDI-1716]: Resolving default values for schema from dataframe

2021-04-07 Thread GitBox
lw309637554 commented on a change in pull request #2765: URL: https://github.com/apache/hudi/pull/2765#discussion_r608395652 ## File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionUtils.scala ## @@ -49,7 +51,48 @@ object AvroConversionUtils {

<    1   2