[GitHub] [hudi] yanghua commented on pull request #2747: [HUDI-1743] Added support for SqlFileBasedTransformer

2021-03-31 Thread GitBox
yanghua commented on pull request #2747: URL: https://github.com/apache/hudi/pull/2747#issuecomment-811679455 > @yanghua - I've fixed the build, can you please merge this code? Thanks, IMO, can you add a unit test for the feature. -- This is an automated message from the Apache Git

[GitHub] [hudi] xurunbai commented on pull request #2755: Add HoodieFlinkClient InsertOverwrite

2021-03-31 Thread GitBox
xurunbai commented on pull request #2755: URL: https://github.com/apache/hudi/pull/2755#issuecomment-811678698 Add HoodieFlinkClient InsertOverwrite -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hudi] xurunbai opened a new pull request #2755: Add HoodieFlinkClient InsertOverwrite

2021-03-31 Thread GitBox
xurunbai opened a new pull request #2755: URL: https://github.com/apache/hudi/pull/2755 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the pu

[GitHub] [hudi] liujinhui1994 commented on a change in pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2021-03-31 Thread GitBox
liujinhui1994 commented on a change in pull request #2438: URL: https://github.com/apache/hudi/pull/2438#discussion_r605394316 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/KafkaOffsetGen.java ## @@ -247,6 +266,32 @@ private Long delayOff

[GitHub] [hudi] liujinhui1994 commented on a change in pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2021-03-31 Thread GitBox
liujinhui1994 commented on a change in pull request #2438: URL: https://github.com/apache/hudi/pull/2438#discussion_r605393953 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/HoodieDeltaStreamer.java ## @@ -553,6 +555,11 @@ public DeltaSyncSe

[GitHub] [hudi] liujinhui1994 commented on a change in pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2021-03-31 Thread GitBox
liujinhui1994 commented on a change in pull request #2438: URL: https://github.com/apache/hudi/pull/2438#discussion_r605393270 ## File path: hudi-integ-test/src/main/java/org/apache/hudi/integ/testsuite/HoodieDeltaStreamerWrapper.java ## @@ -65,7 +65,7 @@ public void scheduleC

[GitHub] [hudi] vingov commented on pull request #2747: [HUDI-1743] Added support for SqlFileBasedTransformer

2021-03-31 Thread GitBox
vingov commented on pull request #2747: URL: https://github.com/apache/hudi/pull/2747#issuecomment-811654731 @yanghua - I've fixed the build, can you please merge this code? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] nsivabalan closed pull request #1929: [HUDI-1160] Support update partial fields for CoW table

2021-03-31 Thread GitBox
nsivabalan closed pull request #1929: URL: https://github.com/apache/hudi/pull/1929 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [hudi] nsivabalan commented on pull request #1929: [HUDI-1160] Support update partial fields for CoW table

2021-03-31 Thread GitBox
nsivabalan commented on pull request #1929: URL: https://github.com/apache/hudi/pull/1929#issuecomment-811627648 Closing in favor of https://github.com/apache/hudi/pull/2666 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] codecov-io edited a comment on pull request #2754: [HUDI-1751] DeltaStreamer print many unnecessary warn log

2021-03-31 Thread GitBox
codecov-io edited a comment on pull request #2754: URL: https://github.com/apache/hudi/pull/2754#issuecomment-811625350 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2754?src=pr&el=h1) Report > Merging [#2754](https://codecov.io/gh/apache/hudi/pull/2754?src=pr&el=desc) (dcc1fc4) in

[GitHub] [hudi] codecov-io commented on pull request #2754: [HUDI-1751] DeltaStreamer print many unnecessary warn log

2021-03-31 Thread GitBox
codecov-io commented on pull request #2754: URL: https://github.com/apache/hudi/pull/2754#issuecomment-811625350 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2754?src=pr&el=h1) Report > Merging [#2754](https://codecov.io/gh/apache/hudi/pull/2754?src=pr&el=desc) (dcc1fc4) into [ma

[GitHub] [hudi] codecov-io edited a comment on pull request #2651: [HUDI-1591] [RFC-26] Improve Hoodie Table Query Performance And Ease Of Use Fo…

2021-03-31 Thread GitBox
codecov-io edited a comment on pull request #2651: URL: https://github.com/apache/hudi/pull/2651#issuecomment-794945140 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2651?src=pr&el=h1) Report > Merging [#2651](https://codecov.io/gh/apache/hudi/pull/2651?src=pr&el=desc) (bb49f77) in

[GitHub] [hudi] ssdong commented on issue #2707: [SUPPORT] insert_ovewrite_table failed on archiving

2021-03-31 Thread GitBox
ssdong commented on issue #2707: URL: https://github.com/apache/hudi/issues/2707#issuecomment-811619487 @jsbali To give out extra insights and details, as @zherenyu831 has posted in the beginning: ``` [20210323080718__replacecommit__COMPLETED]: size : 0 [20210323081449__replacecomm

[GitHub] [hudi] tooptoop4 commented on issue #2609: [SUPPORT] Presto hudi query slow when compared to parquet

2021-03-31 Thread GitBox
tooptoop4 commented on issue #2609: URL: https://github.com/apache/hudi/issues/2609#issuecomment-811614768 @rshanmugam1 see https://prestodb.io/blog/2020/08/04/prestodb-and-hudi and https://github.com/prestodb/presto/commit/9fd2459d98efd0809023b175ba53775466b74cc6 and https://github.com

[jira] [Updated] (HUDI-1751) DeltaStream print many unnecessary warn log

2021-03-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1751: - Labels: pull-request-available (was: ) > DeltaStream print many unnecessary warn log > --

[GitHub] [hudi] codecov-io edited a comment on pull request #2651: [HUDI-1591] [RFC-26] Improve Hoodie Table Query Performance And Ease Of Use Fo…

2021-03-31 Thread GitBox
codecov-io edited a comment on pull request #2651: URL: https://github.com/apache/hudi/pull/2651#issuecomment-794945140 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2651?src=pr&el=h1) Report > Merging [#2651](https://codecov.io/gh/apache/hudi/pull/2651?src=pr&el=desc) (201e4ff) in

[GitHub] [hudi] codecov-io edited a comment on pull request #2651: [HUDI-1591] [RFC-26] Improve Hoodie Table Query Performance And Ease Of Use Fo…

2021-03-31 Thread GitBox
codecov-io edited a comment on pull request #2651: URL: https://github.com/apache/hudi/pull/2651#issuecomment-794945140 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2651?src=pr&el=h1) Report > Merging [#2651](https://codecov.io/gh/apache/hudi/pull/2651?src=pr&el=desc) (201e4ff) in

[GitHub] [hudi] li36909 opened a new pull request #2754: [HUDI-1751] DeltaStreamer print many unnecessary warn log

2021-03-31 Thread GitBox
li36909 opened a new pull request #2754: URL: https://github.com/apache/hudi/pull/2754 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the pul

[GitHub] [hudi] codecov-io commented on pull request #2752: [HUDI-1749] Clean/Compaction/Rollback command maybe never exit when operation fail

2021-03-31 Thread GitBox
codecov-io commented on pull request #2752: URL: https://github.com/apache/hudi/pull/2752#issuecomment-811610689 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2752?src=pr&el=h1) Report > Merging [#2752](https://codecov.io/gh/apache/hudi/pull/2752?src=pr&el=desc) (2152760) into [ma

[jira] [Created] (HUDI-1751) DeltaStream print many unnecessary warn log

2021-03-31 Thread lrz (Jira)
lrz created HUDI-1751: - Summary: DeltaStream print many unnecessary warn log Key: HUDI-1751 URL: https://issues.apache.org/jira/browse/HUDI-1751 Project: Apache Hudi Issue Type: Improvement R

[GitHub] [hudi] codecov-io commented on pull request #2753: [HUDI-1750] Fail to load user's class if user move hudi-spark-bundle jar into spark classpath

2021-03-31 Thread GitBox
codecov-io commented on pull request #2753: URL: https://github.com/apache/hudi/pull/2753#issuecomment-811608059 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2753?src=pr&el=h1) Report > Merging [#2753](https://codecov.io/gh/apache/hudi/pull/2753?src=pr&el=desc) (43eb4f1) into [ma

[jira] [Updated] (HUDI-1750) Fail to load user's class if user move hudi-spark-bundle_2.11-0.7.0.jar into spark classpath

2021-03-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1750: - Labels: pull-request-available (was: ) > Fail to load user's class if user move hudi-spark-bundle

[GitHub] [hudi] li36909 opened a new pull request #2753: [HUDI-1750] Fail to load user's class if user move hudi-spark-bundle jar into spark classpath

2021-03-31 Thread GitBox
li36909 opened a new pull request #2753: URL: https://github.com/apache/hudi/pull/2753 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the pul

[GitHub] [hudi] garyli1019 commented on a change in pull request #2721: [HUDI-1720] when query incr view of mor table which has many delete records use sparksql/hive-beeline, StackOverflowError

2021-03-31 Thread GitBox
garyli1019 commented on a change in pull request #2721: URL: https://github.com/apache/hudi/pull/2721#discussion_r605342627 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/realtime/RealtimeCompactedRecordReader.java ## @@ -95,15 +103,24 @@ public boolean next

[jira] [Created] (HUDI-1750) Fail to load user's class if user move hudi-spark-bundle_2.11-0.7.0.jar into spark classpath

2021-03-31 Thread lrz (Jira)
lrz created HUDI-1750: - Summary: Fail to load user's class if user move hudi-spark-bundle_2.11-0.7.0.jar into spark classpath Key: HUDI-1750 URL: https://issues.apache.org/jira/browse/HUDI-1750 Project: Apache Hu

[GitHub] [hudi] li36909 commented on pull request #2752: [HUDI-1749] Clean/Compaction/Rollback command maybe never exit when operation fail

2021-03-31 Thread GitBox
li36909 commented on pull request #2752: URL: https://github.com/apache/hudi/pull/2752#issuecomment-811596187 cc @nsivabalan could you help to take a look, thank you -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[jira] [Updated] (HUDI-1749) Clean/Compaction/Rollback command maybe never exit when operation fail

2021-03-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1749: - Labels: pull-request-available (was: ) > Clean/Compaction/Rollback command maybe never exit when

[GitHub] [hudi] li36909 opened a new pull request #2752: [HUDI-1749] Clean/Compaction/Rollback command maybe never exit when operation fail

2021-03-31 Thread GitBox
li36909 opened a new pull request #2752: URL: https://github.com/apache/hudi/pull/2752 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the pul

[GitHub] [hudi] pengzhiwei2018 commented on a change in pull request #2651: [HUDI-1591] [RFC-26] Improve Hoodie Table Query Performance And Ease Of Use Fo…

2021-03-31 Thread GitBox
pengzhiwei2018 commented on a change in pull request #2651: URL: https://github.com/apache/hudi/pull/2651#discussion_r605337987 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/HoodieWriterUtils.scala ## @@ -81,4 +82,53 @@ object HoodieWriterUtils

[jira] [Created] (HUDI-1749) Clean/Compaction/Rollback command maybe never exit when operation fail

2021-03-31 Thread lrz (Jira)
lrz created HUDI-1749: - Summary: Clean/Compaction/Rollback command maybe never exit when operation fail Key: HUDI-1749 URL: https://issues.apache.org/jira/browse/HUDI-1749 Project: Apache Hudi Issue Typ

[jira] [Updated] (HUDI-1749) Clean/Compaction/Rollback command maybe never exit when operation fail

2021-03-31 Thread lrz (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lrz updated HUDI-1749: -- Fix Version/s: 0.9.0 > Clean/Compaction/Rollback command maybe never exit when operation fail >

[GitHub] [hudi] codecov-io commented on pull request #2751: [HUDI-1748] Read operation will possiblity fail on mor table rt view when a write operations is concurrency running

2021-03-31 Thread GitBox
codecov-io commented on pull request #2751: URL: https://github.com/apache/hudi/pull/2751#issuecomment-811590982 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2751?src=pr&el=h1) Report > Merging [#2751](https://codecov.io/gh/apache/hudi/pull/2751?src=pr&el=desc) (2de7140) into [ma

[GitHub] [hudi] garyli1019 commented on pull request #2722: [HUDI-1722]hive beeline/spark-sql query specified field on mor table occur NPE

2021-03-31 Thread GitBox
garyli1019 commented on pull request #2722: URL: https://github.com/apache/hudi/pull/2722#issuecomment-811589593 @xiarixiaoyao Thanks for your contribution. Looks like you are able to reproduce this problem in the unit test. Is that possible to add the unit test to this pr as well? -- T

[GitHub] [hudi] codecov-io edited a comment on pull request #2325: [HUDI-699]Fix CompactionCommand and add unit test for CompactionCommand

2021-03-31 Thread GitBox
codecov-io edited a comment on pull request #2325: URL: https://github.com/apache/hudi/pull/2325#issuecomment-742860619 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2325?src=pr&el=h1) Report > Merging [#2325](https://codecov.io/gh/apache/hudi/pull/2325?src=pr&el=desc) (24790dc) in

[GitHub] [hudi] codecov-io edited a comment on pull request #2651: [HUDI-1591] [RFC-26] Improve Hoodie Table Query Performance And Ease Of Use Fo…

2021-03-31 Thread GitBox
codecov-io edited a comment on pull request #2651: URL: https://github.com/apache/hudi/pull/2651#issuecomment-794945140 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2651?src=pr&el=h1) Report > Merging [#2651](https://codecov.io/gh/apache/hudi/pull/2651?src=pr&el=desc) (fd68e0f) in

[GitHub] [hudi] pengzhiwei2018 commented on a change in pull request #2651: [HUDI-1591] [RFC-26] Improve Hoodie Table Query Performance And Ease Of Use Fo…

2021-03-31 Thread GitBox
pengzhiwei2018 commented on a change in pull request #2651: URL: https://github.com/apache/hudi/pull/2651#discussion_r605330589 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/HoodieWriterUtils.scala ## @@ -81,4 +82,53 @@ object HoodieWriterUtils

[GitHub] [hudi] rshanmugam1 commented on issue #2609: [SUPPORT] Presto hudi query slow when compared to parquet

2021-03-31 Thread GitBox
rshanmugam1 commented on issue #2609: URL: https://github.com/apache/hudi/issues/2609#issuecomment-811582823 Thanks very much Sudha and team. i will look in that direction and make sure that the cause. if that is the case, any pointers how to fix it. or any reference how it got fixe

[GitHub] [hudi] li36909 commented on pull request #2751: [HUDI-1748] Read operation will possiblity fail on mor table rt view when a write operations is concurrency running

2021-03-31 Thread GitBox
li36909 commented on pull request #2751: URL: https://github.com/apache/hudi/pull/2751#issuecomment-811582115 cc @nsivabalan could you help to take a look, thank you -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[jira] [Updated] (HUDI-1748) Read operation will possibility fail on mor table rt view when a write operations is concurrency running

2021-03-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1748: - Labels: pull-request-available (was: ) > Read operation will possibility fail on mor table rt vie

[GitHub] [hudi] li36909 opened a new pull request #2751: [HUDI-1748] Read operation will possiblity fail on mor table rt view when a write operations is concurrency running

2021-03-31 Thread GitBox
li36909 opened a new pull request #2751: URL: https://github.com/apache/hudi/pull/2751 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the pul

[jira] [Created] (HUDI-1748) Read operation will possibility fail on mor table rt view when a write operations is concurrency running

2021-03-31 Thread lrz (Jira)
lrz created HUDI-1748: - Summary: Read operation will possibility fail on mor table rt view when a write operations is concurrency running Key: HUDI-1748 URL: https://issues.apache.org/jira/browse/HUDI-1748 Projec

[GitHub] [hudi] zherenyu831 edited a comment on issue #2707: [SUPPORT] insert_ovewrite_table failed on archiving

2021-03-31 Thread GitBox
zherenyu831 edited a comment on issue #2707: URL: https://github.com/apache/hudi/issues/2707#issuecomment-811565897 @jsbali For make everything clear: On archiving: Since the first replacecommit have 0 partitionToReplaceFileIds, and it failed at https://github.com/apache/hudi/

[GitHub] [hudi] zherenyu831 edited a comment on issue #2707: [SUPPORT] insert_ovewrite_table failed on archiving

2021-03-31 Thread GitBox
zherenyu831 edited a comment on issue #2707: URL: https://github.com/apache/hudi/issues/2707#issuecomment-811565897 @jsbali For make everything clear: On archiving: Since the first replacecommit have 0 partitionToReplaceFileIds, and it failed at https://github.com/apache/hudi/

[GitHub] [hudi] zherenyu831 commented on issue #2707: [SUPPORT] insert_ovewrite_table failed on archiving

2021-03-31 Thread GitBox
zherenyu831 commented on issue #2707: URL: https://github.com/apache/hudi/issues/2707#issuecomment-811565897 @jsbali For make everything clear: On archiving: Since the first replacecommit have 0 partitionToReplaceFileIds, and it failed at https://github.com/apache/hudi/blob/ma

[jira] [Created] (HUDI-1747) Deltastreamer incremental read is not working on the MOR table

2021-03-31 Thread Vinoth Govindarajan (Jira)
Vinoth Govindarajan created HUDI-1747: - Summary: Deltastreamer incremental read is not working on the MOR table Key: HUDI-1747 URL: https://issues.apache.org/jira/browse/HUDI-1747 Project: Apache

[GitHub] [hudi] nsivabalan closed pull request #2750: [HUDI-1754] [HOT_FIX] Revert "[HUDI-1526] Translate the api partitionBy in spark datasource (#2431) due to usage of unavailable apis with older sp

2021-03-31 Thread GitBox
nsivabalan closed pull request #2750: URL: https://github.com/apache/hudi/pull/2750 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [hudi] nsivabalan closed issue #2748: Are we stopping support for spark <2.4.4 with hudi 0.8.0?

2021-03-31 Thread GitBox
nsivabalan closed issue #2748: URL: https://github.com/apache/hudi/issues/2748 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please cont

[GitHub] [hudi] nsivabalan commented on issue #2748: Are we stopping support for spark <2.4.4 with hudi 0.8.0?

2021-03-31 Thread GitBox
nsivabalan commented on issue #2748: URL: https://github.com/apache/hudi/issues/2748#issuecomment-811513554 @aditiwari01 : don't think we are looking to support spark versions < 2.4.3. Its always been the case, and its a documentation issue. we will fix it. -- This is an automated messa

[GitHub] [hudi] codecov-io edited a comment on pull request #2747: [HUDI-1743] Added support for SqlFileBasedTransformer

2021-03-31 Thread GitBox
codecov-io edited a comment on pull request #2747: URL: https://github.com/apache/hudi/pull/2747#issuecomment-810800755 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [hudi] rubenssoto commented on issue #2294: [SUPPORT] java.lang.IllegalArgumentException: Can not create a Path from an empty string on non partitioned COW table

2021-03-31 Thread GitBox
rubenssoto commented on issue #2294: URL: https://github.com/apache/hudi/issues/2294#issuecomment-811477227 @bvaradar is it expected that this bug exists on 0.7.0? What problem this bug cause? -- This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [hudi] umehrot2 commented on a change in pull request #2651: [HUDI-1591] [RFC-26] Improve Hoodie Table Query Performance And Ease Of Use Fo…

2021-03-31 Thread GitBox
umehrot2 commented on a change in pull request #2651: URL: https://github.com/apache/hudi/pull/2651#discussion_r605196012 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/HoodieWriterUtils.scala ## @@ -81,4 +82,53 @@ object HoodieWriterUtils {

[GitHub] [hudi] codecov-io edited a comment on pull request #2747: [HUDI-1743] Added support for SqlFileBasedTransformer

2021-03-31 Thread GitBox
codecov-io edited a comment on pull request #2747: URL: https://github.com/apache/hudi/pull/2747#issuecomment-810800755 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2747?src=pr&el=h1) Report > Merging [#2747](https://codecov.io/gh/apache/hudi/pull/2747?src=pr&el=desc) (61d0222) in

[GitHub] [hudi] codecov-io edited a comment on pull request #2747: [HUDI-1743] Added support for SqlFileBasedTransformer

2021-03-31 Thread GitBox
codecov-io edited a comment on pull request #2747: URL: https://github.com/apache/hudi/pull/2747#issuecomment-810800755 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2747?src=pr&el=h1) Report > Merging [#2747](https://codecov.io/gh/apache/hudi/pull/2747?src=pr&el=desc) (61d0222) in

[GitHub] [hudi] codecov-io edited a comment on pull request #2749: [HUDI-1744][Rollback] rollback fail on mor table when the partition path hasn't any files

2021-03-31 Thread GitBox
codecov-io edited a comment on pull request #2749: URL: https://github.com/apache/hudi/pull/2749#issuecomment-811147597 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2749?src=pr&el=h1) Report > Merging [#2749](https://codecov.io/gh/apache/hudi/pull/2749?src=pr&el=desc) (e97262f) in

[GitHub] [hudi] jsbali commented on pull request #2678: Added support for replace commits in commit showpartitions, commit sh…

2021-03-31 Thread GitBox
jsbali commented on pull request #2678: URL: https://github.com/apache/hudi/pull/2678#issuecomment-811284724 Created the jira https://issues.apache.org/jira/browse/HUDI-1746. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[jira] [Commented] (HUDI-1746) Added support for replace commits in hudi-cli

2021-03-31 Thread Jagmeet Bali (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312583#comment-17312583 ] Jagmeet Bali commented on HUDI-1746: https://github.com/apache/hudi/pull/2678 > Added

[jira] [Created] (HUDI-1746) Added support for replace commits in hudi-cli

2021-03-31 Thread Jagmeet Bali (Jira)
Jagmeet Bali created HUDI-1746: -- Summary: Added support for replace commits in hudi-cli Key: HUDI-1746 URL: https://issues.apache.org/jira/browse/HUDI-1746 Project: Apache Hudi Issue Type: New F

[GitHub] [hudi] aditiwari01 commented on issue #2748: Are we stopping support for spark <2.4.4 with hudi 0.8.0?

2021-03-31 Thread GitBox
aditiwari01 commented on issue #2748: URL: https://github.com/apache/hudi/issues/2748#issuecomment-811280058 Yes. I faced the issue with 2.4.0 version itself. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [hudi] nsivabalan commented on issue #2748: Are we stopping support for spark <2.4.4 with hudi 0.8.0?

2021-03-31 Thread GitBox
nsivabalan commented on issue #2748: URL: https://github.com/apache/hudi/issues/2748#issuecomment-811277730 @vinothchandar : guess issue is w/ spark2.4.1. looks like the api is available from 2.4.2 and not available in 2.4.1. -- This is an automated message from the Apache Git Service.

[GitHub] [hudi] kimberlyamandalu commented on issue #2696: Metadata and runtime exceptions in Hudi 0.7.0 on AWS Glue

2021-03-31 Thread GitBox
kimberlyamandalu commented on issue #2696: URL: https://github.com/apache/hudi/issues/2696#issuecomment-811249839 > @kimberlyamandalu Can you try turning off the metadata table in hoodie to get your pipeline unblocked ? > > ``` > hoodie.metadata.enable=false > ``` > > Th

[GitHub] [hudi] vinothchandar commented on issue #2748: Are we stopping support for spark <2.4.4 with hudi 0.8.0?

2021-03-31 Thread GitBox
vinothchandar commented on issue #2748: URL: https://github.com/apache/hudi/issues/2748#issuecomment-811224728 @aditiwari01 This is a bit of my bad. Actually we have always required Spark 2.4 for a while now (we need `spark-avro` from there). So as long as Spark 2.4 works and Spark 3.x wor

[GitHub] [hudi] vinothchandar commented on issue #2748: Are we stopping support for spark <2.4.4 with hudi 0.8.0?

2021-03-31 Thread GitBox
vinothchandar commented on issue #2748: URL: https://github.com/apache/hudi/issues/2748#issuecomment-811202865 @nsivabalan this would be a release blocker and we need to fix the RC if spark 2.x has issues. agree? -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [hudi] nsivabalan edited a comment on issue #2648: [SUPPORT] a NPE error when reading MOR table in spark datasource

2021-03-31 Thread GitBox
nsivabalan edited a comment on issue #2648: URL: https://github.com/apache/hudi/issues/2648#issuecomment-811188216 @n3nash : can you please add severity label if you have sense of it. just reminding. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hudi] nsivabalan commented on pull request #2655: [WIP] [HUDI-1615] Fixing null schema for delete operation in spark datasource

2021-03-31 Thread GitBox
nsivabalan commented on pull request #2655: URL: https://github.com/apache/hudi/pull/2655#issuecomment-811192144 yeah, I haven't got a chance to fix it. once its ready, will let you know. -- This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [hudi] nsivabalan commented on issue #2717: [SUPPORT] run_sync_tool support hive3.1.2 on hadoop3.1.4

2021-03-31 Thread GitBox
nsivabalan commented on issue #2717: URL: https://github.com/apache/hudi/issues/2717#issuecomment-811181956 sure. Closing this out. we can continue the discussion in PR. [Here](https://issues.apache.org/jira/browse/HUDI-1721) is the right jira (closed the other one as duplicate) -- This

[GitHub] [hudi] nsivabalan closed issue #2717: [SUPPORT] run_sync_tool support hive3.1.2 on hadoop3.1.4

2021-03-31 Thread GitBox
nsivabalan closed issue #2717: URL: https://github.com/apache/hudi/issues/2717 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please cont

[GitHub] [hudi] nsivabalan commented on issue #2648: [SUPPORT] a NPE error when reading MOR table in spark datasource

2021-03-31 Thread GitBox
nsivabalan commented on issue #2648: URL: https://github.com/apache/hudi/issues/2648#issuecomment-811188216 @n3nash : can you please add severity label if you have sense of it. just reminding. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [hudi] nsivabalan commented on issue #2609: [SUPPORT] Presto hudi query slow when compared to parquet

2021-03-31 Thread GitBox
nsivabalan commented on issue #2609: URL: https://github.com/apache/hudi/issues/2609#issuecomment-811177897 thanks Sudha. if you feel you are done w/ you response, feel free to remove the "awaiting-community-help" label -- This is an automated message from the Apache Git Service. To res

[jira] [Updated] (HUDI-1724) run_sync_tool support for hive3.1.2 on hadoop3.1.4

2021-03-31 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-1724: -- Labels: sev:critical user-support-issues (was: sev:triage user-support-issues) > run_s

[GitHub] [hudi] nsivabalan commented on issue #2633: Empty File Slice causing application to fail in small files optimization code

2021-03-31 Thread GitBox
nsivabalan commented on issue #2633: URL: https://github.com/apache/hudi/issues/2633#issuecomment-811179179 @n3nash : Can you please file a tracking jira and close this out. Do add labels w/ severity as appropriate to both issues and jira -- This is an automated message from the Apache

[GitHub] [hudi] yanghua commented on pull request #2747: [HUDI-1743] Added support for SqlFileBasedTransformer

2021-03-31 Thread GitBox
yanghua commented on pull request #2747: URL: https://github.com/apache/hudi/pull/2747#issuecomment-811163812 @vingov Thanks, but the CI has failed, would you please check the reason? If you not sure, you can push an empty commit to retrigger the Travis. -- This is an automated message f

[GitHub] [hudi] nsivabalan commented on issue #2748: Are we stopping support for spark <2.4.4 with hudi 0.8.0?

2021-03-31 Thread GitBox
nsivabalan commented on issue #2748: URL: https://github.com/apache/hudi/issues/2748#issuecomment-811154406 thanks a lot for raising this @aditiwari01 . we are looking into it. https://issues.apache.org/jira/browse/HUDI-1745 -- This is an automated message from the Apache Git Serv

[GitHub] [hudi] codecov-io commented on pull request #2749: [HUDI-1744][Rollback] rollback fail on mor table when the partition path hasn't any files

2021-03-31 Thread GitBox
codecov-io commented on pull request #2749: URL: https://github.com/apache/hudi/pull/2749#issuecomment-811147597 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2749?src=pr&el=h1) Report > Merging [#2749](https://codecov.io/gh/apache/hudi/pull/2749?src=pr&el=desc) (e0dcab7) into [ma

[jira] [Updated] (HUDI-1745) Hudi compilation fails w/ spark version < 2.4.4 due to usage of unavailable spark api

2021-03-31 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-1745: -- Labels: sev:critical (was: ) > Hudi compilation fails w/ spark version < 2.4.4 due to u

[jira] [Updated] (HUDI-1745) Hudi compilation fails w/ spark version < 2.4.4 due to usage of unavailable spark api

2021-03-31 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-1745: -- Description: [https://github.com/apache/hudi/issues/2748]   PR thats of interest: [htt

[jira] [Created] (HUDI-1745) Hudi compilation fails w/ spark version < 2.4.4 due to usage of unavailable spark api

2021-03-31 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-1745: - Summary: Hudi compilation fails w/ spark version < 2.4.4 due to usage of unavailable spark api Key: HUDI-1745 URL: https://issues.apache.org/jira/browse/HUDI-1745

[GitHub] [hudi] nsivabalan opened a new pull request #2750: Revert "[HUDI-1526] Translate the api partitionBy in spark datasource (#2431) due to usage of incompatabile apis with older spark versions

2021-03-31 Thread GitBox
nsivabalan opened a new pull request #2750: URL: https://github.com/apache/hudi/pull/2750 This reverts commit 26da4f546275e8ab6496537743efe73510cb723d. ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.ht

[jira] [Updated] (HUDI-1744) [Rollback] rollback fail on mor table when the partition path hasn't any files

2021-03-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1744: - Labels: pull-request-available (was: ) > [Rollback] rollback fail on mor table when the partition

[GitHub] [hudi] li36909 commented on pull request #2749: [HUDI-1744][Rollback] rollback fail on mor table when the partition path hasn't any files

2021-03-31 Thread GitBox
li36909 commented on pull request #2749: URL: https://github.com/apache/hudi/pull/2749#issuecomment-811060626 cc @n3nash could you help to take a look, thank you -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] codecov-io edited a comment on pull request #2745: [HUDI-1737][hudi-client] Code Cleanup: Extract common method in HoodieCreateHandle & FlinkCreateHandle

2021-03-31 Thread GitBox
codecov-io edited a comment on pull request #2745: URL: https://github.com/apache/hudi/pull/2745#issuecomment-810102368 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2745?src=pr&el=h1) Report > Merging [#2745](https://codecov.io/gh/apache/hudi/pull/2745?src=pr&el=desc) (2d6a76c) in

[GitHub] [hudi] li36909 opened a new pull request #2749: [Rollback] rollback fail on mor table when the partition path hasn't any files

2021-03-31 Thread GitBox
li36909 opened a new pull request #2749: URL: https://github.com/apache/hudi/pull/2749 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the pul

[jira] [Created] (HUDI-1744) [Rollback] rollback fail on mor table when the partition path hasn't any files

2021-03-31 Thread lrz (Jira)
lrz created HUDI-1744: - Summary: [Rollback] rollback fail on mor table when the partition path hasn't any files Key: HUDI-1744 URL: https://issues.apache.org/jira/browse/HUDI-1744 Project: Apache Hudi I

[GitHub] [hudi] nsivabalan commented on a change in pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2021-03-31 Thread GitBox
nsivabalan commented on a change in pull request #2438: URL: https://github.com/apache/hudi/pull/2438#discussion_r604841403 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/KafkaOffsetGen.java ## @@ -247,6 +266,32 @@ private Long delayOffset

[GitHub] [hudi] nsivabalan commented on a change in pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2021-03-31 Thread GitBox
nsivabalan commented on a change in pull request #2438: URL: https://github.com/apache/hudi/pull/2438#discussion_r604840433 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/HoodieDeltaStreamer.java ## @@ -553,6 +555,11 @@ public DeltaSyncServi

[GitHub] [hudi] nsivabalan commented on a change in pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2021-03-31 Thread GitBox
nsivabalan commented on a change in pull request #2438: URL: https://github.com/apache/hudi/pull/2438#discussion_r604828288 ## File path: hudi-integ-test/src/main/java/org/apache/hudi/integ/testsuite/HoodieDeltaStreamerWrapper.java ## @@ -65,7 +65,7 @@ public void scheduleComp

[GitHub] [hudi] nsivabalan commented on pull request #2520: [HUDI-1446] Support skip bootstrapIndex's init in abstract fs view init

2021-03-31 Thread GitBox
nsivabalan commented on pull request #2520: URL: https://github.com/apache/hudi/pull/2520#issuecomment-811001660 btw, CI build is failed. do check it out as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [hudi] nsivabalan commented on a change in pull request #2520: [HUDI-1446] Support skip bootstrapIndex's init in abstract fs view init

2021-03-31 Thread GitBox
nsivabalan commented on a change in pull request #2520: URL: https://github.com/apache/hudi/pull/2520#discussion_r604825768 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java ## @@ -67,6 +68,7 @@ public static final String HOODIE_TIM

[GitHub] [hudi] Sugamber commented on issue #2637: [SUPPORT] - Partial Update : update few columns of a table

2021-03-31 Thread GitBox
Sugamber commented on issue #2637: URL: https://github.com/apache/hudi/issues/2637#issuecomment-810995008 Thank you!!! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [hudi] RocMarshal commented on a change in pull request #2745: [HUDI-1737][hudi-client] Code Cleanup: Extract common method in HoodieCreateHandle & FlinkCreateHandle

2021-03-31 Thread GitBox
RocMarshal commented on a change in pull request #2745: URL: https://github.com/apache/hudi/pull/2745#discussion_r604791723 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieCreateHandle.java ## @@ -179,29 +181,47 @@ public IOType getIOType()

[GitHub] [hudi] RocMarshal commented on a change in pull request #2745: [HUDI-1737][hudi-client] Code Cleanup: Extract common method in HoodieCreateHandle & FlinkCreateHandle

2021-03-31 Thread GitBox
RocMarshal commented on a change in pull request #2745: URL: https://github.com/apache/hudi/pull/2745#discussion_r604791723 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieCreateHandle.java ## @@ -179,29 +181,47 @@ public IOType getIOType()

[GitHub] [hudi] vingov edited a comment on pull request #2747: [HUDI_1743] Added support for SqlFileBasedTransformer

2021-03-31 Thread GitBox
vingov edited a comment on pull request #2747: URL: https://github.com/apache/hudi/pull/2747#issuecomment-81095 @yanghua - I've already filled Jira and linked in the description https://issues.apache.org/jira/browse/HUDI-1743 Updated the title as well. -- This is an automated

[GitHub] [hudi] RocMarshal commented on a change in pull request #2745: [HUDI-1737][hudi-client] Code Cleanup: Extract common method in HoodieCreateHandle & FlinkCreateHandle

2021-03-31 Thread GitBox
RocMarshal commented on a change in pull request #2745: URL: https://github.com/apache/hudi/pull/2745#discussion_r604785572 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieCreateHandle.java ## @@ -180,28 +180,34 @@ public IOType getIOType()

[GitHub] [hudi] vingov edited a comment on pull request #2747: [MINOR] Added support for SqlFileBasedTransformer

2021-03-31 Thread GitBox
vingov edited a comment on pull request #2747: URL: https://github.com/apache/hudi/pull/2747#issuecomment-81095 @yanghua - I've already filled Jira and linked in the description https://issues.apache.org/jira/browse/HUDI-1743 Do you want me to update the title of this pull reques

[GitHub] [hudi] vingov commented on pull request #2747: [MINOR] Added support for SqlFileBasedTransformer

2021-03-31 Thread GitBox
vingov commented on pull request #2747: URL: https://github.com/apache/hudi/pull/2747#issuecomment-81095 @yanghua - I've already filled Jira and linked in the description https://issues.apache.org/jira/browse/HUDI-1743 Do you want me to update the title of this pull request?

[GitHub] [hudi] codecov-io edited a comment on pull request #2742: [HUDI-1738] Emit deletes for flink MOR table streaming read

2021-03-31 Thread GitBox
codecov-io edited a comment on pull request #2742: URL: https://github.com/apache/hudi/pull/2742#issuecomment-810101347 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2742?src=pr&el=h1) Report > Merging [#2742](https://codecov.io/gh/apache/hudi/pull/2742?src=pr&el=desc) (e99b8c5) in

[GitHub] [hudi] codecov-io edited a comment on pull request #2325: [HUDI-699]Fix CompactionCommand and add unit test for CompactionCommand

2021-03-31 Thread GitBox
codecov-io edited a comment on pull request #2325: URL: https://github.com/apache/hudi/pull/2325#issuecomment-742860619 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2325?src=pr&el=h1) Report > Merging [#2325](https://codecov.io/gh/apache/hudi/pull/2325?src=pr&el=desc) (9cb7bf6) in

[GitHub] [hudi] yanghua commented on pull request #2747: [MINOR] Added support for SqlFileBasedTransformer

2021-03-31 Thread GitBox
yanghua commented on pull request #2747: URL: https://github.com/apache/hudi/pull/2747#issuecomment-810887489 @vingov Thanks for your contribution. IMO, this is a feature, it would be better to file a Jira ticket to track it. -- This is an automated message from the Apache Git Service. T

[GitHub] [hudi] yanghua merged pull request #2746: [MINOR] Delete useless UpsertPartitioner for flink integration

2021-03-31 Thread GitBox
yanghua merged pull request #2746: URL: https://github.com/apache/hudi/pull/2746 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please co

[hudi] branch master updated (aa0da72 -> fe16d0d)

2021-03-31 Thread vinoyang
This is an automated email from the ASF dual-hosted git repository. vinoyang pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from aa0da72 Preparation for Avro update (#2650) add fe16d0d [MINOR] Delete useless UpsertPartitioner for flink in

[GitHub] [hudi] liujinhui1994 commented on pull request #2710: [RFC-20][HUDI-648] Implement error log/table for Datasource/DeltaStreamer/WriteClient/Compaction writes

2021-03-31 Thread GitBox
liujinhui1994 commented on pull request #2710: URL: https://github.com/apache/hudi/pull/2710#issuecomment-810842963 @lw309637554 Please help review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

  1   2   >