[GitHub] [hudi] codecov-io commented on pull request #2520: [HUDI-1446] Support skip bootstrapIndex's init in abstract fs view init

2021-02-28 Thread GitBox
codecov-io commented on pull request #2520: URL: https://github.com/apache/hudi/pull/2520#issuecomment-787713017 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2520?src=pr=h1) Report > Merging [#2520](https://codecov.io/gh/apache/hudi/pull/2520?src=pr=desc) (ef091c8) into

[GitHub] [hudi] n3nash commented on pull request #2374: [HUDI-845] Added locking capability to allow multiple writers

2021-02-28 Thread GitBox
n3nash commented on pull request #2374: URL: https://github.com/apache/hudi/pull/2374#issuecomment-787681396 @vinothchandar Code is ready for review. This is an automated message from the Apache Git Service. To respond to

[GitHub] [hudi] n3nash commented on a change in pull request #2611: [HUDI-1646] Provide mechanism to read uncommitted data through InputFormat

2021-02-28 Thread GitBox
n3nash commented on a change in pull request #2611: URL: https://github.com/apache/hudi/pull/2611#discussion_r584465066 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieHiveUtils.java ## @@ -62,6 +67,7 @@ public static final String

[GitHub] [hudi] n3nash commented on a change in pull request #2611: [HUDI-1646] Provide mechanism to read uncommitted data through InputFormat

2021-02-28 Thread GitBox
n3nash commented on a change in pull request #2611: URL: https://github.com/apache/hudi/pull/2611#discussion_r584465066 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieHiveUtils.java ## @@ -62,6 +67,7 @@ public static final String

[GitHub] [hudi] n3nash commented on a change in pull request #2611: [HUDI-1646] Provide mechanism to read uncommitted data through InputFormat

2021-02-28 Thread GitBox
n3nash commented on a change in pull request #2611: URL: https://github.com/apache/hudi/pull/2611#discussion_r584465066 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieHiveUtils.java ## @@ -62,6 +67,7 @@ public static final String

[GitHub] [hudi] codecov-io edited a comment on pull request #2580: [HUDI 1623] Introduce start & end commit times to timeline

2021-02-28 Thread GitBox
codecov-io edited a comment on pull request #2580: URL: https://github.com/apache/hudi/pull/2580#issuecomment-780218110 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2580?src=pr=h1) Report > Merging [#2580](https://codecov.io/gh/apache/hudi/pull/2580?src=pr=desc) (326d233) into

[GitHub] [hudi] codecov-io edited a comment on pull request #2580: [HUDI 1623] Introduce start & end commit times to timeline

2021-02-28 Thread GitBox
codecov-io edited a comment on pull request #2580: URL: https://github.com/apache/hudi/pull/2580#issuecomment-780218110 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[jira] [Assigned] (HUDI-1647) Supports snapshot read for Flink

2021-02-28 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen reassigned HUDI-1647: Assignee: Danny Chen > Supports snapshot read for Flink > > >

[jira] [Created] (HUDI-1647) Supports snapshot read for Flink

2021-02-28 Thread Danny Chen (Jira)
Danny Chen created HUDI-1647: Summary: Supports snapshot read for Flink Key: HUDI-1647 URL: https://issues.apache.org/jira/browse/HUDI-1647 Project: Apache Hudi Issue Type: Sub-task

[jira] [Closed] (HUDI-1638) Some improvements to BucketAssignFunction

2021-02-28 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-1638. Fixed via master branch: 7a11de12764d8f68f296c6e68a22822318bfbefa > Some improvements to BucketAssignFunction

[jira] [Comment Edited] (HUDI-1638) Some improvements to BucketAssignFunction

2021-02-28 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17292631#comment-17292631 ] Danny Chen edited comment on HUDI-1638 at 3/1/21, 5:51 AM: --- Fixed via master

[jira] [Updated] (HUDI-1638) Some improvements to BucketAssignFunction

2021-02-28 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-1638: - Status: In Progress (was: Open) > Some improvements to BucketAssignFunction >

[jira] [Resolved] (HUDI-1638) Some improvements to BucketAssignFunction

2021-02-28 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-1638. -- Resolution: Fixed > Some improvements to BucketAssignFunction >

[GitHub] [hudi] liujinhui1994 commented on pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2021-02-28 Thread GitBox
liujinhui1994 commented on pull request #2438: URL: https://github.com/apache/hudi/pull/2438#issuecomment-787662595 The current implementation is mainly in KafkaOffsetGen @wangxianghu This is an automated message from the

[GitHub] [hudi] liujinhui1994 closed pull request #2337: [HUDI-982] Flink support mor table

2021-02-28 Thread GitBox
liujinhui1994 closed pull request #2337: URL: https://github.com/apache/hudi/pull/2337 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[jira] [Assigned] (HUDI-1638) Some improvements to BucketAssignFunction

2021-02-28 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen reassigned HUDI-1638: Fix Version/s: 0.8.0 Assignee: Danny Chen > Some improvements to BucketAssignFunction >

[GitHub] [hudi] nsivabalan commented on pull request #2612: [HUDI-1563] Adding hudi file sizing/ small file management blog

2021-02-28 Thread GitBox
nsivabalan commented on pull request #2612: URL: https://github.com/apache/hudi/pull/2612#issuecomment-787658034 ![Screen Shot 2021-03-01 at 12 32 47 AM](https://user-images.githubusercontent.com/513218/109456334-9a3d8100-7a26-11eb-881e-5d1e2523185f.png) ![Screen Shot 2021-03-01 at

[GitHub] [hudi] wangxianghu commented on pull request #2337: [HUDI-982] Flink support mor table

2021-02-28 Thread GitBox
wangxianghu commented on pull request #2337: URL: https://github.com/apache/hudi/pull/2337#issuecomment-787657819 @liujinhui1994 It seems this pr is fixed by https://github.com/apache/hudi/commit/7a11de12764d8f68f296c6e68a22822318bfbefa ?

[jira] [Updated] (HUDI-1563) Documentation on small file handling

2021-02-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1563: - Labels: pull-request-available user-support-issues (was: user-support-issues) > Documentation

[GitHub] [hudi] nsivabalan opened a new pull request #2612: [HUDI-1563] Adding hudi file sizing/ small file management blog

2021-02-28 Thread GitBox
nsivabalan opened a new pull request #2612: URL: https://github.com/apache/hudi/pull/2612 ## What is the purpose of the pull request *Adding hudi file sizing blog* ## Brief change log - *Adding hudi file sizing blog* ## Verify this pull request Built the

[jira] [Assigned] (HUDI-1563) Documentation on small file handling

2021-02-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-1563: - Assignee: sivabalan narayanan (was: Nishith Agarwal) > Documentation on small

[GitHub] [hudi] codecov-io edited a comment on pull request #2580: [HUDI 1623] Introduce start & end commit times to timeline

2021-02-28 Thread GitBox
codecov-io edited a comment on pull request #2580: URL: https://github.com/apache/hudi/pull/2580#issuecomment-780218110 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2580?src=pr=h1) Report > Merging [#2580](https://codecov.io/gh/apache/hudi/pull/2580?src=pr=desc) (90d46f8) into

[GitHub] [hudi] codecov-io commented on pull request #2611: [HUDI-1646] Provide mechanism to read uncommitted data through InputFormat

2021-02-28 Thread GitBox
codecov-io commented on pull request #2611: URL: https://github.com/apache/hudi/pull/2611#issuecomment-787641189 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2611?src=pr=h1) Report > Merging [#2611](https://codecov.io/gh/apache/hudi/pull/2611?src=pr=desc) (dc7874d) into

[jira] [Commented] (HUDI-1063) Save in Google Cloud Storage not working

2021-02-28 Thread Volodymyr Burenin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17292615#comment-17292615 ] Volodymyr Burenin commented on HUDI-1063: - Would be nice to see the used hadoop configuration and

[jira] [Updated] (HUDI-1646) Allow support for pre-commit validation

2021-02-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1646: - Labels: pull-request-available (was: ) > Allow support for pre-commit validation >

[GitHub] [hudi] satishkotha opened a new pull request #2611: [HUDI-1646] Provide mechanism to read uncommitted data through InputFormat

2021-02-28 Thread GitBox
satishkotha opened a new pull request #2611: URL: https://github.com/apache/hudi/pull/2611 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of

[jira] [Created] (HUDI-1646) Allow support for pre-commit validation

2021-02-28 Thread satish (Jira)
satish created HUDI-1646: Summary: Allow support for pre-commit validation Key: HUDI-1646 URL: https://issues.apache.org/jira/browse/HUDI-1646 Project: Apache Hudi Issue Type: New Feature

[GitHub] [hudi] satishkotha commented on pull request #2610: [HUDI-1644] Do not delete older rollback instants as part of rollback…

2021-02-28 Thread GitBox
satishkotha commented on pull request #2610: URL: https://github.com/apache/hudi/pull/2610#issuecomment-787631465 > @satishkotha High level looks good to me, can we confirm if we have a equivalent test case on the archiving of rollback instants that simulates the same behavior of not

[jira] [Created] (HUDI-1645) Add unit test to verify clean and rollback instants are archived correctly

2021-02-28 Thread satish (Jira)
satish created HUDI-1645: Summary: Add unit test to verify clean and rollback instants are archived correctly Key: HUDI-1645 URL: https://issues.apache.org/jira/browse/HUDI-1645 Project: Apache Hudi

[jira] [Closed] (HUDI-1632) Supports merge on read write mode for Flink writer

2021-02-28 Thread vinoyang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang closed HUDI-1632. -- Resolution: Implemented Implemented via master branch: 7a11de12764d8f68f296c6e68a22822318bfbefa > Supports

[jira] [Assigned] (HUDI-1632) Supports merge on read write mode for Flink writer

2021-02-28 Thread vinoyang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang reassigned HUDI-1632: -- Assignee: Danny Chen > Supports merge on read write mode for Flink writer >

[jira] [Updated] (HUDI-1632) Supports merge on read write mode for Flink writer

2021-02-28 Thread vinoyang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang updated HUDI-1632: --- Fix Version/s: 0.8.0 > Supports merge on read write mode for Flink writer >

[GitHub] [hudi] yanghua merged pull request #2593: [HUDI-1632] Supports merge on read write mode for Flink writer

2021-02-28 Thread GitBox
yanghua merged pull request #2593: URL: https://github.com/apache/hudi/pull/2593 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[hudi] branch master updated (be257b5 -> 7a11de1)

2021-02-28 Thread vinoyang
This is an automated email from the ASF dual-hosted git repository. vinoyang pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from be257b5 [Hudi-1583]: Fix bug that Hudi will skip remaining log files if there is logFile with zero size in

[GitHub] [hudi] wangxianghu commented on pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2021-02-28 Thread GitBox
wangxianghu commented on pull request #2438: URL: https://github.com/apache/hudi/pull/2438#issuecomment-787616261 > I will add the unit test, and then please review Hi @liujinhui1994 sorry for the day. Can we keep all these changes in `KafkaOffsetGen`, this seems more elegant

[GitHub] [hudi] n3nash commented on pull request #2610: [HUDI-1644] Do not delete older rollback instants as part of rollback…

2021-02-28 Thread GitBox
n3nash commented on pull request #2610: URL: https://github.com/apache/hudi/pull/2610#issuecomment-787615017 @satishkotha High level looks good to me, can we confirm if we have a equivalent test case on the archiving of rollback instants that simulates the same behavior of not leaving

[GitHub] [hudi] codecov-io commented on pull request #2610: [HUDI-1644] Do not delete older rollback instants as part of rollback…

2021-02-28 Thread GitBox
codecov-io commented on pull request #2610: URL: https://github.com/apache/hudi/pull/2610#issuecomment-787602584 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2610?src=pr=h1) Report > Merging [#2610](https://codecov.io/gh/apache/hudi/pull/2610?src=pr=desc) (b108e6f) into

[GitHub] [hudi] codecov-io edited a comment on pull request #2580: [HUDI 1623] Introduce start & end commit times to timeline

2021-02-28 Thread GitBox
codecov-io edited a comment on pull request #2580: URL: https://github.com/apache/hudi/pull/2580#issuecomment-780218110 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2580?src=pr=h1) Report > Merging [#2580](https://codecov.io/gh/apache/hudi/pull/2580?src=pr=desc) (1065c5c) into

[GitHub] [hudi] codecov-io edited a comment on pull request #2593: [HUDI-1632] Supports merge on read write mode for Flink writer

2021-02-28 Thread GitBox
codecov-io edited a comment on pull request #2593: URL: https://github.com/apache/hudi/pull/2593#issuecomment-784220708 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2593?src=pr=h1) Report > Merging [#2593](https://codecov.io/gh/apache/hudi/pull/2593?src=pr=desc) (c22ac8d) into

[GitHub] [hudi] danny0405 commented on a change in pull request #2593: [HUDI-1632] Supports merge on read write mode for Flink writer

2021-02-28 Thread GitBox
danny0405 commented on a change in pull request #2593: URL: https://github.com/apache/hudi/pull/2593#discussion_r584416641 ## File path: hudi-flink/src/main/java/org/apache/hudi/operator/compact/CompactEvent.java ## @@ -0,0 +1,47 @@ +/* + * Licensed to the Apache Software

[GitHub] [hudi] pengzhiwei2018 commented on issue #2609: [SUPPORT] Presto hudi query slow when compared to parquet

2021-02-28 Thread GitBox
pengzhiwei2018 commented on issue #2609: URL: https://github.com/apache/hudi/issues/2609#issuecomment-787593403 Hi @ramachandranms , do query cow table by presto? I have found an issue that query hudi cow table slow than parquet by presto before. And you can try this fix at:

[GitHub] [hudi] danny0405 commented on a change in pull request #2593: [HUDI-1632] Supports merge on read write mode for Flink writer

2021-02-28 Thread GitBox
danny0405 commented on a change in pull request #2593: URL: https://github.com/apache/hudi/pull/2593#discussion_r584411150 ## File path: hudi-flink/src/main/java/org/apache/hudi/operator/compact/CompactCommitEvent.java ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache

[GitHub] [hudi] danny0405 commented on a change in pull request #2593: [HUDI-1632] Supports merge on read write mode for Flink writer

2021-02-28 Thread GitBox
danny0405 commented on a change in pull request #2593: URL: https://github.com/apache/hudi/pull/2593#discussion_r584410935 ## File path: hudi-flink/src/main/java/org/apache/hudi/operator/FlinkOptions.java ## @@ -165,6 +165,42 @@ private FlinkOptions() {

[GitHub] [hudi] danny0405 commented on a change in pull request #2593: [HUDI-1632] Supports merge on read write mode for Flink writer

2021-02-28 Thread GitBox
danny0405 commented on a change in pull request #2593: URL: https://github.com/apache/hudi/pull/2593#discussion_r584410723 ## File path: hudi-flink/src/main/java/org/apache/hudi/operator/FlinkOptions.java ## @@ -165,6 +165,42 @@ private FlinkOptions() {

[GitHub] [hudi] danny0405 commented on a change in pull request #2593: [HUDI-1632] Supports merge on read write mode for Flink writer

2021-02-28 Thread GitBox
danny0405 commented on a change in pull request #2593: URL: https://github.com/apache/hudi/pull/2593#discussion_r584409961 ## File path: hudi-flink/src/main/java/org/apache/hudi/operator/FlinkOptions.java ## @@ -165,6 +165,42 @@ private FlinkOptions() {

[GitHub] [hudi] codecov-io edited a comment on pull request #2374: [HUDI-845] Added locking capability to allow multiple writers

2021-02-28 Thread GitBox
codecov-io edited a comment on pull request #2374: URL: https://github.com/apache/hudi/pull/2374#issuecomment-750782300 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2374?src=pr=h1) Report > Merging [#2374](https://codecov.io/gh/apache/hudi/pull/2374?src=pr=desc) (2d7d890) into

[jira] [Updated] (HUDI-1644) Do not delete rollback instants in RollbackActionExecutor

2021-02-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1644: - Labels: pull-request-available (was: ) > Do not delete rollback instants in

[GitHub] [hudi] satishkotha opened a new pull request #2610: [HUDI-1644] Do not delete older rollback instants as part of rollback…

2021-02-28 Thread GitBox
satishkotha opened a new pull request #2610: URL: https://github.com/apache/hudi/pull/2610 ## What is the purpose of the pull request Archival can take care of removing old rollback instants cleanly ## Brief change log * rollback instants are cleaned up by archival

[jira] [Created] (HUDI-1644) Do not delete rollback instants in RollbackActionExecutor

2021-02-28 Thread satish (Jira)
satish created HUDI-1644: Summary: Do not delete rollback instants in RollbackActionExecutor Key: HUDI-1644 URL: https://issues.apache.org/jira/browse/HUDI-1644 Project: Apache Hudi Issue Type: Bug

[jira] [Closed] (HUDI-1539) Bug in HoodieCombineRealtimeRecordReader returns wrong results

2021-02-28 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish closed HUDI-1539. > Bug in HoodieCombineRealtimeRecordReader returns wrong results >

[jira] [Resolved] (HUDI-1539) Bug in HoodieCombineRealtimeRecordReader returns wrong results

2021-02-28 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish resolved HUDI-1539. -- Resolution: Fixed > Bug in HoodieCombineRealtimeRecordReader returns wrong results >

[GitHub] [hudi] teeyog commented on a change in pull request #2475: [HUDI-1527] automatically infer the data directory, users only need to specify the table directory

2021-02-28 Thread GitBox
teeyog commented on a change in pull request #2475: URL: https://github.com/apache/hudi/pull/2475#discussion_r584401119 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/DefaultSource.scala ## @@ -84,6 +88,26 @@ class DefaultSource extends

[GitHub] [hudi] vinothchandar commented on a change in pull request #2494: [HUDI-1552] Improve performance of key lookups from base file in Metadata Table.

2021-02-28 Thread GitBox
vinothchandar commented on a change in pull request #2494: URL: https://github.com/apache/hudi/pull/2494#discussion_r584393784 ## File path: hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java ## @@ -112,13 +113,59 @@ private void initIfNeeded()

[GitHub] [hudi] nsivabalan commented on a change in pull request #2494: [HUDI-1552] Improve performance of key lookups from base file in Metadata Table.

2021-02-28 Thread GitBox
nsivabalan commented on a change in pull request #2494: URL: https://github.com/apache/hudi/pull/2494#discussion_r584328039 ## File path: hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java ## @@ -112,13 +113,59 @@ private void initIfNeeded() {

[GitHub] [hudi] codecov-io edited a comment on pull request #2577: [HUDI-1618] Fixing NPE with Parquet src in multi table delta streamer

2021-02-28 Thread GitBox
codecov-io edited a comment on pull request #2577: URL: https://github.com/apache/hudi/pull/2577#issuecomment-779312995 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2577?src=pr=h1) Report > Merging [#2577](https://codecov.io/gh/apache/hudi/pull/2577?src=pr=desc) (d5fb81f) into

[GitHub] [hudi] nsivabalan commented on a change in pull request #2577: [HUDI-1618] Fixing NPE with Parquet src in multi table delta streamer

2021-02-28 Thread GitBox
nsivabalan commented on a change in pull request #2577: URL: https://github.com/apache/hudi/pull/2577#discussion_r584311235 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/functional/TestHoodieDeltaStreamer.java ## @@ -965,16 +969,30 @@ public void

[jira] [Commented] (HUDI-1063) Save in Google Cloud Storage not working

2021-02-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17292433#comment-17292433 ] sivabalan narayanan commented on HUDI-1063: --- [~vburenin]: Your help here is much appreciated. 

[GitHub] [hudi] lw309637554 commented on pull request #2136: [HUDI-37] Persist the HoodieIndex type in the hoodie.properties file

2021-02-28 Thread GitBox
lw309637554 commented on pull request #2136: URL: https://github.com/apache/hudi/pull/2136#issuecomment-787468796 > @lw309637554 @vinothchandar : can you folks get this to completion, its been open for a while. Would be nice to have this in. We might also add more documentation in fax or

[GitHub] [hudi] lw309637554 commented on pull request #2160: [HUDI-865] Improve Hive Syncing by directly translating avro schema to Hive types

2021-02-28 Thread GitBox
lw309637554 commented on pull request #2160: URL: https://github.com/apache/hudi/pull/2160#issuecomment-787467468 > @lw309637554 : Can you please check the feedback and address them. would be nice to have this in. okay

[GitHub] [hudi] nsivabalan commented on pull request #2577: [HUDI-1618] Fixing NPE with Parquet src in multi table delta streamer

2021-02-28 Thread GitBox
nsivabalan commented on pull request #2577: URL: https://github.com/apache/hudi/pull/2577#issuecomment-787466857 @yanghua : addressed all comments. have responded to one of your feedback. Feel free to check it out when you can.

[GitHub] [hudi] nsivabalan commented on a change in pull request #2577: [HUDI-1618] Fixing NPE with Parquet src in multi table delta streamer

2021-02-28 Thread GitBox
nsivabalan commented on a change in pull request #2577: URL: https://github.com/apache/hudi/pull/2577#discussion_r584311235 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/functional/TestHoodieDeltaStreamer.java ## @@ -965,16 +969,30 @@ public void

[GitHub] [hudi] lw309637554 commented on a change in pull request #2475: [HUDI-1527] automatically infer the data directory, users only need to specify the table directory

2021-02-28 Thread GitBox
lw309637554 commented on a change in pull request #2475: URL: https://github.com/apache/hudi/pull/2475#discussion_r584307500 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/DefaultSource.scala ## @@ -84,6 +88,26 @@ class DefaultSource extends

[GitHub] [hudi] garyli1019 commented on a change in pull request #2593: [HUDI-1632] Supports merge on read write mode for Flink writer

2021-02-28 Thread GitBox
garyli1019 commented on a change in pull request #2593: URL: https://github.com/apache/hudi/pull/2593#discussion_r584291118 ## File path: hudi-flink/src/main/java/org/apache/hudi/operator/FlinkOptions.java ## @@ -165,6 +165,42 @@ private FlinkOptions() {

[GitHub] [hudi] nsivabalan commented on a change in pull request #2596: [HUDI-1636] Support Builder Pattern To Build Table Properties For HoodieTableConfig

2021-02-28 Thread GitBox
nsivabalan commented on a change in pull request #2596: URL: https://github.com/apache/hudi/pull/2596#discussion_r584292305 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java ## @@ -258,4 +260,164 @@ public String getArchivelogFolder()