[jira] [Assigned] (HUDI-1509) Major performance degradation due to rewriting records with default values

2021-01-08 Thread Nishith Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishith Agarwal reassigned HUDI-1509: - Assignee: Nishith Agarwal > Major performance degradation due to rewriting records with d

[GitHub] [hudi] codecov-io commented on pull request #2421: [Hudi-1502] MOR rollback and restore support for metadata sync

2021-01-08 Thread GitBox
codecov-io commented on pull request #2421: URL: https://github.com/apache/hudi/pull/2421#issuecomment-757112911 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2421?src=pr&el=h1) Report > Merging [#2421](https://codecov.io/gh/apache/hudi/pull/2421?src=pr&el=desc) (0227058) into [ma

[GitHub] [hudi] satishkotha commented on pull request #2422: [WIP] [HUDI-1276] [HUDI-1459] Make Clustering/ReplaceCommit and Metadata table be compatible

2021-01-08 Thread GitBox
satishkotha commented on pull request #2422: URL: https://github.com/apache/hudi/pull/2422#issuecomment-757110216 > @satishkotha is driving this to completion. @prashantwason you might want to chime in on the algorithm part and let us know if you see red flags, when you get a chance. >

[GitHub] [hudi] n3nash commented on a change in pull request #2379: [HUDI-1399] support a independent clustering spark job to asynchronously clustering

2021-01-08 Thread GitBox
n3nash commented on a change in pull request #2379: URL: https://github.com/apache/hudi/pull/2379#discussion_r554302023 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/functional/TestHoodieDeltaStreamer.java ## @@ -672,14 +681,72 @@ public void testInlineC

[GitHub] [hudi] n3nash commented on a change in pull request #2379: [HUDI-1399] support a independent clustering spark job to asynchronously clustering

2021-01-08 Thread GitBox
n3nash commented on a change in pull request #2379: URL: https://github.com/apache/hudi/pull/2379#discussion_r554302023 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/functional/TestHoodieDeltaStreamer.java ## @@ -672,14 +681,72 @@ public void testInlineC

[GitHub] [hudi] n3nash commented on a change in pull request #2379: [HUDI-1399] support a independent clustering spark job to asynchronously clustering

2021-01-08 Thread GitBox
n3nash commented on a change in pull request #2379: URL: https://github.com/apache/hudi/pull/2379#discussion_r554302023 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/functional/TestHoodieDeltaStreamer.java ## @@ -672,14 +681,72 @@ public void testInlineC

[GitHub] [hudi] n3nash commented on a change in pull request #2379: [HUDI-1399] support a independent clustering spark job to asynchronously clustering

2021-01-08 Thread GitBox
n3nash commented on a change in pull request #2379: URL: https://github.com/apache/hudi/pull/2379#discussion_r554301665 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieClusteringJob.java ## @@ -0,0 +1,174 @@ +/* + * Licensed to the Apache Software Fo

[GitHub] [hudi] n3nash commented on a change in pull request #2379: [HUDI-1399] support a independent clustering spark job to asynchronously clustering

2021-01-08 Thread GitBox
n3nash commented on a change in pull request #2379: URL: https://github.com/apache/hudi/pull/2379#discussion_r554301213 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/AbstractHoodieWriteClient.java ## @@ -711,12 +711,30 @@ public void rollbac

[GitHub] [hudi] vinothchandar commented on pull request #1938: [HUDI-920] Support Incremental query for MOR table

2021-01-08 Thread GitBox
vinothchandar commented on pull request #1938: URL: https://github.com/apache/hudi/pull/1938#issuecomment-757103037 Thanks @garyli1019 ! You are awesome. Will take a pass, after CI is happy. and land, This is an auto

[GitHub] [hudi] wangxianghu commented on pull request #2419: Hudi 1421

2021-01-08 Thread GitBox
wangxianghu commented on pull request #2419: URL: https://github.com/apache/hudi/pull/2419#issuecomment-757100778 Hi @loukey-lj thanks for your contribution please check why ci failed This is an automated message from the

[GitHub] [hudi] garyli1019 commented on pull request #1938: [HUDI-920] Support Incremental query for MOR table

2021-01-08 Thread GitBox
garyli1019 commented on pull request #1938: URL: https://github.com/apache/hudi/pull/1938#issuecomment-757097457 @vinothchandar addressed comments. You are right, the inconsistency was caused by the test data generator. The CI was happy after I changed from `generateUpdates` to `generateUn

[GitHub] [hudi] garyli1019 commented on a change in pull request #1938: [HUDI-920] Support Incremental query for MOR table

2021-01-08 Thread GitBox
garyli1019 commented on a change in pull request #1938: URL: https://github.com/apache/hudi/pull/1938#discussion_r554291761 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieInputFormatUtils.java ## @@ -470,4 +471,45 @@ private static HoodieBaseFile

[GitHub] [hudi] garyli1019 commented on a change in pull request #1938: [HUDI-920] Support Incremental query for MOR table

2021-01-08 Thread GitBox
garyli1019 commented on a change in pull request #1938: URL: https://github.com/apache/hudi/pull/1938#discussion_r554291505 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieInputFormatUtils.java ## @@ -470,4 +471,45 @@ private static HoodieBaseFile

[GitHub] [hudi] garyli1019 commented on a change in pull request #1938: [HUDI-920] Support Incremental query for MOR table

2021-01-08 Thread GitBox
garyli1019 commented on a change in pull request #1938: URL: https://github.com/apache/hudi/pull/1938#discussion_r554291421 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/MergeOnReadIncrementalRelation.scala ## @@ -0,0 +1,218 @@ +/* + * Licensed

[GitHub] [hudi] garyli1019 commented on a change in pull request #1938: [HUDI-920] Support Incremental query for MOR table

2021-01-08 Thread GitBox
garyli1019 commented on a change in pull request #1938: URL: https://github.com/apache/hudi/pull/1938#discussion_r554291307 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/MergeOnReadIncrementalRelation.scala ## @@ -0,0 +1,218 @@ +/* + * Licensed

[GitHub] [hudi] garyli1019 commented on a change in pull request #1938: [HUDI-920] Support Incremental query for MOR table

2021-01-08 Thread GitBox
garyli1019 commented on a change in pull request #1938: URL: https://github.com/apache/hudi/pull/1938#discussion_r554287262 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieInputFormatUtils.java ## @@ -470,4 +471,45 @@ private static HoodieBaseFile

[GitHub] [hudi] vinothchandar commented on pull request #2379: [HUDI-1399] support a independent clustering spark job to asynchronously clustering

2021-01-08 Thread GitBox
vinothchandar commented on pull request #2379: URL: https://github.com/apache/hudi/pull/2379#issuecomment-757090817 @n3nash can you shepherd this and land ? @satishkotha is helping out with the other PR. This is an auto

[GitHub] [hudi] vinothchandar commented on a change in pull request #2421: [Hudi-1502] MOR rollback and restore support for metadata sync

2021-01-08 Thread GitBox
vinothchandar commented on a change in pull request #2421: URL: https://github.com/apache/hudi/pull/2421#discussion_r554264580 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieAppendHandle.java ## @@ -164,6 +164,7 @@ private void init(HoodieR

[GitHub] [hudi] umehrot2 commented on a change in pull request #2417: [HUDI-1479] Use HoodieEngineContext to parallelize fetching of partiton paths

2021-01-08 Thread GitBox
umehrot2 commented on a change in pull request #2417: URL: https://github.com/apache/hudi/pull/2417#discussion_r554278832 ## File path: hudi-common/src/main/java/org/apache/hudi/metadata/FileSystemBackedTableMetadata.java ## @@ -49,12 +60,48 @@ public FileSystemBackedTableMet

[GitHub] [hudi] umehrot2 commented on a change in pull request #2417: [HUDI-1479] Use HoodieEngineContext to parallelize fetching of partiton paths

2021-01-08 Thread GitBox
umehrot2 commented on a change in pull request #2417: URL: https://github.com/apache/hudi/pull/2417#discussion_r554270231 ## File path: hudi-common/src/main/java/org/apache/hudi/common/engine/HoodieMREngineContext.java ## @@ -0,0 +1,91 @@ +/* + * Licensed to the Apache Softwar

[GitHub] [hudi] vinothchandar commented on a change in pull request #2417: [HUDI-1479] Use HoodieEngineContext to parallelize fetching of partiton paths

2021-01-08 Thread GitBox
vinothchandar commented on a change in pull request #2417: URL: https://github.com/apache/hudi/pull/2417#discussion_r554260296 ## File path: hudi-common/src/main/java/org/apache/hudi/common/engine/HoodieMREngineContext.java ## @@ -0,0 +1,91 @@ +/* + * Licensed to the Apache So

[GitHub] [hudi] vinothchandar commented on a change in pull request #2417: [HUDI-1479] Use HoodieEngineContext to parallelize fetching of partiton paths

2021-01-08 Thread GitBox
vinothchandar commented on a change in pull request #2417: URL: https://github.com/apache/hudi/pull/2417#discussion_r554259024 ## File path: hudi-common/src/main/java/org/apache/hudi/metadata/FileSystemBackedTableMetadata.java ## @@ -49,12 +60,48 @@ public FileSystemBackedTab

[GitHub] [hudi] vinothchandar commented on pull request #2422: [WIP] [HUDI-1276] [HUDI-1459] Make Clustering/ReplaceCommit and Metadata table be compatible

2021-01-08 Thread GitBox
vinothchandar commented on pull request #2422: URL: https://github.com/apache/hudi/pull/2422#issuecomment-757058068 @satishkotha is driving this to completion. @prashantwason you might want to chime in on the algorithm part and let us know if you see red flags, when you get a chance.

[GitHub] [hudi] vinothchandar commented on pull request #2422: [WIP] [HUDI-1276] [HUDI-1459] Make Clustering/ReplaceCommit and Metadata table be compatible

2021-01-08 Thread GitBox
vinothchandar commented on pull request #2422: URL: https://github.com/apache/hudi/pull/2422#issuecomment-757056595 We should probably keep the archival code as a defense, with a `log.warn()` This is an automated message from

[hudi] branch master updated (17df517 -> c151147)

2021-01-08 Thread vinoyang
This is an automated email from the ASF dual-hosted git repository. vinoyang pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 17df517 [HUDI-1510] Move HoodieEngineContext and its dependencies to hudi-common (#2410) add c151147 [MINOR]

[GitHub] [hudi] yanghua merged pull request #2415: [MINOR] Sync HUDI-1196 to FlinkWriteHelper

2021-01-08 Thread GitBox
yanghua merged pull request #2415: URL: https://github.com/apache/hudi/pull/2415 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] satishkotha commented on a change in pull request #2422: [WIP] [HUDI-1276] [HUDI-1459] Make Clustering/ReplaceCommit and Metadata table be compatible

2021-01-08 Thread GitBox
satishkotha commented on a change in pull request #2422: URL: https://github.com/apache/hudi/pull/2422#discussion_r554244115 ## File path: hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java ## @@ -92,13 +95,36 @@ case HoodieTimeline.SAVEPOIN

[GitHub] [hudi] umehrot2 commented on a change in pull request #2417: [HUDI-1479] Use HoodieEngineContext to parallelize fetching of partiton paths

2021-01-08 Thread GitBox
umehrot2 commented on a change in pull request #2417: URL: https://github.com/apache/hudi/pull/2417#discussion_r554245704 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/view/FileSystemViewManager.java ## @@ -159,12 +160,12 @@ private static HoodieTableFil

[GitHub] [hudi] umehrot2 commented on a change in pull request #2417: [HUDI-1479] Use HoodieEngineContext to parallelize fetching of partiton paths

2021-01-08 Thread GitBox
umehrot2 commented on a change in pull request #2417: URL: https://github.com/apache/hudi/pull/2417#discussion_r554244342 ## File path: hudi-common/src/main/java/org/apache/hudi/metadata/FileSystemBackedTableMetadata.java ## @@ -64,6 +111,6 @@ public FileSystemBackedTableMeta

[GitHub] [hudi] umehrot2 commented on a change in pull request #2417: [HUDI-1479] Use HoodieEngineContext to parallelize fetching of partiton paths

2021-01-08 Thread GitBox
umehrot2 commented on a change in pull request #2417: URL: https://github.com/apache/hudi/pull/2417#discussion_r554243274 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/view/FileSystemViewManager.java ## @@ -159,12 +160,12 @@ private static HoodieTableFil

[GitHub] [hudi] umehrot2 commented on a change in pull request #2417: [HUDI-1479] Use HoodieEngineContext to parallelize fetching of partiton paths

2021-01-08 Thread GitBox
umehrot2 commented on a change in pull request #2417: URL: https://github.com/apache/hudi/pull/2417#discussion_r554243165 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieRealtimeInputFormatUtils.java ## @@ -82,7 +82,7 @@ HoodieTableMetaClie

[GitHub] [hudi] umehrot2 commented on a change in pull request #2417: [HUDI-1479] Use HoodieEngineContext to parallelize fetching of partiton paths

2021-01-08 Thread GitBox
umehrot2 commented on a change in pull request #2417: URL: https://github.com/apache/hudi/pull/2417#discussion_r554242901 ## File path: hudi-common/src/main/java/org/apache/hudi/metadata/FileSystemBackedTableMetadata.java ## @@ -49,12 +60,48 @@ public FileSystemBackedTableMet

[GitHub] [hudi] umehrot2 commented on a change in pull request #2417: [HUDI-1479] Use HoodieEngineContext to parallelize fetching of partiton paths

2021-01-08 Thread GitBox
umehrot2 commented on a change in pull request #2417: URL: https://github.com/apache/hudi/pull/2417#discussion_r554242390 ## File path: hudi-common/src/main/java/org/apache/hudi/metadata/FileSystemBackedTableMetadata.java ## @@ -49,12 +60,48 @@ public FileSystemBackedTableMet

[GitHub] [hudi] umehrot2 commented on a change in pull request #2417: [HUDI-1479] Use HoodieEngineContext to parallelize fetching of partiton paths

2021-01-08 Thread GitBox
umehrot2 commented on a change in pull request #2417: URL: https://github.com/apache/hudi/pull/2417#discussion_r554241357 ## File path: hudi-common/src/main/java/org/apache/hudi/common/engine/HoodieMREngineContext.java ## @@ -0,0 +1,91 @@ +/* + * Licensed to the Apache Softwar

[GitHub] [hudi] umehrot2 commented on a change in pull request #2417: [HUDI-1479] Use HoodieEngineContext to parallelize fetching of partiton paths

2021-01-08 Thread GitBox
umehrot2 commented on a change in pull request #2417: URL: https://github.com/apache/hudi/pull/2417#discussion_r554237778 ## File path: hudi-common/src/main/java/org/apache/hudi/common/config/HoodieMetadataConfig.java ## @@ -39,6 +39,7 @@ // Validate contents of Metadata Ta

[GitHub] [hudi] umehrot2 commented on a change in pull request #2417: [HUDI-1479] Use HoodieEngineContext to parallelize fetching of partiton paths

2021-01-08 Thread GitBox
umehrot2 commented on a change in pull request #2417: URL: https://github.com/apache/hudi/pull/2417#discussion_r554237558 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java ## @@ -221,8 +221,9 @@ public Hood

[GitHub] [hudi] prashantwason commented on a change in pull request #2417: [HUDI-1479] Use HoodieEngineContext to parallelize fetching of partiton paths

2021-01-08 Thread GitBox
prashantwason commented on a change in pull request #2417: URL: https://github.com/apache/hudi/pull/2417#discussion_r554237450 ## File path: hudi-common/src/main/java/org/apache/hudi/metadata/FileSystemBackedTableMetadata.java ## @@ -49,12 +60,48 @@ public FileSystemBackedTab

[GitHub] [hudi] vinothchandar commented on pull request #2422: [WIP] [HUDI-1276] [HUDI-1459] Make Clustering/ReplaceCommit and Metadata table be compatible

2021-01-08 Thread GitBox
vinothchandar commented on pull request #2422: URL: https://github.com/apache/hudi/pull/2422#issuecomment-757038688 cc @prashantwason @satishkotha kindly take a quick pass. I still have to - add tests, handle the existing deletion code in archival. but the algorithm is there in th

[jira] [Updated] (HUDI-1276) delete replaced file groups during clean

2021-01-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1276: - Labels: pull-request-available (was: ) > delete replaced file groups during clean > -

[GitHub] [hudi] vinothchandar opened a new pull request #2422: [WIP] [HUDI-1276] [HUDI-1459] Make Clustering/ReplaceCommit and Metadata ta…

2021-01-08 Thread GitBox
vinothchandar opened a new pull request #2422: URL: https://github.com/apache/hudi/pull/2422 …ble be compatible ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* #

[jira] [Commented] (HUDI-1276) delete replaced file groups during clean

2021-01-08 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17261638#comment-17261638 ] Vinoth Chandar commented on HUDI-1276: -- Case 1: So, we can only restore within the co

[GitHub] [hudi] bvaradar commented on issue #2409: [SUPPORT] Spark structured Streaming writes to Hudi and synchronizes Hive to create only read-optimized tables without creating real-time tables

2021-01-08 Thread GitBox
bvaradar commented on issue #2409: URL: https://github.com/apache/hudi/issues/2409#issuecomment-757035695 @wosow : The _rt table syncing happens after _ro table and I see an HiveMetaStore exception when updating commit time in the _ro table saying that the table does not exist. This is wei

[GitHub] [hudi] Rap70r closed issue #2416: When will version 0.6.1 be released?

2021-01-08 Thread GitBox
Rap70r closed issue #2416: URL: https://github.com/apache/hudi/issues/2416 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[GitHub] [hudi] prashanthvg89 commented on pull request #2069: [WIP][HUDI-945] Cleanup Spillable map files eagerly for DiskBasedMap

2021-01-08 Thread GitBox
prashanthvg89 commented on pull request #2069: URL: https://github.com/apache/hudi/pull/2069#issuecomment-756969267 I saw this failure in Spark streaming job writing to Hudi. Is this still being worked on and is there an ETA for this? Hudi version: 0.6.0 Spark version: 2.4.0 EM

[GitHub] [hudi] codecov-io commented on pull request #2420: [HUDI-1514] Avoid raw type use for parameter of interface

2021-01-08 Thread GitBox
codecov-io commented on pull request #2420: URL: https://github.com/apache/hudi/pull/2420#issuecomment-756960638 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2420?src=pr&el=h1) Report > Merging [#2420](https://codecov.io/gh/apache/hudi/pull/2420?src=pr&el=desc) (fb35a0f) into [ma

[jira] [Commented] (HUDI-1276) delete replaced file groups during clean

2021-01-08 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17261540#comment-17261540 ] Prashant Wason commented on HUDI-1276: -- One more think to verify is how the restore w

[jira] [Commented] (HUDI-1276) delete replaced file groups during clean

2021-01-08 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17261508#comment-17261508 ] Vinoth Chandar commented on HUDI-1276: -- >But, probably we can ask users to KEEP_LATES

[jira] [Assigned] (HUDI-1276) delete replaced file groups during clean

2021-01-08 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-1276: Assignee: Vinoth Chandar (was: satish) > delete replaced file groups during clean > --

[jira] [Updated] (HUDI-1276) delete replaced file groups during clean

2021-01-08 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1276: - Status: Open (was: New) > delete replaced file groups during clean >

[jira] [Commented] (HUDI-1276) delete replaced file groups during clean

2021-01-08 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17261507#comment-17261507 ] satish commented on HUDI-1276: -- [~vinoth] high level looks reasonable. Few edge cases below.

[GitHub] [hudi] vinothchandar commented on a change in pull request #2421: [WIP][Hudi-1502] MOR rollback and restore support for metadata sync

2021-01-08 Thread GitBox
vinothchandar commented on a change in pull request #2421: URL: https://github.com/apache/hudi/pull/2421#discussion_r554079488 ## File path: hudi-common/src/test/java/org/apache/hudi/common/table/TestTimelineUtils.java ## @@ -181,10 +187,113 @@ public void testRestoreInstants(

[jira] [Commented] (HUDI-1276) delete replaced file groups during clean

2021-01-08 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17261437#comment-17261437 ] Vinoth Chandar commented on HUDI-1276: -- [~satishkotha] Dumping my thoughts here. I sp

[GitHub] [hudi] nsivabalan commented on a change in pull request #2421: [WIP][Hudi-1502] MOR rollback and restore support for metadata sync

2021-01-08 Thread GitBox
nsivabalan commented on a change in pull request #2421: URL: https://github.com/apache/hudi/pull/2421#discussion_r554055131 ## File path: hudi-common/src/test/java/org/apache/hudi/common/table/TestTimelineUtils.java ## @@ -181,10 +187,113 @@ public void testRestoreInstants() t

[GitHub] [hudi] nsivabalan opened a new pull request #2421: [WIP][Hudi-1502] MOR rollback and restore support for metadata sync

2021-01-08 Thread GitBox
nsivabalan opened a new pull request #2421: URL: https://github.com/apache/hudi/pull/2421 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the

[GitHub] [hudi] codecov-io edited a comment on pull request #2415: [MINOR] Sync HUDI-1196 to FlinkWriteHelper

2021-01-08 Thread GitBox
codecov-io edited a comment on pull request #2415: URL: https://github.com/apache/hudi/pull/2415#issuecomment-756825549 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2415?src=pr&el=h1) Report > Merging [#2415](https://codecov.io/gh/apache/hudi/pull/2415?src=pr&el=desc) (1dd00dd) in

[GitHub] [hudi] codecov-io edited a comment on pull request #2415: [MINOR] Sync HUDI-1196 to FlinkWriteHelper

2021-01-08 Thread GitBox
codecov-io edited a comment on pull request #2415: URL: https://github.com/apache/hudi/pull/2415#issuecomment-756825549 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2415?src=pr&el=h1) Report > Merging [#2415](https://codecov.io/gh/apache/hudi/pull/2415?src=pr&el=desc) (1dd00dd) in

[GitHub] [hudi] lw309637554 commented on a change in pull request #2417: [HUDI-1479] Use HoodieEngineContext to parallelize fetching of partiton paths

2021-01-08 Thread GitBox
lw309637554 commented on a change in pull request #2417: URL: https://github.com/apache/hudi/pull/2417#discussion_r554041498 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/view/FileSystemViewManager.java ## @@ -159,12 +160,12 @@ private static HoodieTable

[GitHub] [hudi] lw309637554 edited a comment on pull request #2418: [HUDI-1266] Add unit test for validating replacecommit rollback

2021-01-08 Thread GitBox
lw309637554 edited a comment on pull request #2418: URL: https://github.com/apache/hudi/pull/2418#issuecomment-756830958 @satishkotha just two minor comments This is an automated message from the Apache Git Service. To respon

[jira] [Updated] (HUDI-1514) Avoid raw type use for parameter of interface

2021-01-08 Thread puyvqi (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] puyvqi updated HUDI-1514: - Status: Patch Available (was: In Progress) > Avoid raw type use for parameter of interface > --

[GitHub] [hudi] lw309637554 commented on pull request #2418: [HUDI-1266] Add unit test for validating replacecommit rollback

2021-01-08 Thread GitBox
lw309637554 commented on pull request #2418: URL: https://github.com/apache/hudi/pull/2418#issuecomment-756830958 @satishkotha just two minor comment This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hudi] lw309637554 commented on a change in pull request #2418: [HUDI-1266] Add unit test for validating replacecommit rollback

2021-01-08 Thread GitBox
lw309637554 commented on a change in pull request #2418: URL: https://github.com/apache/hudi/pull/2418#discussion_r554025924 ## File path: hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/table/action/rollback/HoodieClientRollbackTestBase.java ## @@ -96,4 +99,61 @@

[GitHub] [hudi] lw309637554 commented on a change in pull request #2418: [HUDI-1266] Add unit test for validating replacecommit rollback

2021-01-08 Thread GitBox
lw309637554 commented on a change in pull request #2418: URL: https://github.com/apache/hudi/pull/2418#discussion_r554024960 ## File path: hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/table/action/rollback/HoodieClientRollbackTestBase.java ## @@ -96,4 +99,61 @@

[jira] [Updated] (HUDI-1514) Avoid raw type use for parameter of interface

2021-01-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1514: - Labels: pull-request-available (was: ) > Avoid raw type use for parameter of interface > --

[GitHub] [hudi] puyvqi opened a new pull request #2420: [HUDI-1514] Avoid raw type use for parameter of interface

2021-01-08 Thread GitBox
puyvqi opened a new pull request #2420: URL: https://github.com/apache/hudi/pull/2420 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the pull

[GitHub] [hudi] codecov-io commented on pull request #2415: [MINOR] Sync HUDI-1196 to FlinkWriteHelper

2021-01-08 Thread GitBox
codecov-io commented on pull request #2415: URL: https://github.com/apache/hudi/pull/2415#issuecomment-756825549 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2415?src=pr&el=h1) Report > Merging [#2415](https://codecov.io/gh/apache/hudi/pull/2415?src=pr&el=desc) (1dd00dd) into [ma

[GitHub] [hudi] Rap70r commented on issue #2416: When will version 0.6.1 be released?

2021-01-08 Thread GitBox
Rap70r commented on issue #2416: URL: https://github.com/apache/hudi/issues/2416#issuecomment-756818455 Hello vinothchandar, Thank you for getting back to me. Yes, I would like to use Hudi with EMR 6.2.0 and Spark 3.0. Thank you --

[GitHub] [hudi] garyli1019 commented on pull request #1938: [HUDI-920] Support Incremental query for MOR table

2021-01-08 Thread GitBox
garyli1019 commented on pull request #1938: URL: https://github.com/apache/hudi/pull/1938#issuecomment-756781554 Thanks, will update this PR before Friday night PST. This is an automated message from the Apache Git Service. T

[jira] [Updated] (HUDI-1514) Avoid raw type use for parameter of interface

2021-01-08 Thread puyvqi (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] puyvqi updated HUDI-1514: - Status: In Progress (was: Open) > Avoid raw type use for parameter of interface > -

[GitHub] [hudi] loukey-lj opened a new pull request #2419: Hudi 1421

2021-01-08 Thread GitBox
loukey-lj opened a new pull request #2419: URL: https://github.com/apache/hudi/pull/2419 Improvement of failure recovery for HoodieFlinkStreamer. When HoodieFlinkStreamer failure recovery from a checkpoint or savepoint KeyedWriteProcessOperator will get an instant time from flink sta

[GitHub] [hudi] codecov-io edited a comment on pull request #2375: [HUDI-1332] Introduce FlinkHoodieBloomIndex to hudi-flink-client

2021-01-08 Thread GitBox
codecov-io edited a comment on pull request #2375: URL: https://github.com/apache/hudi/pull/2375#issuecomment-752011479 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2375?src=pr&el=h1) Report > Merging [#2375](https://codecov.io/gh/apache/hudi/pull/2375?src=pr&el=desc) (095fd3d) in