[
https://issues.apache.org/jira/browse/HUDI-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nishith Agarwal reassigned HUDI-1509:
-
Assignee: Nishith Agarwal
> Major performance degradation due to rewriting records with d
codecov-io commented on pull request #2421:
URL: https://github.com/apache/hudi/pull/2421#issuecomment-757112911
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2421?src=pr&el=h1) Report
> Merging
[#2421](https://codecov.io/gh/apache/hudi/pull/2421?src=pr&el=desc) (0227058)
into
[ma
satishkotha commented on pull request #2422:
URL: https://github.com/apache/hudi/pull/2422#issuecomment-757110216
> @satishkotha is driving this to completion. @prashantwason you might want
to chime in on the algorithm part and let us know if you see red flags, when
you get a chance.
>
n3nash commented on a change in pull request #2379:
URL: https://github.com/apache/hudi/pull/2379#discussion_r554302023
##
File path:
hudi-utilities/src/test/java/org/apache/hudi/utilities/functional/TestHoodieDeltaStreamer.java
##
@@ -672,14 +681,72 @@ public void testInlineC
n3nash commented on a change in pull request #2379:
URL: https://github.com/apache/hudi/pull/2379#discussion_r554302023
##
File path:
hudi-utilities/src/test/java/org/apache/hudi/utilities/functional/TestHoodieDeltaStreamer.java
##
@@ -672,14 +681,72 @@ public void testInlineC
n3nash commented on a change in pull request #2379:
URL: https://github.com/apache/hudi/pull/2379#discussion_r554302023
##
File path:
hudi-utilities/src/test/java/org/apache/hudi/utilities/functional/TestHoodieDeltaStreamer.java
##
@@ -672,14 +681,72 @@ public void testInlineC
n3nash commented on a change in pull request #2379:
URL: https://github.com/apache/hudi/pull/2379#discussion_r554301665
##
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieClusteringJob.java
##
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Fo
n3nash commented on a change in pull request #2379:
URL: https://github.com/apache/hudi/pull/2379#discussion_r554301213
##
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/AbstractHoodieWriteClient.java
##
@@ -711,12 +711,30 @@ public void rollbac
vinothchandar commented on pull request #1938:
URL: https://github.com/apache/hudi/pull/1938#issuecomment-757103037
Thanks @garyli1019 ! You are awesome.
Will take a pass, after CI is happy. and land,
This is an auto
wangxianghu commented on pull request #2419:
URL: https://github.com/apache/hudi/pull/2419#issuecomment-757100778
Hi @loukey-lj thanks for your contribution
please check why ci failed
This is an automated message from the
garyli1019 commented on pull request #1938:
URL: https://github.com/apache/hudi/pull/1938#issuecomment-757097457
@vinothchandar addressed comments. You are right, the inconsistency was
caused by the test data generator. The CI was happy after I changed from
`generateUpdates` to `generateUn
garyli1019 commented on a change in pull request #1938:
URL: https://github.com/apache/hudi/pull/1938#discussion_r554291761
##
File path:
hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieInputFormatUtils.java
##
@@ -470,4 +471,45 @@ private static HoodieBaseFile
garyli1019 commented on a change in pull request #1938:
URL: https://github.com/apache/hudi/pull/1938#discussion_r554291505
##
File path:
hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieInputFormatUtils.java
##
@@ -470,4 +471,45 @@ private static HoodieBaseFile
garyli1019 commented on a change in pull request #1938:
URL: https://github.com/apache/hudi/pull/1938#discussion_r554291421
##
File path:
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/MergeOnReadIncrementalRelation.scala
##
@@ -0,0 +1,218 @@
+/*
+ * Licensed
garyli1019 commented on a change in pull request #1938:
URL: https://github.com/apache/hudi/pull/1938#discussion_r554291307
##
File path:
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/MergeOnReadIncrementalRelation.scala
##
@@ -0,0 +1,218 @@
+/*
+ * Licensed
garyli1019 commented on a change in pull request #1938:
URL: https://github.com/apache/hudi/pull/1938#discussion_r554287262
##
File path:
hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieInputFormatUtils.java
##
@@ -470,4 +471,45 @@ private static HoodieBaseFile
vinothchandar commented on pull request #2379:
URL: https://github.com/apache/hudi/pull/2379#issuecomment-757090817
@n3nash can you shepherd this and land ? @satishkotha is helping out with
the other PR.
This is an auto
vinothchandar commented on a change in pull request #2421:
URL: https://github.com/apache/hudi/pull/2421#discussion_r554264580
##
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieAppendHandle.java
##
@@ -164,6 +164,7 @@ private void init(HoodieR
umehrot2 commented on a change in pull request #2417:
URL: https://github.com/apache/hudi/pull/2417#discussion_r554278832
##
File path:
hudi-common/src/main/java/org/apache/hudi/metadata/FileSystemBackedTableMetadata.java
##
@@ -49,12 +60,48 @@ public
FileSystemBackedTableMet
umehrot2 commented on a change in pull request #2417:
URL: https://github.com/apache/hudi/pull/2417#discussion_r554270231
##
File path:
hudi-common/src/main/java/org/apache/hudi/common/engine/HoodieMREngineContext.java
##
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Softwar
vinothchandar commented on a change in pull request #2417:
URL: https://github.com/apache/hudi/pull/2417#discussion_r554260296
##
File path:
hudi-common/src/main/java/org/apache/hudi/common/engine/HoodieMREngineContext.java
##
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache So
vinothchandar commented on a change in pull request #2417:
URL: https://github.com/apache/hudi/pull/2417#discussion_r554259024
##
File path:
hudi-common/src/main/java/org/apache/hudi/metadata/FileSystemBackedTableMetadata.java
##
@@ -49,12 +60,48 @@ public
FileSystemBackedTab
vinothchandar commented on pull request #2422:
URL: https://github.com/apache/hudi/pull/2422#issuecomment-757058068
@satishkotha is driving this to completion. @prashantwason you might want to
chime in on the algorithm part and let us know if you see red flags, when you
get a chance.
vinothchandar commented on pull request #2422:
URL: https://github.com/apache/hudi/pull/2422#issuecomment-757056595
We should probably keep the archival code as a defense, with a `log.warn()`
This is an automated message from
This is an automated email from the ASF dual-hosted git repository.
vinoyang pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git.
from 17df517 [HUDI-1510] Move HoodieEngineContext and its dependencies to
hudi-common (#2410)
add c151147 [MINOR]
yanghua merged pull request #2415:
URL: https://github.com/apache/hudi/pull/2415
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
satishkotha commented on a change in pull request #2422:
URL: https://github.com/apache/hudi/pull/2422#discussion_r554244115
##
File path:
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java
##
@@ -92,13 +95,36 @@
case HoodieTimeline.SAVEPOIN
umehrot2 commented on a change in pull request #2417:
URL: https://github.com/apache/hudi/pull/2417#discussion_r554245704
##
File path:
hudi-common/src/main/java/org/apache/hudi/common/table/view/FileSystemViewManager.java
##
@@ -159,12 +160,12 @@ private static HoodieTableFil
umehrot2 commented on a change in pull request #2417:
URL: https://github.com/apache/hudi/pull/2417#discussion_r554244342
##
File path:
hudi-common/src/main/java/org/apache/hudi/metadata/FileSystemBackedTableMetadata.java
##
@@ -64,6 +111,6 @@ public
FileSystemBackedTableMeta
umehrot2 commented on a change in pull request #2417:
URL: https://github.com/apache/hudi/pull/2417#discussion_r554243274
##
File path:
hudi-common/src/main/java/org/apache/hudi/common/table/view/FileSystemViewManager.java
##
@@ -159,12 +160,12 @@ private static HoodieTableFil
umehrot2 commented on a change in pull request #2417:
URL: https://github.com/apache/hudi/pull/2417#discussion_r554243165
##
File path:
hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieRealtimeInputFormatUtils.java
##
@@ -82,7 +82,7 @@
HoodieTableMetaClie
umehrot2 commented on a change in pull request #2417:
URL: https://github.com/apache/hudi/pull/2417#discussion_r554242901
##
File path:
hudi-common/src/main/java/org/apache/hudi/metadata/FileSystemBackedTableMetadata.java
##
@@ -49,12 +60,48 @@ public
FileSystemBackedTableMet
umehrot2 commented on a change in pull request #2417:
URL: https://github.com/apache/hudi/pull/2417#discussion_r554242390
##
File path:
hudi-common/src/main/java/org/apache/hudi/metadata/FileSystemBackedTableMetadata.java
##
@@ -49,12 +60,48 @@ public
FileSystemBackedTableMet
umehrot2 commented on a change in pull request #2417:
URL: https://github.com/apache/hudi/pull/2417#discussion_r554241357
##
File path:
hudi-common/src/main/java/org/apache/hudi/common/engine/HoodieMREngineContext.java
##
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Softwar
umehrot2 commented on a change in pull request #2417:
URL: https://github.com/apache/hudi/pull/2417#discussion_r554237778
##
File path:
hudi-common/src/main/java/org/apache/hudi/common/config/HoodieMetadataConfig.java
##
@@ -39,6 +39,7 @@
// Validate contents of Metadata Ta
umehrot2 commented on a change in pull request #2417:
URL: https://github.com/apache/hudi/pull/2417#discussion_r554237558
##
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java
##
@@ -221,8 +221,9 @@ public Hood
prashantwason commented on a change in pull request #2417:
URL: https://github.com/apache/hudi/pull/2417#discussion_r554237450
##
File path:
hudi-common/src/main/java/org/apache/hudi/metadata/FileSystemBackedTableMetadata.java
##
@@ -49,12 +60,48 @@ public
FileSystemBackedTab
vinothchandar commented on pull request #2422:
URL: https://github.com/apache/hudi/pull/2422#issuecomment-757038688
cc @prashantwason @satishkotha kindly take a quick pass.
I still have to - add tests, handle the existing deletion code in archival.
but the algorithm is there in th
[
https://issues.apache.org/jira/browse/HUDI-1276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-1276:
-
Labels: pull-request-available (was: )
> delete replaced file groups during clean
> -
vinothchandar opened a new pull request #2422:
URL: https://github.com/apache/hudi/pull/2422
…ble be compatible
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contributing.html before opening a
pull request.*
#
[
https://issues.apache.org/jira/browse/HUDI-1276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17261638#comment-17261638
]
Vinoth Chandar commented on HUDI-1276:
--
Case 1: So, we can only restore within the co
bvaradar commented on issue #2409:
URL: https://github.com/apache/hudi/issues/2409#issuecomment-757035695
@wosow : The _rt table syncing happens after _ro table and I see an
HiveMetaStore exception when updating commit time in the _ro table saying that
the table does not exist. This is wei
Rap70r closed issue #2416:
URL: https://github.com/apache/hudi/issues/2416
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the speci
prashanthvg89 commented on pull request #2069:
URL: https://github.com/apache/hudi/pull/2069#issuecomment-756969267
I saw this failure in Spark streaming job writing to Hudi. Is this still
being worked on and is there an ETA for this?
Hudi version: 0.6.0
Spark version: 2.4.0
EM
codecov-io commented on pull request #2420:
URL: https://github.com/apache/hudi/pull/2420#issuecomment-756960638
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2420?src=pr&el=h1) Report
> Merging
[#2420](https://codecov.io/gh/apache/hudi/pull/2420?src=pr&el=desc) (fb35a0f)
into
[ma
[
https://issues.apache.org/jira/browse/HUDI-1276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17261540#comment-17261540
]
Prashant Wason commented on HUDI-1276:
--
One more think to verify is how the restore w
[
https://issues.apache.org/jira/browse/HUDI-1276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17261508#comment-17261508
]
Vinoth Chandar commented on HUDI-1276:
--
>But, probably we can ask users to KEEP_LATES
[
https://issues.apache.org/jira/browse/HUDI-1276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinoth Chandar reassigned HUDI-1276:
Assignee: Vinoth Chandar (was: satish)
> delete replaced file groups during clean
> --
[
https://issues.apache.org/jira/browse/HUDI-1276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinoth Chandar updated HUDI-1276:
-
Status: Open (was: New)
> delete replaced file groups during clean
>
[
https://issues.apache.org/jira/browse/HUDI-1276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17261507#comment-17261507
]
satish commented on HUDI-1276:
--
[~vinoth] high level looks reasonable. Few edge cases below.
vinothchandar commented on a change in pull request #2421:
URL: https://github.com/apache/hudi/pull/2421#discussion_r554079488
##
File path:
hudi-common/src/test/java/org/apache/hudi/common/table/TestTimelineUtils.java
##
@@ -181,10 +187,113 @@ public void testRestoreInstants(
[
https://issues.apache.org/jira/browse/HUDI-1276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17261437#comment-17261437
]
Vinoth Chandar commented on HUDI-1276:
--
[~satishkotha] Dumping my thoughts here. I sp
nsivabalan commented on a change in pull request #2421:
URL: https://github.com/apache/hudi/pull/2421#discussion_r554055131
##
File path:
hudi-common/src/test/java/org/apache/hudi/common/table/TestTimelineUtils.java
##
@@ -181,10 +187,113 @@ public void testRestoreInstants() t
nsivabalan opened a new pull request #2421:
URL: https://github.com/apache/hudi/pull/2421
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contributing.html before opening a
pull request.*
## What is the purpose of the
codecov-io edited a comment on pull request #2415:
URL: https://github.com/apache/hudi/pull/2415#issuecomment-756825549
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2415?src=pr&el=h1) Report
> Merging
[#2415](https://codecov.io/gh/apache/hudi/pull/2415?src=pr&el=desc) (1dd00dd)
in
codecov-io edited a comment on pull request #2415:
URL: https://github.com/apache/hudi/pull/2415#issuecomment-756825549
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2415?src=pr&el=h1) Report
> Merging
[#2415](https://codecov.io/gh/apache/hudi/pull/2415?src=pr&el=desc) (1dd00dd)
in
lw309637554 commented on a change in pull request #2417:
URL: https://github.com/apache/hudi/pull/2417#discussion_r554041498
##
File path:
hudi-common/src/main/java/org/apache/hudi/common/table/view/FileSystemViewManager.java
##
@@ -159,12 +160,12 @@ private static HoodieTable
lw309637554 edited a comment on pull request #2418:
URL: https://github.com/apache/hudi/pull/2418#issuecomment-756830958
@satishkotha just two minor comments
This is an automated message from the Apache Git Service.
To respon
[
https://issues.apache.org/jira/browse/HUDI-1514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
puyvqi updated HUDI-1514:
-
Status: Patch Available (was: In Progress)
> Avoid raw type use for parameter of interface
> --
lw309637554 commented on pull request #2418:
URL: https://github.com/apache/hudi/pull/2418#issuecomment-756830958
@satishkotha just two minor comment
This is an automated message from the Apache Git Service.
To respond to the
lw309637554 commented on a change in pull request #2418:
URL: https://github.com/apache/hudi/pull/2418#discussion_r554025924
##
File path:
hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/table/action/rollback/HoodieClientRollbackTestBase.java
##
@@ -96,4 +99,61 @@
lw309637554 commented on a change in pull request #2418:
URL: https://github.com/apache/hudi/pull/2418#discussion_r554024960
##
File path:
hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/table/action/rollback/HoodieClientRollbackTestBase.java
##
@@ -96,4 +99,61 @@
[
https://issues.apache.org/jira/browse/HUDI-1514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-1514:
-
Labels: pull-request-available (was: )
> Avoid raw type use for parameter of interface
> --
puyvqi opened a new pull request #2420:
URL: https://github.com/apache/hudi/pull/2420
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contributing.html before opening a
pull request.*
## What is the purpose of the pull
codecov-io commented on pull request #2415:
URL: https://github.com/apache/hudi/pull/2415#issuecomment-756825549
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2415?src=pr&el=h1) Report
> Merging
[#2415](https://codecov.io/gh/apache/hudi/pull/2415?src=pr&el=desc) (1dd00dd)
into
[ma
Rap70r commented on issue #2416:
URL: https://github.com/apache/hudi/issues/2416#issuecomment-756818455
Hello vinothchandar,
Thank you for getting back to me. Yes, I would like to use Hudi with EMR
6.2.0 and Spark 3.0.
Thank you
--
garyli1019 commented on pull request #1938:
URL: https://github.com/apache/hudi/pull/1938#issuecomment-756781554
Thanks, will update this PR before Friday night PST.
This is an automated message from the Apache Git Service.
T
[
https://issues.apache.org/jira/browse/HUDI-1514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
puyvqi updated HUDI-1514:
-
Status: In Progress (was: Open)
> Avoid raw type use for parameter of interface
> -
loukey-lj opened a new pull request #2419:
URL: https://github.com/apache/hudi/pull/2419
Improvement of failure recovery for HoodieFlinkStreamer.
When HoodieFlinkStreamer failure recovery from a checkpoint or savepoint
KeyedWriteProcessOperator will get an instant time from flink sta
codecov-io edited a comment on pull request #2375:
URL: https://github.com/apache/hudi/pull/2375#issuecomment-752011479
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2375?src=pr&el=h1) Report
> Merging
[#2375](https://codecov.io/gh/apache/hudi/pull/2375?src=pr&el=desc) (095fd3d)
in
70 matches
Mail list logo