satishkotha commented on a change in pull request #2275:
URL: https://github.com/apache/hudi/pull/2275#discussion_r533106354
##
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clustering/update/UpdateStrategy.java
##
@@ -0,0 +1,32 @@
+/*
+
zhedoubushishi commented on a change in pull request #2208:
URL: https://github.com/apache/hudi/pull/2208#discussion_r533108221
##
File path:
hudi-spark/src/main/scala/org/apache/hudi/MergeOnReadSnapshotRelation.scala
##
@@ -113,9 +113,6 @@ class MergeOnReadSnapshotRelation(va
codecov-io edited a comment on pull request #2266:
URL: https://github.com/apache/hudi/pull/2266#issuecomment-731104115
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2266?src=pr&el=h1) Report
> Merging
[#2266](https://codecov.io/gh/apache/hudi/pull/2266?src=pr&el=desc) (7f84b12)
in
codecov-io edited a comment on pull request #2266:
URL: https://github.com/apache/hudi/pull/2266#issuecomment-731104115
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2266?src=pr&el=h1) Report
> Merging
[#2266](https://codecov.io/gh/apache/hudi/pull/2266?src=pr&el=desc) (7f84b12)
in
vinothchandar commented on a change in pull request #2275:
URL: https://github.com/apache/hudi/pull/2275#discussion_r533062562
##
File path:
hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/table/action/commit/TestCopyOnWriteActionExecutor.java
##
@@ -456,4 +479,95
vinothchandar commented on pull request #2275:
URL: https://github.com/apache/hudi/pull/2275#issuecomment-736206669
@satishkotha weirdly I still cannot assign this to you. :/
Could you help review this?
This is an au
bithw1 commented on issue #2290:
URL: https://github.com/apache/hudi/issues/2290#issuecomment-736192199
Thanks @nsivabalan. All I am using is apache open sourced, such as spark,
hive, hudi etc. Is open sourced hudi capable of differentiating updates and
deletes? If yes, what should I do to
bithw1 closed issue #2288:
URL: https://github.com/apache/hudi/issues/2288
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the speci
bithw1 commented on issue #2288:
URL: https://github.com/apache/hudi/issues/2288#issuecomment-736181454
Thanks @bvaradar for the good explanation.
This is an automated message from the Apache Git Service.
To respond to the m
nsivabalan commented on issue #2290:
URL: https://github.com/apache/hudi/issues/2290#issuecomment-736180816
yes, just set the operation type to "UPSERT". hudi is capable of
differentiating updates and deletes. I guess you are using AWSDmsAvroPayload as
payload class. It should work. Let us
bithw1 commented on issue #2290:
URL: https://github.com/apache/hudi/issues/2290#issuecomment-736173244
@nsivabalan please help take a look,thanks!
This is an automated message from the Apache Git Service.
To respond to the m
[
https://issues.apache.org/jira/browse/HUDI-1276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
satish updated HUDI-1276:
-
Priority: Blocker (was: Major)
> delete replaced file groups during clean
> -
[
https://issues.apache.org/jira/browse/HUDI-1353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
satish updated HUDI-1353:
-
Priority: Blocker (was: Major)
> Incremental timeline support for pending clustering operations
> ---
[
https://issues.apache.org/jira/browse/HUDI-1352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
satish closed HUDI-1352.
> Add FileSystemView API to query pending clustering operations
> --
[
https://issues.apache.org/jira/browse/HUDI-1352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
satish resolved HUDI-1352.
--
Resolution: Fixed
> Add FileSystemView API to query pending clustering operations
>
[
https://issues.apache.org/jira/browse/HUDI-1352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
satish updated HUDI-1352:
-
Status: Open (was: New)
> Add FileSystemView API to query pending clustering operations
> ---
satishkotha commented on pull request #2263:
URL: https://github.com/apache/hudi/pull/2263#issuecomment-735989890
@n3nash Please take another look.
This is an automated message from the Apache Git Service.
To respond to the m
satishkotha commented on a change in pull request #2263:
URL: https://github.com/apache/hudi/pull/2263#discussion_r532841404
##
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/cluster/strategy/ScheduleClusteringStrategy.java
##
@@ -0,0 +1,1
satishkotha commented on a change in pull request #2263:
URL: https://github.com/apache/hudi/pull/2263#discussion_r532841241
##
File path:
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/execution/bulkinsert/RDDCustomColumnsSortPartitioner.java
##
@@ -0,0 +1,66 @@
satishkotha commented on a change in pull request #2263:
URL: https://github.com/apache/hudi/pull/2263#discussion_r532840414
##
File path:
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/clustering/run/SparkBulkInsertBasedRunClusteringStrategy.java
##
@@ -0,
satishkotha commented on a change in pull request #2263:
URL: https://github.com/apache/hudi/pull/2263#discussion_r532840414
##
File path:
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/clustering/run/SparkBulkInsertBasedRunClusteringStrategy.java
##
@@ -0,
satishkotha commented on a change in pull request #2263:
URL: https://github.com/apache/hudi/pull/2263#discussion_r532839804
##
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/cluster/strategy/PartitionAwareScheduleClusteringStrategy.java
#
satishkotha commented on a change in pull request #2263:
URL: https://github.com/apache/hudi/pull/2263#discussion_r532839521
##
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieClusteringConfig.java
##
@@ -0,0 +1,155 @@
+/*
+ * Licensed to t
codecov-io edited a comment on pull request #2263:
URL: https://github.com/apache/hudi/pull/2263#issuecomment-730664726
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2263?src=pr&el=h1) Report
> Merging
[#2263](https://codecov.io/gh/apache/hudi/pull/2263?src=pr&el=desc) (6e8ea21)
in
codecov-io edited a comment on pull request #2263:
URL: https://github.com/apache/hudi/pull/2263#issuecomment-730664726
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2263?src=pr&el=h1) Report
> Merging
[#2263](https://codecov.io/gh/apache/hudi/pull/2263?src=pr&el=desc) (6e8ea21)
in
codecov-io edited a comment on pull request #2263:
URL: https://github.com/apache/hudi/pull/2263#issuecomment-730664726
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2263?src=pr&el=h1) Report
> Merging
[#2263](https://codecov.io/gh/apache/hudi/pull/2263?src=pr&el=desc) (6e8ea21)
in
satishkotha commented on a change in pull request #2263:
URL: https://github.com/apache/hudi/pull/2263#discussion_r532829345
##
File path:
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/cluster/SparkRunClusteringCommitActionExecutor.java
##
@@ -0,0 +1
prashantwason commented on a change in pull request #2266:
URL: https://github.com/apache/hudi/pull/2266#discussion_r532827780
##
File path:
hudi-client/src/main/java/org/apache/hudi/metadata/FSBackedTableMetadataWriter.java
##
@@ -74,84 +60,109 @@
import org.apache.hudi.exce
satishkotha commented on a change in pull request #2263:
URL: https://github.com/apache/hudi/pull/2263#discussion_r532824235
##
File path:
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/cluster/SparkRunClusteringCommitActionExecutor.java
##
@@ -0,0 +1
satishkotha commented on a change in pull request #2263:
URL: https://github.com/apache/hudi/pull/2263#discussion_r532821349
##
File path:
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/cluster/SparkScheduleClusteringActionExecutor.java
##
@@ -0,0 +1,
vinothchandar commented on a change in pull request #2266:
URL: https://github.com/apache/hudi/pull/2266#discussion_r532821022
##
File path:
hudi-client/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataWriter.java
##
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Sof
vinothchandar commented on a change in pull request #2266:
URL: https://github.com/apache/hudi/pull/2266#discussion_r532820609
##
File path:
hudi-client/src/main/java/org/apache/hudi/metadata/FSBackedTableMetadataWriter.java
##
@@ -74,84 +60,109 @@
import org.apache.hudi.exce
satishkotha commented on a change in pull request #2263:
URL: https://github.com/apache/hudi/pull/2263#discussion_r532819893
##
File path:
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/cluster/SparkRunClusteringCommitActionExecutor.java
##
@@ -0,0 +1
satishkotha commented on a change in pull request #2263:
URL: https://github.com/apache/hudi/pull/2263#discussion_r532819029
##
File path:
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/cluster/SparkRunClusteringCommitActionExecutor.java
##
@@ -0,0 +1
satishkotha commented on a change in pull request #2263:
URL: https://github.com/apache/hudi/pull/2263#discussion_r532818631
##
File path:
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/execution/SparkLazyInsertIterable.java
##
@@ -34,14 +35,18 @@
public class
satishkotha commented on a change in pull request #2263:
URL: https://github.com/apache/hudi/pull/2263#discussion_r532817583
##
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/cluster/strategy/ScheduleClusteringStrategy.java
##
@@ -0,0 +1,1
prashantwason commented on a change in pull request #2266:
URL: https://github.com/apache/hudi/pull/2266#discussion_r532815405
##
File path:
hudi-client/src/main/java/org/apache/hudi/metadata/FSBackedTableMetadataWriter.java
##
@@ -74,84 +60,109 @@
import org.apache.hudi.exce
satishkotha commented on a change in pull request #2263:
URL: https://github.com/apache/hudi/pull/2263#discussion_r532815506
##
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/cluster/strategy/RunClusteringStrategy.java
##
@@ -0,0 +1,67 @@
satishkotha commented on a change in pull request #2263:
URL: https://github.com/apache/hudi/pull/2263#discussion_r532813640
##
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/cluster/strategy/PartitionAwareScheduleClusteringStrategy.java
#
satishkotha commented on a change in pull request #2263:
URL: https://github.com/apache/hudi/pull/2263#discussion_r532813395
##
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTable.java
##
@@ -326,6 +327,28 @@ public HoodieActiveTimeline ge
satishkotha commented on a change in pull request #2263:
URL: https://github.com/apache/hudi/pull/2263#discussion_r532813114
##
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieClusteringConfig.java
##
@@ -0,0 +1,155 @@
+/*
+ * Licensed to t
satishkotha commented on a change in pull request #2263:
URL: https://github.com/apache/hudi/pull/2263#discussion_r532812941
##
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieClusteringConfig.java
##
@@ -0,0 +1,155 @@
+/*
+ * Licensed to t
satishkotha commented on a change in pull request #2263:
URL: https://github.com/apache/hudi/pull/2263#discussion_r532812266
##
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/AbstractHoodieWriteClient.java
##
@@ -726,6 +751,54 @@ private void ro
prashantwason commented on pull request #2216:
URL: https://github.com/apache/hudi/pull/2216#issuecomment-735950933
@vinothchandar I have removed the oldNumWrites field.
This is an automated message from the Apache Git Servic
vinothchandar commented on issue #2280:
URL: https://github.com/apache/hudi/issues/2280#issuecomment-735945530
Looks like balaji did beat me to it. :)
This is an automated message from the Apache Git Service.
To respond to
bvaradar commented on issue #2290:
URL: https://github.com/apache/hudi/issues/2290#issuecomment-735940873
@nsivabalan : Can you take a look at this ?
This is an automated message from the Apache Git Service.
To respond to the
bvaradar commented on issue #2288:
URL: https://github.com/apache/hudi/issues/2288#issuecomment-735939031
Yes, these are metadata files used to track the status of operations and is
needed to perform rollbacks if needed. Instead of keeping one file, Hudi tracks
them in separate files to av
bvaradar commented on issue #2284:
URL: https://github.com/apache/hudi/issues/2284#issuecomment-735937171
Hudi provides custom merging semantics. You can plugin your own payload
implementation that instead of overwriting, can have custom merging logic
(HoodieRecordPayload.java). Can you ex
bvaradar edited a comment on issue #2282:
URL: https://github.com/apache/hudi/issues/2282#issuecomment-735934622
It looks like the error is happening during loading the data at
hdfs://nameservice/data/wdt/sqoop/cow/inc/stockout_order_20201125/837b6714-40b3-4a00-bcf5-97a6f33d2af7.parquet
bvaradar commented on issue #2282:
URL: https://github.com/apache/hudi/issues/2282#issuecomment-735934622
It looks like the error is happening during loading the data at
hdfs://nameservice/data/wdt/sqoop/cow/inc/stockout_order_20201125/837b6714-40b3-4a00-bcf5-97a6f33d2af7.parquet
Can
bvaradar commented on issue #2280:
URL: https://github.com/apache/hudi/issues/2280#issuecomment-735927848
@ygordefraga : This could be coming from the increase in the number of
partitions.
This could be related to
https://github.com/apache/hudi/issues/2269#issuecomment-733299492
bvaradar commented on issue #2269:
URL: https://github.com/apache/hudi/issues/2269#issuecomment-735921130
@asharma4-lucid : Sorry for the delay in responding due to Thanksgiving
weekend. It looks like cleaning is the one taking long time. Cleaner (in 0.6)
runs in incremental mode by defau
[
https://issues.apache.org/jira/browse/HUDI-1426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alessio Cuzzocrea updated HUDI-1426:
Description:
In hudi 0.6.0 I've noticed a minor typo in class declaration:
{code:java}
org.
Alessio Cuzzocrea created HUDI-1426:
---
Summary: Typo in class declaration
Key: HUDI-1426
URL: https://issues.apache.org/jira/browse/HUDI-1426
Project: Apache Hudi
Issue Type: Bug
C
codecov-io edited a comment on pull request #2283:
URL: https://github.com/apache/hudi/pull/2283#issuecomment-734137301
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitH
codecov-io edited a comment on pull request #2283:
URL: https://github.com/apache/hudi/pull/2283#issuecomment-734137301
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2283?src=pr&el=h1) Report
> Merging
[#2283](https://codecov.io/gh/apache/hudi/pull/2283?src=pr&el=desc) (2038dda)
in
This is an automated email from the ASF dual-hosted git repository.
leesf pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/master by this push:
new 36ce5bc [HUDI-1424] Write Type changed to BULK_IN
leesf merged pull request #2289:
URL: https://github.com/apache/hudi/pull/2289
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the s
codecov-io edited a comment on pull request #2289:
URL: https://github.com/apache/hudi/pull/2289#issuecomment-735805371
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitH
codecov-io commented on pull request #2289:
URL: https://github.com/apache/hudi/pull/2289#issuecomment-735805371
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2289?src=pr&el=h1) Report
> Merging
[#2289](https://codecov.io/gh/apache/hudi/pull/2289?src=pr&el=desc) (d4fb269)
into
[ma
[
https://issues.apache.org/jira/browse/HUDI-1425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
pengzhiwei reassigned HUDI-1425:
Assignee: pengzhiwei
> Performance loss with the additional hoodieRecords.isEmpty() in
> HoodieSpa
pengzhiwei created HUDI-1425:
Summary: Performance loss with the additional
hoodieRecords.isEmpty() in HoodieSparkSqlWriter#write
Key: HUDI-1425
URL: https://issues.apache.org/jira/browse/HUDI-1425
Projec
bithw1 opened a new issue #2290:
URL: https://github.com/apache/hudi/issues/2290
Hi,
I have created a spark dataframe using the data from the upstream source.
The data contains records that should be Delete and Insert/Update to the hudi
table.(the record has the flag D/U/I)
W
[
https://issues.apache.org/jira/browse/HUDI-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-1424:
-
Labels: pull-request-available (was: )
> Write Type changed to BULK_INSERT when set ENABLE_ROW_WR
pengzhiwei2018 opened a new pull request #2289:
URL: https://github.com/apache/hudi/pull/2289
…ITER_OPT_KEY=true
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contributing.html before opening a
pull request.*
[
https://issues.apache.org/jira/browse/HUDI-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
pengzhiwei reassigned HUDI-1424:
Assignee: pengzhiwei
> Write Type changed to BULK_INSERT when set ENABLE_ROW_WRITER_OPT_KEY=true
>
pengzhiwei created HUDI-1424:
Summary: Write Type changed to BULK_INSERT when set
ENABLE_ROW_WRITER_OPT_KEY=true
Key: HUDI-1424
URL: https://issues.apache.org/jira/browse/HUDI-1424
Project: Apache Hudi
hughfdjackson commented on issue #2265:
URL: https://github.com/apache/hudi/issues/2265#issuecomment-735629366
Hi @vingov, @umehrot2 , @zhedoubushishi - did you find anything that might
shed some light on the above issue?
T
68 matches
Mail list logo
