KnightChess closed pull request #9158: [MINOR] Unpersist only relevent metadata
table RDDs
URL: https://github.com/apache/hudi/pull/9158
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
KnightChess opened a new pull request, #9158:
URL: https://github.com/apache/hudi/pull/9158
### Change Logs
#7914 only remove base table cached rdd, if open mdt, mdt cached rdd need
unpersist too.
### Impact
None
### Risk level (write none, low medium or high
hudi-bot commented on PR #9157:
URL: https://github.com/apache/hudi/pull/9157#issuecomment-1628279986
## CI report:
* d3a74b072d6c1d55f5412946aa4ba75a8c54076a Azure:
hudi-bot commented on PR #8837:
URL: https://github.com/apache/hudi/pull/8837#issuecomment-1628105901
## CI report:
* debb29aa63e665eed1137973fbe94eee395dc286 Azure:
danny0405 commented on code in PR #8054:
URL: https://github.com/apache/hudi/pull/8054#discussion_r1257693948
##
hudi-sync/hudi-sync-common/src/main/java/org/apache/hudi/sync/common/util/SparkDataSourceTableUtils.java:
##
@@ -34,13 +35,21 @@
import static
danny0405 commented on code in PR #8054:
URL: https://github.com/apache/hudi/pull/8054#discussion_r1257693688
##
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/catalog/HiveSchemaUtils.java:
##
@@ -176,45 +177,6 @@ private static DataType
hudi-bot commented on PR #8837:
URL: https://github.com/apache/hudi/pull/8837#issuecomment-1628092201
## CI report:
* debb29aa63e665eed1137973fbe94eee395dc286 Azure:
danny0405 commented on code in PR #9133:
URL: https://github.com/apache/hudi/pull/9133#discussion_r1257689584
##
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/format/CastMap.java:
##
@@ -165,21 +192,132 @@ void add(int pos, LogicalType fromType,
danny0405 commented on code in PR #9133:
URL: https://github.com/apache/hudi/pull/9133#discussion_r1255288699
##
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/format/CastMap.java:
##
@@ -165,21 +192,132 @@ void add(int pos, LogicalType fromType,
danny0405 commented on PR #9118:
URL: https://github.com/apache/hudi/pull/9118#issuecomment-1628084953
Thanks for the contribution @stream2000 , can we add list a table of metrics
we support here, illustrating the usage of each metrics, and then we are clear
whether it deserves to
danny0405 commented on PR #9156:
URL: https://github.com/apache/hudi/pull/9156#issuecomment-1628072273
> because we can make sure that the partition records is sorted.
In the near future, the non-sorted partition code may be introduced, we can
keep it for extensibility now I think.
danny0405 commented on code in PR #9126:
URL: https://github.com/apache/hudi/pull/9126#discussion_r1257684369
##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/compact/strategy/DayBasedAndBoundedIOCompactionStrategy.java:
##
@@ -0,0 +1,71 @@
+/*
+ *
danny0405 commented on issue #9143:
URL: https://github.com/apache/hudi/issues/9143#issuecomment-1628055348
cc @ad1happy2go seems this issue has been fixed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
stream2000 commented on PR #9156:
URL: https://github.com/apache/hudi/pull/9156#issuecomment-1628043527
>
[6515.patch.zip](https://github.com/apache/hudi/files/11997469/6515.patch.zip)
Nice catch, does this fix makes sense to you?
Yes this fix is ok and make less modification. I
danny0405 commented on PR #9156:
URL: https://github.com/apache/hudi/pull/9156#issuecomment-1628023795
[6515.patch.zip](https://github.com/apache/hudi/files/11997469/6515.patch.zip)
Nice catch, does this fix makes sense to you?
--
This is an automated message from the Apache Git
This is an automated email from the ASF dual-hosted git repository.
codope pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/master by this push:
new a0ca3eadba5 [HUDI-6316] Adding corrupted and
codope merged PR #8881:
URL: https://github.com/apache/hudi/pull/8881
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail:
hudi-bot commented on PR #9157:
URL: https://github.com/apache/hudi/pull/9157#issuecomment-1628014896
## CI report:
* d3a74b072d6c1d55f5412946aa4ba75a8c54076a Azure:
danny0405 commented on issue #9138:
URL: https://github.com/apache/hudi/issues/9138#issuecomment-1628004963
Seems an issue of privilege, did you try to grant the permission with cmd:
`grant xxx_action on database xxx_database to user xxx_user;`
--
This is an automated message from the
hudi-bot commented on PR #9157:
URL: https://github.com/apache/hudi/pull/9157#issuecomment-1628004052
## CI report:
* d3a74b072d6c1d55f5412946aa4ba75a8c54076a UNKNOWN
Bot commands
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the
hudi-bot commented on PR #8697:
URL: https://github.com/apache/hudi/pull/8697#issuecomment-1628003363
## CI report:
* 1fc455198964e104536b807792dde341c960de88 Azure:
danny0405 commented on issue #9154:
URL: https://github.com/apache/hudi/issues/9154#issuecomment-1628001354
Did you configure the spillable map path explicitly? Wondering if it is
related with the disk space or something.
--
This is an automated message from the Apache Git Service.
To
hudi-bot commented on PR #8881:
URL: https://github.com/apache/hudi/pull/8881#issuecomment-1627996896
## CI report:
* aa126962099394261801fe5fbf679bd192033f80 Azure:
danny0405 commented on code in PR #9123:
URL: https://github.com/apache/hudi/pull/9123#discussion_r1257660429
##
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala:
##
@@ -1094,6 +1094,11 @@ object HoodieSparkSqlWriter {
if
danny0405 commented on code in PR #9153:
URL: https://github.com/apache/hudi/pull/9153#discussion_r1257656038
##
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/SparkRDDTableServiceClient.java:
##
@@ -291,6 +287,13 @@ private void
nsivabalan opened a new pull request, #9157:
URL: https://github.com/apache/hudi/pull/9157
### Change Logs
Adding tests to validate restore of compaction commit
### Impact
Adding tests to validate restore of compaction commit
### Risk level (write none, low medium
hudi-bot commented on PR #9123:
URL: https://github.com/apache/hudi/pull/9123#issuecomment-1627893231
## CI report:
* beb523cca98b4b62964c485e464d5eb1ce6e25a2 Azure:
hudi-bot commented on PR #8697:
URL: https://github.com/apache/hudi/pull/8697#issuecomment-1627870285
## CI report:
* 6250cd0f2bfe2ba9c3b3053940f2a75be78c2f98 Azure:
hudi-bot commented on PR #8697:
URL: https://github.com/apache/hudi/pull/8697#issuecomment-1627867274
## CI report:
* 6250cd0f2bfe2ba9c3b3053940f2a75be78c2f98 Azure:
nsivabalan commented on code in PR #8697:
URL: https://github.com/apache/hudi/pull/8697#discussion_r1257563336
##
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/ProvidesHoodieConfig.scala:
##
@@ -142,24 +144,27 @@ trait ProvidesHoodieConfig
hudi-bot commented on PR #9123:
URL: https://github.com/apache/hudi/pull/9123#issuecomment-1627855511
## CI report:
* 7708ff75ba467e2156b6396ee2886ec645b7b44f Azure:
hudi-bot commented on PR #8881:
URL: https://github.com/apache/hudi/pull/8881#issuecomment-1627855419
## CI report:
* 7b9d3ca6b52733dc531f54d6bf21ca449cc3254f Azure:
hudi-bot commented on PR #9123:
URL: https://github.com/apache/hudi/pull/9123#issuecomment-1627853487
## CI report:
* 7708ff75ba467e2156b6396ee2886ec645b7b44f Azure:
hudi-bot commented on PR #8881:
URL: https://github.com/apache/hudi/pull/8881#issuecomment-162785
## CI report:
* 7b9d3ca6b52733dc531f54d6bf21ca449cc3254f Azure:
nsivabalan commented on PR #8881:
URL: https://github.com/apache/hudi/pull/8881#issuecomment-1627851978
@codope : done. addressed the feedback
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
nsivabalan commented on PR #9123:
URL: https://github.com/apache/hudi/pull/9123#issuecomment-1627844217
hey @danny0405 @codope : Updated the patch. rebased w/ latest master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub
nsivabalan commented on code in PR #9123:
URL: https://github.com/apache/hudi/pull/9123#discussion_r1257549916
##
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/ProvidesHoodieConfig.scala:
##
@@ -187,29 +218,38 @@ trait ProvidesHoodieConfig
nsivabalan commented on code in PR #9123:
URL: https://github.com/apache/hudi/pull/9123#discussion_r1257548654
##
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala:
##
@@ -1094,6 +1094,11 @@ object HoodieSparkSqlWriter {
if
nsivabalan commented on code in PR #9123:
URL: https://github.com/apache/hudi/pull/9123#discussion_r1257546507
##
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/ProvidesHoodieConfig.scala:
##
@@ -112,6 +113,36 @@ trait ProvidesHoodieConfig
nsivabalan commented on code in PR #9123:
URL: https://github.com/apache/hudi/pull/9123#discussion_r1257546160
##
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/ProvidesHoodieConfig.scala:
##
@@ -187,29 +218,38 @@ trait ProvidesHoodieConfig
nsivabalan commented on code in PR #9123:
URL: https://github.com/apache/hudi/pull/9123#discussion_r1257545983
##
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala:
##
@@ -514,6 +520,29 @@ object DataSourceWriteOptions {
val
This is an automated email from the ASF dual-hosted git repository.
sivabalan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/master by this push:
new 6d3311c34c8 [HUDI-6443] Support
nsivabalan merged PR #9055:
URL: https://github.com/apache/hudi/pull/9055
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail:
huyuanfeng2018 commented on issue #8265:
URL: https://github.com/apache/hudi/issues/8265#issuecomment-1627766770
> > flink-table-planner has been removed from the classpath since flink1.15
>
> Then how the classes in flink-table-planner is loaded then, can we do
similiar loadings in
jlloh commented on issue #8191:
URL: https://github.com/apache/hudi/issues/8191#issuecomment-1627759070
Seeing something similar for Flink 1.16 with Hudi 0.13.1, COW insert with
metadata enabled. Problem seems to occur ~4 hours after the job has been
running. The job is an inline
[
https://issues.apache.org/jira/browse/HUDI-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-6515:
-
Labels: pull-request-available (was: )
> Bucket index row writer write record to wrong handle
>
hudi-bot commented on PR #9156:
URL: https://github.com/apache/hudi/pull/9156#issuecomment-1627738046
## CI report:
* 4416219014959fe02290bf41695abb70c31c1715 Azure:
[
https://issues.apache.org/jira/browse/HUDI-5759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Qijun Fu closed HUDI-5759.
--
Fix Version/s: 0.13.0
Resolution: Fixed
> Hudi do not support add column on mor table with log
>
[
https://issues.apache.org/jira/browse/HUDI-6336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Qijun Fu reassigned HUDI-6336:
--
Assignee: Qijun Fu
> Support TimelineBased Checkpoint Metadata for flink
>
[
https://issues.apache.org/jira/browse/HUDI-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Qijun Fu closed HUDI-6337.
--
Resolution: Won't Fix
> Incremental Clean skip fetch commit metadata for append mode
>
[
https://issues.apache.org/jira/browse/HUDI-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Qijun Fu reassigned HUDI-6515:
--
Assignee: Qijun Fu
> Bucket index row writer write record to wrong handle
>
hudi-bot commented on PR #9156:
URL: https://github.com/apache/hudi/pull/9156#issuecomment-1627697564
## CI report:
* 4416219014959fe02290bf41695abb70c31c1715 Azure:
hudi-bot commented on PR #9156:
URL: https://github.com/apache/hudi/pull/9156#issuecomment-1627695267
## CI report:
* 4416219014959fe02290bf41695abb70c31c1715 UNKNOWN
Bot commands
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the
stream2000 commented on code in PR #9156:
URL: https://github.com/apache/hudi/pull/9156#discussion_r1257466200
##
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestInsertTable.scala:
##
@@ -1247,6 +1247,27 @@ class TestInsertTable extends
stream2000 commented on code in PR #9156:
URL: https://github.com/apache/hudi/pull/9156#discussion_r1257466066
##
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/BucketBulkInsertDataInternalWriterHelper.java:
##
@@ -91,27 +90,20 @@ private
stream2000 opened a new pull request, #9156:
URL: https://github.com/apache/hudi/pull/9156
### Change Logs
Fix bucket index row writer would write record to wrong handle and remove
some useless code since the partition record is sorted.
### Impact
Fix bucket index row
Qijun Fu created HUDI-6515:
--
Summary: Bucket index row writer write record to wrong handle
Key: HUDI-6515
URL: https://issues.apache.org/jira/browse/HUDI-6515
Project: Apache Hudi
Issue Type: Bug
parisni commented on issue #9026:
URL: https://github.com/apache/hudi/issues/9026#issuecomment-1627681838
@yihua
> If the metadata table is queried through Spark datasource directly after
MDT compaction (i.e., no additional log file in the latest file slice), there
is no duplicate.
garyli1019 commented on code in PR #8679:
URL: https://github.com/apache/hudi/pull/8679#discussion_r1257449903
##
rfc/rfc-69/rfc-69.md:
##
@@ -0,0 +1,159 @@
+
+# RFC-69: Hudi 1.X
+
+## Proposers
+
+* Vinoth Chandar
+
+## Approvers
+
+* Hudi PMC
+
+## Status
+
+Under Review
+
[
https://issues.apache.org/jira/browse/HUDI-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
KnightChess closed HUDI-5861.
-
Resolution: Won't Fix
> table can not read date after overwrite table with bucket index
>
[
https://issues.apache.org/jira/browse/HUDI-5091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
KnightChess closed HUDI-5091.
-
Fix Version/s: 0.14.0
Resolution: Won't Fix
> MergeInto syntax merge_condition does not support
[
https://issues.apache.org/jira/browse/HUDI-5091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
KnightChess resolved HUDI-5091.
---
> MergeInto syntax merge_condition does not support Non-Equal
>
zouxxyy created HUDI-6514:
-
Summary: Remove hoodie.auto.commit
Key: HUDI-6514
URL: https://issues.apache.org/jira/browse/HUDI-6514
Project: Apache Hudi
Issue Type: Improvement
Reporter:
yihua commented on issue #9026:
URL: https://github.com/apache/hudi/issues/9026#issuecomment-1627630057
Based on my investigation, it looks like that the HFile does not contain
duplicate entries. If the metadata table is queried through Spark datasource
directly after MDT compaction
64 matches
Mail list logo
