codope merged PR #10962:
URL: https://github.com/apache/hudi/pull/10962
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.
hudi-bot commented on PR #10962:
URL: https://github.com/apache/hudi/pull/10962#issuecomment-2057374962
## CI report:
* 04a008a504f9d9f1b8cfb8ae5199a85ec0fca6fe UNKNOWN
* faf292c41024255b88f01c0f3193b8fd72a2849d Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4
Sagar Sumit created HUDI-7620:
-
Summary: Support querying multiple functional index in single query
Key: HUDI-7620
URL: https://issues.apache.org/jira/browse/HUDI-7620
Project: Apache Hudi
Issue
hudi-bot commented on PR #10956:
URL: https://github.com/apache/hudi/pull/10956#issuecomment-2057374664
## CI report:
* be7795021e2cffe600a109448ed02e5860385b9f Azure:
[FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23
Qiuzhuang commented on issue #8567:
URL: https://github.com/apache/hudi/issues/8567#issuecomment-2057270718
> if you have any pending/inflight in data table timeline, metadata table
compaction will stalled until that gets to completion. may be there is some
lingering pending operation (clus
hudi-bot commented on PR #10956:
URL: https://github.com/apache/hudi/pull/10956#issuecomment-2057219823
## CI report:
* c8f507bcac03c7183893400487a1885400c46853 Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23
[
https://issues.apache.org/jira/browse/HUDI-7378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Guo closed HUDI-7378.
---
Resolution: Fixed
> Fix Spark SQL DML with custom key generator
> ---
[
https://issues.apache.org/jira/browse/HUDI-7378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Guo updated HUDI-7378:
Remaining Estimate: (was: 0.05h)
Original Estimate: (was: 0.05h)
> Fix Spark SQL DML with custo
empcl commented on code in PR #11006:
URL: https://github.com/apache/hudi/pull/11006#discussion_r1565987327
##
hudi-sync/hudi-sync-common/src/main/java/org/apache/hudi/sync/common/util/Parquet2SparkSchemaUtils.java:
##
@@ -140,7 +141,7 @@ private static String convertGroupField(
jonvex commented on code in PR #10956:
URL: https://github.com/apache/hudi/pull/10956#discussion_r1565961863
##
hudi-spark-datasource/hudi-spark3-common/src/main/scala/org/apache/spark/sql/execution/datasources/Spark3ParquetSchemaEvolutionUtils.scala:
##
@@ -0,0 +1,194 @@
+/*
+
jonvex commented on code in PR #10956:
URL: https://github.com/apache/hudi/pull/10956#discussion_r1565959794
##
hudi-spark-datasource/hudi-spark2/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/Spark24ParquetReader.scala:
##
@@ -141,8 +150,20 @@ class Spark24Pa
hudi-bot commented on PR #10962:
URL: https://github.com/apache/hudi/pull/10962#issuecomment-2057093930
## CI report:
* 04a008a504f9d9f1b8cfb8ae5199a85ec0fca6fe UNKNOWN
* 92e3c3d32d8e8923ec6662d43275a0ba22f2b1b5 Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4
hudi-bot commented on PR #10962:
URL: https://github.com/apache/hudi/pull/10962#issuecomment-2057074926
## CI report:
* 04a008a504f9d9f1b8cfb8ae5199a85ec0fca6fe UNKNOWN
* 92e3c3d32d8e8923ec6662d43275a0ba22f2b1b5 Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4
jonvex commented on code in PR #11019:
URL: https://github.com/apache/hudi/pull/11019#discussion_r1565935774
##
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/InsertIntoHoodieTableCommand.scala:
##
@@ -142,12 +142,26 @@ object InsertIntoHoodieT
This is an automated email from the ASF dual-hosted git repository.
codope pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/master by this push:
new db7e93e712c [HUDI-3907] Claim RFC-77 for secondary
codope merged PR #11025:
URL: https://github.com/apache/hudi/pull/11025
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.
nsivabalan commented on code in PR #11018:
URL: https://github.com/apache/hudi/pull/11018#discussion_r1565884029
##
hudi-utilities/src/main/java/org/apache/hudi/utilities/streamer/HoodieStreamer.java:
##
@@ -424,6 +439,11 @@ public static class Config implements Serializable {
hudi-bot commented on PR #10962:
URL: https://github.com/apache/hudi/pull/10962#issuecomment-2056954697
## CI report:
* 04a008a504f9d9f1b8cfb8ae5199a85ec0fca6fe UNKNOWN
* 92e3c3d32d8e8923ec6662d43275a0ba22f2b1b5 Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4
hudi-bot commented on PR #10962:
URL: https://github.com/apache/hudi/pull/10962#issuecomment-2056804490
## CI report:
* 08cb35bbc1a1a44fe1ff00c673b7be938352a8d0 Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23
hudi-bot commented on PR #10962:
URL: https://github.com/apache/hudi/pull/10962#issuecomment-2056789157
## CI report:
* 08cb35bbc1a1a44fe1ff00c673b7be938352a8d0 Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23
hudi-bot commented on PR #11025:
URL: https://github.com/apache/hudi/pull/11025#issuecomment-2056789614
## CI report:
* 09dca4eb61930d84da62f03f4dd0056ce56bdb39 Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23
hudi-bot commented on PR #10962:
URL: https://github.com/apache/hudi/pull/10962#issuecomment-2056774676
## CI report:
* 08cb35bbc1a1a44fe1ff00c673b7be938352a8d0 Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23
Tartarus0zm commented on code in PR #10886:
URL: https://github.com/apache/hudi/pull/10886#discussion_r1565699528
##
hudi-common/src/main/java/org/apache/hudi/common/model/HoodiePartitionMetadata.java:
##
@@ -92,11 +92,12 @@ public int getPartitionDepth() {
/**
* Write
hudi-bot commented on PR #11025:
URL: https://github.com/apache/hudi/pull/11025#issuecomment-2056561659
## CI report:
* 09dca4eb61930d84da62f03f4dd0056ce56bdb39 Azure:
[PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23
ad1happy2go commented on issue #10995:
URL: https://github.com/apache/hudi/issues/10995#issuecomment-2056550465
@brightwon Do you have any more doubts? Feel free to close if you are good
on this. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the messag
hudi-bot commented on PR #11025:
URL: https://github.com/apache/hudi/pull/11025#issuecomment-2056548733
## CI report:
* 09dca4eb61930d84da62f03f4dd0056ce56bdb39 UNKNOWN
Bot commands
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run th
codope commented on PR #10846:
URL: https://github.com/apache/hudi/pull/10846#issuecomment-2056546988
> @codope Have you made more progress on this fix?
Not yet. We need to fix https://issues.apache.org/jira/browse/HUDI-7554 as
well.
--
This is an automated message from the Apache
codope commented on code in PR #10814:
URL: https://github.com/apache/hudi/pull/10814#discussion_r1565590596
##
rfc/rfc-77/rfc-77.md:
##
@@ -0,0 +1,247 @@
+
+
+# RFC-77: Secondary Indexes
+
+## Proposers
+
+- @bhat-vinay
+- @codope
+
+## Approvers
+ - @vinothchandar
+ - @nsivaba
codope commented on code in PR #10814:
URL: https://github.com/apache/hudi/pull/10814#discussion_r1565589697
##
rfc/rfc-77/rfc-77.md:
##
@@ -0,0 +1,247 @@
+
+
+# RFC-77: Secondary Indexes
+
+## Proposers
+
+- @bhat-vinay
+- @codope
+
+## Approvers
+ - @vinothchandar
+ - @nsivaba
codope commented on code in PR #10814:
URL: https://github.com/apache/hudi/pull/10814#discussion_r1565589030
##
rfc/README.md:
##
@@ -111,4 +111,5 @@ The list of all RFCs can be found here.
| 73 | [Multi-Table Transactions](./rfc-73/rfc-73.md)
This is an automated email from the ASF dual-hosted git repository.
danny0405 pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
from 9cfc543bf3b [HUDI-7493] Consistent naming of Cleaner configuration
parameters (#10851)
add 2a0969c9972 [HUDI-76
[
https://issues.apache.org/jira/browse/HUDI-7619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chen closed HUDI-7619.
Resolution: Fixed
Fixed via master branch: 2a0969c9972ef746d377dbddd278ef13bf3d299d
> Remove code duplicate
[
https://issues.apache.org/jira/browse/HUDI-7619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chen updated HUDI-7619:
-
Fix Version/s: 1.0.0
> Remove code duplicates in HoodieTableMetadataUtil
> ---
danny0405 merged PR #11022:
URL: https://github.com/apache/hudi/pull/11022
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apac
codope opened a new pull request, #11025:
URL: https://github.com/apache/hudi/pull/11025
### Change Logs
Claim RFC as per process.
### Impact
none
### Risk level (write none, low medium or high below)
none
### Documentation Update
_Describe any
wombatu-kun commented on PR #10962:
URL: https://github.com/apache/hudi/pull/10962#issuecomment-2056423933
@codope thanks for review! Fixed all by your suggestions and made separate
PR https://github.com/apache/hudi/pull/11022 with extraction of
ClosableIterator creation.
--
This is an
hudi-bot commented on PR #10962:
URL: https://github.com/apache/hudi/pull/10962#issuecomment-2056413847
## CI report:
* 08cb35bbc1a1a44fe1ff00c673b7be938352a8d0 Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23
empcl commented on code in PR #11006:
URL: https://github.com/apache/hudi/pull/11006#discussion_r1565442379
##
hudi-sync/hudi-sync-common/src/main/java/org/apache/hudi/sync/common/util/Parquet2SparkSchemaUtils.java:
##
@@ -140,7 +141,7 @@ private static String convertGroupField(
hudi-bot commented on PR #11005:
URL: https://github.com/apache/hudi/pull/11005#issuecomment-2056315915
## CI report:
* 31be1b14c714e549548033084c41a8b93b331b3c Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23
danny0405 commented on issue #11023:
URL: https://github.com/apache/hudi/issues/11023#issuecomment-2056314854
Weird, it seems this split does not contain any base or log files, it is
peobably caused by ingestion corruption.
--
This is an automated message from the Apache Git Service.
To r
danny0405 commented on issue #11016:
URL: https://github.com/apache/hudi/issues/11016#issuecomment-2056310014
> I can only find all the data for February 25, 2024, and cannot find any
other data. By the way, we have configured metadata synchronization to Hive,
and all the written data can
danny0405 commented on issue #11004:
URL: https://github.com/apache/hudi/issues/11004#issuecomment-2056307205
The write task holds the write statuses in the state which would be
resubmitted to the driver for committing to Hudi.
--
This is an automated message from the Apache Git Service.
waywtdcc closed issue #11024: [BUG] When writing large amounts, primary keys
are duplicated
URL: https://github.com/apache/hudi/issues/11024
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the speci
VitoMakarevich commented on issue #10997:
URL: https://github.com/apache/hudi/issues/10997#issuecomment-2056274777
Hello, thanks for the suggestions! As I said, I'd like to know how I can
speed up this individual part, I know it's option to use MOR in theory, but
it's impossible for our use
hudi-bot commented on PR #11022:
URL: https://github.com/apache/hudi/pull/11022#issuecomment-2056266784
## CI report:
* 444e5dabdd5fb9b581bc36149ad125427e192fd2 Azure:
[PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23
hudi-bot commented on PR #11019:
URL: https://github.com/apache/hudi/pull/11019#issuecomment-2056266617
## CI report:
* f05565ba630aae9378588522c6860bfee558113c Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23
hudi-bot commented on PR #11018:
URL: https://github.com/apache/hudi/pull/11018#issuecomment-2056266429
## CI report:
* c0923360a546fcfd71c0111b9ea29894fa1fe7f3 Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23
jack1234smith opened a new issue, #11023:
URL: https://github.com/apache/hudi/issues/11023
**Describe the problem you faced**
Error exception:
java.util.NoSuchElementException: No value present in Option
at org.apache.hudi.common.util.Option.get(Option.java:89)
at
o
juice411 commented on issue #11016:
URL: https://github.com/apache/hudi/issues/11016#issuecomment-2056158828
The data is written through the flink-mysql-cdc method, from January 1,
2024, to March 31, 2024, with 10,000 records being written to MySQL every day.
After completing one round of w
seekforshell commented on issue #11004:
URL: https://github.com/apache/hudi/issues/11004#issuecomment-2056108890
eg. flink stream job like kafka_source -> window -> bucket_write, when
bucket_write operator failed, the buffer data lost, although checkpoint
failed for the first time, but af
danny0405 commented on issue #11017:
URL: https://github.com/apache/hudi/issues/11017#issuecomment-2056090368
> One is that the new data overwrites the old data
Not sure if you are using the `upsert` operation by using the index for
updating. If you are using the `Flink_STATE` index,
danny0405 commented on issue #11016:
URL: https://github.com/apache/hudi/issues/11016#issuecomment-2056069011
How did you write the earliest data set, are they got updated or just got
lost?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log
danny0405 merged PR #10856:
URL: https://github.com/apache/hudi/pull/10856
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apac
hudi-bot commented on PR #11022:
URL: https://github.com/apache/hudi/pull/11022#issuecomment-2056061482
## CI report:
* 444e5dabdd5fb9b581bc36149ad125427e192fd2 UNKNOWN
Bot commands
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run th
hudi-bot commented on PR #10962:
URL: https://github.com/apache/hudi/pull/10962#issuecomment-2056060441
## CI report:
* 4115c24a4e75904c3c8940cd7d7fd56be020d990 Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23
[
https://issues.apache.org/jira/browse/HUDI-7493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chen closed HUDI-7493.
Fix Version/s: 1.0.0
Resolution: Fixed
Fixed via master branch: 9cfc543bf3bea06b748d2ea2d1341302c2f55
danny0405 commented on code in PR #11006:
URL: https://github.com/apache/hudi/pull/11006#discussion_r1565343193
##
hudi-sync/hudi-sync-common/src/main/java/org/apache/hudi/sync/common/util/Parquet2SparkSchemaUtils.java:
##
@@ -140,7 +141,7 @@ private static String convertGroupFi
This is an automated email from the ASF dual-hosted git repository.
danny0405 pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
from ad493d2ab3c [HUDI-7584] Always read log block lazily and remove
readBlockLazily argument (#11015)
add 9cfc543bf
danny0405 merged PR #10851:
URL: https://github.com/apache/hudi/pull/10851
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apac
hudi-bot commented on PR #10962:
URL: https://github.com/apache/hudi/pull/10962#issuecomment-2056025121
## CI report:
* 4115c24a4e75904c3c8940cd7d7fd56be020d990 Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23
[
https://issues.apache.org/jira/browse/HUDI-7619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-7619:
-
Labels: pull-request-available refactoring (was: refactoring)
> Remove code duplicates in HoodieT
wombatu-kun opened a new pull request, #11022:
URL: https://github.com/apache/hudi/pull/11022
### Change Logs
Trivial refactoring: removed code duplication in HoodieTableMetadataUtil by
extracting of ClosableIterator creation to separate method and
call it from 3 different places.
[
https://issues.apache.org/jira/browse/HUDI-7467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
tao pan updated HUDI-7467:
--
Fix Version/s: 1.0.0
> TestHoodieDeltaStreamer. testAutoGenerateRecordKeys
> ---
[
https://issues.apache.org/jira/browse/HUDI-7467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
tao pan updated HUDI-7467:
--
Component/s: tests-ci
> TestHoodieDeltaStreamer. testAutoGenerateRecordKeys
> --
[
https://issues.apache.org/jira/browse/HUDI-7619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vova Kolmakov updated HUDI-7619:
Summary: Remove code duplicates in HoodieTableMetadataUtil (was: Remove
code duplication in HoodieT
[
https://issues.apache.org/jira/browse/HUDI-7467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
tao pan updated HUDI-7467:
--
Fix Version/s: (was: 1.0.0)
> TestHoodieDeltaStreamer. testAutoGenerateRecordKeys
>
AshinGau closed issue #11003: Different system parse different time zone of
timestamp type from the parquet file created by hudi
URL: https://github.com/apache/hudi/issues/11003
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
Vova Kolmakov created HUDI-7619:
---
Summary: Remove code duplication in HoodieTableMetadataUtil
Key: HUDI-7619
URL: https://issues.apache.org/jira/browse/HUDI-7619
Project: Apache Hudi
Issue Type
[
https://issues.apache.org/jira/browse/HUDI-7619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vova Kolmakov updated HUDI-7619:
Status: In Progress (was: Open)
> Remove code duplication in HoodieTableMetadataUtil
>
AshinGau commented on issue #11003:
URL: https://github.com/apache/hudi/issues/11003#issuecomment-2055907222
By referring to Doris's documentation
[time_zone](https://doris.apache.org/docs/dev/advanced/time-zone/), it displays
timestamp as `absolute time`. `absolute time` is an non-standard
AshinGau commented on issue #11003:
URL: https://github.com/apache/hudi/issues/11003#issuecomment-2055847646
After detailed reading of the document [definition of
isAdjustedToUTC=true](https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#instant-semantics-timestamps-normalize
pontisa95 commented on issue #11011:
URL: https://github.com/apache/hudi/issues/11011#issuecomment-2055841107
hello, i don't know what is the step were we my job is getting blocked
because there are no running job when it happens, and it happens over multiple
process that reply over differe
hudi-bot commented on PR #11006:
URL: https://github.com/apache/hudi/pull/11006#issuecomment-2055749078
## CI report:
* 572d9726940855d2fe56db685e5dacee20163de8 UNKNOWN
* 84784a5c3b108563d7454998f1ce6a28aa80f150 Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4
hudi-bot commented on PR #11005:
URL: https://github.com/apache/hudi/pull/11005#issuecomment-2055748819
## CI report:
* c0ca195bf69614784e60bd51d300df04a61fdf21 Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23
hudi-bot commented on PR #11005:
URL: https://github.com/apache/hudi/pull/11005#issuecomment-2055715653
## CI report:
* c0ca195bf69614784e60bd51d300df04a61fdf21 Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23
101 - 175 of 175 matches
Mail list logo
