hudi-bot commented on PR #11272:
URL: https://github.com/apache/hudi/pull/11272#issuecomment-2126334104
## CI report:
* 9c2adf6ccc3eaa2975e72301c37852667e051578 UNKNOWN
Bot commands
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run th
[
https://issues.apache.org/jira/browse/HUDI-4491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-4491:
-
Labels: pull-request-available (was: )
> Re-enable TestHoodieFlinkQuickstart
> -
a-erofeev opened a new pull request, #11272:
URL: https://github.com/apache/hudi/pull/11272
### Change Logs
Added dependency flink-metrics-dropwizard to
hudi-examples/hudi-examples-flink
### Impact
no
### Risk level (write none, low medium or high below)
no
a-erofeev commented on PR #11251:
URL: https://github.com/apache/hudi/pull/11251#issuecomment-2126173947
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
bvaradar commented on code in PR #11271:
URL: https://github.com/apache/hudi/pull/11271#discussion_r1610840996
##
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/LSMTimeline.java:
##
@@ -158,13 +159,18 @@ public static int
latestSnapshotVersion(HoodieTableMetaCl
xuzifu666 commented on code in PR #11267:
URL: https://github.com/apache/hudi/pull/11267#discussion_r1610831406
##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/lock/FileSystemBasedLockProvider.java:
##
@@ -96,6 +96,7 @@ public void close() {
xuzifu666 closed pull request #11267: [HUDI-7783] Fix connection leak in
FileSystemBasedLockProvider
URL: https://github.com/apache/hudi/pull/11267
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to th
danny0405 commented on code in PR #11146:
URL: https://github.com/apache/hudi/pull/11146#discussion_r1610825092
##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##
@@ -540,6 +547,51 @@ private HoodieFunctionalIndexDef
danny0405 commented on code in PR #11146:
URL: https://github.com/apache/hudi/pull/11146#discussion_r1610824721
##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##
@@ -437,9 +448,7 @@ private boolean initializeFromFil
danny0405 commented on code in PR #11146:
URL: https://github.com/apache/hudi/pull/11146#discussion_r1610823837
##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##
@@ -410,6 +413,14 @@ private boolean initializeFromFi
danny0405 commented on code in PR #11267:
URL: https://github.com/apache/hudi/pull/11267#discussion_r1610822698
##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/lock/FileSystemBasedLockProvider.java:
##
@@ -96,6 +96,7 @@ public void close() {
danny0405 commented on issue #11268:
URL: https://github.com/apache/hudi/issues/11268#issuecomment-2125996412
> We reviewed the deltastreamer code and noticed that the deltastreamer can
read commits from Kafka consumer groups
If the consumer `offset` is what the Hoodie checkpoint pers
danny0405 commented on code in PR #11224:
URL: https://github.com/apache/hudi/pull/11224#discussion_r1610819252
##
hudi-spark-datasource/hudi-spark3.5.x/src/main/scala/org/apache/spark/sql/adapter/Spark3_5Adapter.scala:
##
@@ -54,7 +54,7 @@ class Spark3_5Adapter extends BaseSpar
danny0405 commented on code in PR #11271:
URL: https://github.com/apache/hudi/pull/11271#discussion_r1610818043
##
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/LSMTimeline.java:
##
@@ -158,13 +159,18 @@ public static int
latestSnapshotVersion(HoodieTableMetaC
hudi-bot commented on PR #11271:
URL: https://github.com/apache/hudi/pull/11271#issuecomment-2125988423
## CI report:
* 818e29cf9483b5b3a17724a490e5c602f5b408c4 UNKNOWN
Bot commands
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run th
bvaradar opened a new pull request, #11271:
URL: https://github.com/apache/hudi/pull/11271
Found during backwards compatibility testing. Sometimes, Archiving would not
have run for a Hudi table. In that case, LSMTimeline must gracefully handle
instead of throwing File not found exception.
Jason-liujc commented on issue #6143:
URL: https://github.com/apache/hudi/issues/6143#issuecomment-2125944255
This happened for us and the root cause is we have concurrent running spark
jobs writing to the same insert_overwrite table, different partition. Our jobs
didn't have proper concurr
This is an automated email from the ASF dual-hosted git repository.
sivabalan pushed a commit to branch branch-0.x
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/branch-0.x by this push:
new 2e39b41be07 [HUDI-7784] Fix serde of Ho
nsivabalan merged PR #11270:
URL: https://github.com/apache/hudi/pull/11270
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apa
This is an automated email from the ASF dual-hosted git repository.
sivabalan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/master by this push:
new e1aa1bcb4af [HUDI-7784] Fix serde of HoodieHado
nsivabalan merged PR #11269:
URL: https://github.com/apache/hudi/pull/11269
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apa
ehurheap commented on issue #9903:
URL: https://github.com/apache/hudi/issues/9903#issuecomment-2125644288
I also ran into this problem when running:
`compaction validate --instant 20240516172801913`
The validation itself appears to complete ok because I see this output:
```
2
hudi-bot commented on PR #11270:
URL: https://github.com/apache/hudi/pull/11270#issuecomment-2125580506
## CI report:
* fd1bc98346a08d14b6f110a70827b27e888e77f3 UNKNOWN
Bot commands
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run th
hudi-bot commented on PR #11269:
URL: https://github.com/apache/hudi/pull/11269#issuecomment-2125492570
## CI report:
* 6ea8a900dbcd6c815270d07e51ed3683360462e5 UNKNOWN
Bot commands
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run th
yihua opened a new pull request, #11270:
URL: https://github.com/apache/hudi/pull/11270
### Change Logs
PR for master: https://github.com/apache/hudi/pull/11269
This PR targets at `branch-0.x`.
This PR fixes the issue that `HoodieHadoopConfiguration` is not properly
(de)seri
[
https://issues.apache.org/jira/browse/HUDI-7784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-7784:
-
Labels: hoodie-storage pull-request-available (was: hoodie-storage)
> Fix serde of HoodieHadoopCo
yihua opened a new pull request, #11269:
URL: https://github.com/apache/hudi/pull/11269
### Change Logs
This PR fixes the issue that `HoodieHadoopConfiguration` is not properly
(de)serialized by Kryo in Spark. A new test is added to validate the
(de)serialization. Before the fix, t
[
https://issues.apache.org/jira/browse/HUDI-7784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Guo updated HUDI-7784:
Labels: hoodie-storage (was: )
> Fix serde of HoodieHadoopConfiguration in Spark
>
[
https://issues.apache.org/jira/browse/HUDI-7784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Guo updated HUDI-7784:
Story Points: 2
> Fix serde of HoodieHadoopConfiguration in Spark
>
Ethan Guo created HUDI-7784:
---
Summary: Fix serde of HoodieHadoopConfiguration in Spark
Key: HUDI-7784
URL: https://issues.apache.org/jira/browse/HUDI-7784
Project: Apache Hudi
Issue Type: Improveme
[
https://issues.apache.org/jira/browse/HUDI-7784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Guo reassigned HUDI-7784:
---
Assignee: Ethan Guo
> Fix serde of HoodieHadoopConfiguration in Spark
> -
[
https://issues.apache.org/jira/browse/HUDI-7784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Guo updated HUDI-7784:
Fix Version/s: 0.15.0
> Fix serde of HoodieHadoopConfiguration in Spark
> --
[
https://issues.apache.org/jira/browse/HUDI-7784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Guo updated HUDI-7784:
Fix Version/s: 1.0.0
> Fix serde of HoodieHadoopConfiguration in Spark
> ---
codope commented on code in PR #11146:
URL: https://github.com/apache/hudi/pull/11146#discussion_r1610397009
##
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java:
##
@@ -783,4 +784,101 @@ public int
getNumFileGroupsForPartition(MetadataPartitionT
codope commented on code in PR #11146:
URL: https://github.com/apache/hudi/pull/11146#discussion_r1610379136
##
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataPayload.java:
##
@@ -295,6 +311,11 @@ protected HoodieMetadataPayload(String key, int type,
this.
dinhphu2k1-gif commented on issue #1:
URL: https://github.com/apache/hudi/issues/1#issuecomment-2125370032
@kon-si when i build hbase jar, where can i put it to profile hudi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
codope commented on code in PR #11146:
URL: https://github.com/apache/hudi/pull/11146#discussion_r1610368740
##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##
@@ -540,6 +547,51 @@ private HoodieFunctionalIndexDefini
kon-si commented on issue #1:
URL: https://github.com/apache/hudi/issues/1#issuecomment-2125323910
> @kon-si how to package Hudi again? Can you help me?
There are instruction on how to do it in the README of the hudi project.
--
This is an automated message from the Apache G
codope commented on code in PR #11146:
URL: https://github.com/apache/hudi/pull/11146#discussion_r1610329349
##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##
@@ -437,9 +448,7 @@ private boolean initializeFromFilesy
shubhamn21 commented on issue #11254:
URL: https://github.com/apache/hudi/issues/11254#issuecomment-2125294694
I think it may have to do something with AWSGlue compatibility. The
[documentation](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-format-hudi.html#aws-glue-pro
codope commented on code in PR #11146:
URL: https://github.com/apache/hudi/pull/11146#discussion_r1610294310
##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##
@@ -540,6 +547,51 @@ private HoodieFunctionalIndexDefini
soumilshah1995 closed issue #11258: [SUPPORT] Hudi SQL Based Transformer Fails
when trying to provide SQL File as input
URL: https://github.com/apache/hudi/issues/11258
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
soumilshah1995 commented on issue #11258:
URL: https://github.com/apache/hudi/issues/11258#issuecomment-2125096672
Thanks man
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment
codope commented on code in PR #11146:
URL: https://github.com/apache/hudi/pull/11146#discussion_r1610214658
##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##
@@ -410,6 +413,14 @@ private boolean initializeFromFiles
hudi-bot commented on PR #11043:
URL: https://github.com/apache/hudi/pull/11043#issuecomment-2125080635
## CI report:
* 2915ad105601b17cf6915c299c13e9933032a10b Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23
soumilshah1995 commented on issue #11258:
URL: https://github.com/apache/hudi/issues/11258#issuecomment-2125075068
really let me try
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
Pavan792reddy commented on issue #11263:
URL: https://github.com/apache/hudi/issues/11263#issuecomment-2125069736
the messageid is generating from the pulsar topic but it was generating as
**__messageId**| not the _MessageId_ ,
--
This is an automated message from the Apache Git Service.
codope commented on PR #11043:
URL: https://github.com/apache/hudi/pull/11043#issuecomment-2125053288
@KnightChess Thanks for your review. I have addressed your comments. Can you
please take a look again?
--
This is an automated message from the Apache Git Service.
To respond to the messa
codope commented on PR #11043:
URL: https://github.com/apache/hudi/pull/11043#issuecomment-2125051546
> Do we need to use parallel stream to improve efficiency in `allFiles.map`?
I tried but it seems like the usual `scala.collection.parallel` does not
work with Scala 2.13 (which is us
codope commented on code in PR #11043:
URL: https://github.com/apache/hudi/pull/11043#discussion_r1610168033
##
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/BloomFiltersIndexSupport.scala:
##
@@ -0,0 +1,87 @@
+/*
+ * Licensed to the Apache Software Foun
codope commented on code in PR #11043:
URL: https://github.com/apache/hudi/pull/11043#discussion_r1610167078
##
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/BloomFiltersIndexSupport.scala:
##
@@ -0,0 +1,87 @@
+/*
+ * Licensed to the Apache Software Foun
codope commented on code in PR #11043:
URL: https://github.com/apache/hudi/pull/11043#discussion_r1610166499
##
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/BloomFiltersIndexSupport.scala:
##
@@ -0,0 +1,87 @@
+/*
+ * Licensed to the Apache Software Foun
dinhphu2k1-gif commented on issue #1:
URL: https://github.com/apache/hudi/issues/1#issuecomment-2124992604
@kon-si how to package Hudi again? Can you help me?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
hudi-bot commented on PR #11043:
URL: https://github.com/apache/hudi/pull/11043#issuecomment-2124955598
## CI report:
* 2915ad105601b17cf6915c299c13e9933032a10b Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23
Pavan792reddy commented on issue #11263:
URL: https://github.com/apache/hudi/issues/11263#issuecomment-2124908753
@ad1happy2go i have made all the changes it was working as expected. Now
the script was failing with below error
--
This is an automated message from the Apache Git Service.
KishanFairmatic opened a new issue, #11268:
URL: https://github.com/apache/hudi/issues/11268
Our requirement is actually this: Supporting multiple deltastreamers writing
to a single hudi table
[https://github.com/apache/hudi/issues/6718](https://github.com/apache/hudi/issues/6718
hudi-bot commented on PR #11267:
URL: https://github.com/apache/hudi/pull/11267#issuecomment-2124564766
## CI report:
* 7f32187c42483782e078d1094a639af1fdcc0055 UNKNOWN
Bot commands
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run th
[
https://issues.apache.org/jira/browse/HUDI-7783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-7783:
-
Labels: pull-request-available (was: )
> Fix connection leak in FileSystemBasedLockProvider
> ---
xuzifu666 opened a new pull request, #11267:
URL: https://github.com/apache/hudi/pull/11267
### Change Logs
if fs.hdfs.impl.disable.cache is true，FileSystemBasedLockProvider memory
would increase all the time to oom due to FileSystem Cache is too large，so we
should close fs in close
xy created HUDI-7783:
Summary: Fix connection leak in FileSystemBasedLockProvider
Key: HUDI-7783
URL: https://issues.apache.org/jira/browse/HUDI-7783
Project: Apache Hudi
Issue Type: Improvement
leesf commented on PR #11224:
URL: https://github.com/apache/hudi/pull/11224#issuecomment-2124474355
> > When executed on a Delta table, this may result in an error.
>
> What action are we executing here?
like `INSERT OVERWRITE delta.`/tmp/delta-table` SELECT col1 as id FROM
VA
[
https://issues.apache.org/jira/browse/HUDI-7781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chen closed HUDI-7781.
Resolution: Fixed
Fixed via master branch:c7d2fc05fd7f285abd36c561217bf67de4e0479f
> Filter wrong partition
[
https://issues.apache.org/jira/browse/HUDI-7781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chen updated HUDI-7781:
-
Fix Version/s: 1.0.0
> Filter wrong partitions when using
> hoodie.datasource.write.partitions.to.delete
This is an automated email from the ASF dual-hosted git repository.
danny0405 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/master by this push:
new c7d2fc05fd7 [HUDI-7781] Filter wrong partitions
danny0405 merged PR #11260:
URL: https://github.com/apache/hudi/pull/11260
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apac
danny0405 commented on code in PR #11146:
URL: https://github.com/apache/hudi/pull/11146#discussion_r1609596497
##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##
@@ -540,6 +547,51 @@ private HoodieFunctionalIndexDef
SuneethaYamani commented on issue #11168:
URL: https://github.com/apache/hudi/issues/11168#issuecomment-2124294672
@ad1happy2go
I am using 2.12-0.14.0 hudi jar
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
U
codope commented on code in PR #11154:
URL: https://github.com/apache/hudi/pull/11154#discussion_r1609536039
##
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSchemaUtils.scala:
##
@@ -93,14 +93,14 @@ object HoodieSchemaUtils {
// in the ta
Zouxxyy closed pull request #11218: [WIP] HoodieClusteringJob support purge
pending clustering job if conflict
URL: https://github.com/apache/hudi/pull/11218
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above t
69 matches
Mail list logo
