abhisheksahani91 commented on issue #10270:
URL: https://github.com/apache/hudi/issues/10270#issuecomment-1849503205
@ad1happy2go
I have scaled the infra and compaction execution time has been reduced from
20 minutes to 10.
But I have one doubt, on every compaction, the number of
danny0405 commented on code in PR #10266:
URL: https://github.com/apache/hudi/pull/10266#discussion_r1422044423
##
rfc/rfc-76/rfc-76.md:
##
@@ -0,0 +1,238 @@
+
+# RFC-[74]: [support EventTimeBasedCompactionStrategy]
+
+## Proposers
+
+- @waitingF
+
+## Approvers
+ - @
+ - @
+
+#
danny0405 commented on code in PR #10266:
URL: https://github.com/apache/hudi/pull/10266#discussion_r1422041532
##
rfc/rfc-76/rfc-76.md:
##
@@ -0,0 +1,238 @@
+
+# RFC-[74]: [support EventTimeBasedCompactionStrategy]
+
+## Proposers
+
+- @waitingF
+
+## Approvers
+ - @
+ - @
+
+#
hudi-bot commented on PR #10272:
URL: https://github.com/apache/hudi/pull/10272#issuecomment-1849483356
## CI report:
* 9e4cc4f0b2c56a589284434c7c9bd1ed12f6d568 Azure:
[PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21
hudi-bot commented on PR #10295:
URL: https://github.com/apache/hudi/pull/10295#issuecomment-1849483513
## CI report:
* d55b91488de445e6ec0e2261ba0a824cd4b0ad25 Azure:
[PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21
hudi-bot commented on PR #10297:
URL: https://github.com/apache/hudi/pull/10297#issuecomment-1849483572
## CI report:
* fe0a9fb6f96859b5d2bc2254899f2f5c8624d841 Azure:
[PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21
[
https://issues.apache.org/jira/browse/HUDI-7208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-7208:
-
Labels: pull-request-available (was: )
> Do writing stage should shutdown with error when insert
hudi-bot commented on PR #10297:
URL: https://github.com/apache/hudi/pull/10297#issuecomment-1849474557
## CI report:
* fe0a9fb6f96859b5d2bc2254899f2f5c8624d841 UNKNOWN
Bot commands
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run th
hudi-bot commented on PR #10295:
URL: https://github.com/apache/hudi/pull/10295#issuecomment-1849474516
## CI report:
* d55b91488de445e6ec0e2261ba0a824cd4b0ad25 Azure:
[PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21
hudi-bot commented on PR #10272:
URL: https://github.com/apache/hudi/pull/10272#issuecomment-1849474384
## CI report:
* 9e4cc4f0b2c56a589284434c7c9bd1ed12f6d568 Azure:
[PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21
hudi-bot commented on PR #10295:
URL: https://github.com/apache/hudi/pull/10295#issuecomment-1849465863
## CI report:
* d55b91488de445e6ec0e2261ba0a824cd4b0ad25 Azure:
[PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21
hudi-bot commented on PR #10176:
URL: https://github.com/apache/hudi/pull/10176#issuecomment-1849465480
## CI report:
* 1dfeda49c7863ac379aee22181fc6178876ba3a3 Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21
ad1happy2go commented on issue #10262:
URL: https://github.com/apache/hudi/issues/10262#issuecomment-1849462873
@CodyPin Can you try the demo first with stable version i.e. 0.14.0 and see
if that works?
--
This is an automated message from the Apache Git Service.
To respond to the messag
ad1happy2go commented on issue #10262:
URL: https://github.com/apache/hudi/issues/10262#issuecomment-1849462165
@georgepap9808 `java.lang.NoSuchMethodError:
scala.Function1.$init$(Lscala/Function1;)V` is due to scala version is
conflicting. can you try build with -Dscala-2.11
--
This is
ad1happy2go commented on issue #9977:
URL: https://github.com/apache/hudi/issues/9977#issuecomment-1849459064
@mzheng-plaid I looked into your code and found out that number of fields in
your dataset is more than 100 which is the cause for this issue. I set this
config `("spark.sql.codegen.
[
https://issues.apache.org/jira/browse/HUDI-7208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xy updated HUDI-7208:
-
Description:
when user execute merge into or insert into to a table，if source table data is
not right（such as source sche
yuruguo commented on PR #10295:
URL: https://github.com/apache/hudi/pull/10295#issuecomment-1849450278
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
T
[
https://issues.apache.org/jira/browse/HUDI-7209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-7209:
-
Labels: pull-request-available (was: )
> Add configuration to skip not exists file in streaming r
yuruguo commented on PR #10295:
URL: https://github.com/apache/hudi/pull/10295#issuecomment-1849449272
hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
xuzifu666 opened a new pull request, #10297:
URL: https://github.com/apache/hudi/pull/10297
### Change Logs
when user execute merge into or insert into to a table，if source table data
is not right（such as source schema is not consistent to target table），should
shutdown and throw erro
[
https://issues.apache.org/jira/browse/HUDI-7209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruguo Yu updated HUDI-7209:
---
Description:
In `streaming reading`, if there are a large number of files in metada,
especially archive files
[
https://issues.apache.org/jira/browse/HUDI-7209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruguo Yu updated HUDI-7209:
---
Description:
In `streaming reading`, if there are a large number of files in metada,
especially archive files
[
https://issues.apache.org/jira/browse/HUDI-7209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruguo Yu updated HUDI-7209:
---
Attachment: 289447957-f25cda8d-e75c-4380-b660-8ad347c4a6ca.png
> Add configuration to skip not exists file in
[
https://issues.apache.org/jira/browse/HUDI-7209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruguo Yu updated HUDI-7209:
---
Fix Version/s: 1.0.0
> Add configuration to skip not exists file in streaming read
> -
Ruguo Yu created HUDI-7209:
--
Summary: Add configuration to skip not exists file in streaming
read
Key: HUDI-7209
URL: https://issues.apache.org/jira/browse/HUDI-7209
Project: Apache Hudi
Issue Type
[
https://issues.apache.org/jira/browse/HUDI-7208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xy updated HUDI-7208:
-
Description:
when user execute merge into or insert into to a table，if source table data is
not right（such as source sche
yuruguo commented on code in PR #10295:
URL: https://github.com/apache/hudi/pull/10295#discussion_r1422008400
##
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/source/IncrementalInputSplits.java:
##
@@ -356,7 +356,22 @@ private List getIncInputSplits(
LOG.
[
https://issues.apache.org/jira/browse/HUDI-7208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xy updated HUDI-7208:
-
Attachment: CAPTURE_2023-12-11_144901.jpg
CAPTURE_2023-12-11_144849.jpg
CAPTURE_2023-12-11_
xy created HUDI-7208:
Summary: Do writing stage should shutdown with error when insert
failed to reduce user execute time and show error details
Key: HUDI-7208
URL: https://issues.apache.org/jira/browse/HUDI-7208
[
https://issues.apache.org/jira/browse/HUDI-7207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ma Jian updated HUDI-7207:
--
Priority: Blocker (was: Major)
> Concurrent archiving and data reading leads to missing data in query results.
Ma Jian created HUDI-7207:
-
Summary: Concurrent archiving and data reading leads to missing
data in query results.
Key: HUDI-7207
URL: https://issues.apache.org/jira/browse/HUDI-7207
Project: Apache Hudi
danny0405 commented on code in PR #10295:
URL: https://github.com/apache/hudi/pull/10295#discussion_r1421970338
##
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/source/IncrementalInputSplits.java:
##
@@ -356,7 +356,22 @@ private List getIncInputSplits(
LO
ad1happy2go commented on issue #10287:
URL: https://github.com/apache/hudi/issues/10287#issuecomment-1849367305
@soumilshah1995 I saw this error before. Can you put your hudi bundle jars
in the jars folder and set spark configurations in spark-defaults.yaml.
--
This is an automated messa
ad1happy2go commented on issue #10296:
URL: https://github.com/apache/hudi/issues/10296#issuecomment-1849364651
@masthanmca It looks like the Spark error and not hudi. There are some
struct fields in your data which you are trying to map with schema having Map
data type.
--
This is an a
ad1happy2go commented on issue #10273:
URL: https://github.com/apache/hudi/issues/10273#issuecomment-1849361768
@MikeMccree Can you let us know the hive sync configurations you are using?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on
hudi-bot commented on PR #10153:
URL: https://github.com/apache/hudi/pull/10153#issuecomment-1849356018
## CI report:
* 76e1d519c42565b99b35837c3c767b600cdee311 Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21
hudi-bot commented on PR #10295:
URL: https://github.com/apache/hudi/pull/10295#issuecomment-1849350309
## CI report:
* d55b91488de445e6ec0e2261ba0a824cd4b0ad25 Azure:
[PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21
hudi-bot commented on PR #10153:
URL: https://github.com/apache/hudi/pull/10153#issuecomment-1849349982
## CI report:
* 76e1d519c42565b99b35837c3c767b600cdee311 UNKNOWN
Bot commands
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run th
masthanmca opened a new issue, #10296:
URL: https://github.com/apache/hudi/issues/10296
**_Tips before filing an issue_**
- Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)? yes
- Join the mailing list to engage in conversations and get faster support at
dev
hudi-bot commented on PR #10295:
URL: https://github.com/apache/hudi/pull/10295#issuecomment-1849318100
## CI report:
* d55b91488de445e6ec0e2261ba0a824cd4b0ad25 UNKNOWN
Bot commands
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run th
hudi-bot commented on PR #10272:
URL: https://github.com/apache/hudi/pull/10272#issuecomment-1849318007
## CI report:
* 9e4cc4f0b2c56a589284434c7c9bd1ed12f6d568 Azure:
[PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21
hudi-bot commented on PR #10176:
URL: https://github.com/apache/hudi/pull/10176#issuecomment-1849317825
## CI report:
* 00d6025996b63ead6e710533a1bb005571c6db5c Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21
hudi-bot commented on PR #10272:
URL: https://github.com/apache/hudi/pull/10272#issuecomment-1849312466
## CI report:
* 9e4cc4f0b2c56a589284434c7c9bd1ed12f6d568 UNKNOWN
Bot commands
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run th
hudi-bot commented on PR #10176:
URL: https://github.com/apache/hudi/pull/10176#issuecomment-1849312296
## CI report:
* 00d6025996b63ead6e710533a1bb005571c6db5c Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21
yuruguo opened a new pull request, #10295:
URL: https://github.com/apache/hudi/pull/10295
### Change Logs
_Describe context and summary for this change. Highlight if any code was
copied._
### Impact
_Describe any public API or user-facing feature change or any performanc
codope commented on code in PR #10272:
URL: https://github.com/apache/hudi/pull/10272#discussion_r1421929085
##
hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/HoodieDatasetBulkInsertHelper.scala:
##
@@ -186,7 +179,9 @@ object HoodieDatasetBulkInsertHelper
}
This is an automated email from the ASF dual-hosted git repository.
sivabalan pushed a commit to branch release-0.14.0-siva-0.14.1
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/release-0.14.0-siva-0.14.1 by
this push:
new 68f371
nsivabalan merged PR #10284:
URL: https://github.com/apache/hudi/pull/10284
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apa
bhat-vinay commented on PR #10272:
URL: https://github.com/apache/hudi/pull/10272#issuecomment-1849294331
Thanks for the comments @codope. Addressed comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL ab
bhat-vinay commented on code in PR #10272:
URL: https://github.com/apache/hudi/pull/10272#discussion_r1421924315
##
hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/HoodieDatasetBulkInsertHelper.scala:
##
@@ -243,21 +238,17 @@ object HoodieDatasetBulkInsertHelper
bhat-vinay commented on code in PR #10272:
URL: https://github.com/apache/hudi/pull/10272#discussion_r1421923242
##
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/BulkInsertDataInternalWriterHelper.java:
##
@@ -124,7 +129,31 @@ public void write(
bhat-vinay commented on code in PR #10272:
URL: https://github.com/apache/hudi/pull/10272#discussion_r1421922943
##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java:
##
@@ -1390,6 +1390,10 @@ public boolean shouldAllowMultiWriteOnSameIns
sagarlakshmipathy opened a new pull request, #10294:
URL: https://github.com/apache/hudi/pull/10294
Refer: https://github.com/apache/hudi/pull/10202
### Change Logs
Update SQL Queries compatibility matrix
Redshift spectrum support Hudi MOR RO tables but doesn't support Snaps
This is an automated email from the ASF dual-hosted git repository.
vbalaji pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
from 5fb00ef506f [HUDI-7201] Schema Evolution: use target schema if source
is empty (#10288)
add a7c01f6874b [HUDI-717
bvaradar commented on PR #10242:
URL: https://github.com/apache/hudi/pull/10242#issuecomment-1849276715
@wecharyu : Thanks a lot for the contribution
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
bvaradar merged PR #10242:
URL: https://github.com/apache/hudi/pull/10242
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apach
sagarlakshmipathy opened a new pull request, #10293:
URL: https://github.com/apache/hudi/pull/10293
### Change Logs
Added pointers to sync to Glue from Spark and running sync tool on EMR.
### Impact
Doc update
### Risk level (write none, low medium or high below)
danny0405 commented on issue #10285:
URL: https://github.com/apache/hudi/issues/10285#issuecomment-1849247468
It is designed for batch execution job, maybe we should add back the
decision with `clean.async.enabled` check:
https://github.com/apache/hudi/pull/6515, we trigger the cleaning on
zyclove commented on PR #10288:
URL: https://github.com/apache/hudi/pull/10288#issuecomment-1849234079
@nsivabalan @danny0405
Seeing that there are still 34 issues to be fixed in release 0.14.1, we are
very looking forward to it.
We hope it can be speeded up. We also hope to switch t
YannByron commented on issue #10252:
URL: https://github.com/apache/hudi/issues/10252#issuecomment-1849234421
@vinothchandar, i'm also glad to work with you guys.
Honestly, item 1 (a native reader in velox for mor table) is beyond my
ability.
I can implement a gluten-hudi module i
sagarlakshmipathy closed pull request #10202: [MINOR] [DOCS] Update SQL Queries
compatibility matrix
URL: https://github.com/apache/hudi/pull/10202
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to th
[
https://issues.apache.org/jira/browse/HUDI-7206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan closed HUDI-7206.
-
Fix Version/s: 0.14.1
Assignee: sivabalan narayanan
Resolution: Fixed
> Fi
hudi-bot commented on PR #10242:
URL: https://github.com/apache/hudi/pull/10242#issuecomment-1849042637
## CI report:
* 84b76bad9e89a8ca7a61b74c9faf8ddcdef511f6 Azure:
[FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21
This is an automated email from the ASF dual-hosted git repository.
sivabalan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/master by this push:
new 5fb00ef506f [HUDI-7201] Schema Evolution: use t
nsivabalan merged PR #10288:
URL: https://github.com/apache/hudi/pull/10288
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apa
nsivabalan commented on PR #10288:
URL: https://github.com/apache/hudi/pull/10288#issuecomment-1849037426
https://github.com/apache/hudi/assets/513218/d570388c-afe2-48b0-b464-8aae82c2fc47";>
--
This is an automated message from the Apache Git Service.
To respond to the message, please
This is an automated email from the ASF dual-hosted git repository.
sivabalan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/master by this push:
new d91ad84230c [HUDI-7206] Fixing auto deletion of
nsivabalan merged PR #10292:
URL: https://github.com/apache/hudi/pull/10292
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apa
hudi-bot commented on PR #10153:
URL: https://github.com/apache/hudi/pull/10153#issuecomment-1849027683
## CI report:
* 76e1d519c42565b99b35837c3c767b600cdee311 Azure:
[FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21
hudi-bot commented on PR #10242:
URL: https://github.com/apache/hudi/pull/10242#issuecomment-1848999290
## CI report:
* 89db390bfa9a38d13a08ba3647a1c22097d9c179 Azure:
[CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2
hudi-bot commented on PR #10242:
URL: https://github.com/apache/hudi/pull/10242#issuecomment-1848997553
## CI report:
* 89db390bfa9a38d13a08ba3647a1c22097d9c179 Azure:
[CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2
wecharyu commented on code in PR #10242:
URL: https://github.com/apache/hudi/pull/10242#discussion_r1421770295
##
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/TimelineUtils.java:
##
@@ -82,22 +84,40 @@ public static List
getWrittenPartitions(HoodieTimeline ti
hudi-bot commented on PR #10153:
URL: https://github.com/apache/hudi/pull/10153#issuecomment-1848982595
## CI report:
* 4054985bed685d995f568217400a7f66bf591a72 Azure:
[FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21
hudi-bot commented on PR #10153:
URL: https://github.com/apache/hudi/pull/10153#issuecomment-1848980984
## CI report:
* 4054985bed685d995f568217400a7f66bf591a72 Azure:
[FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21
seethb commented on issue #10181:
URL: https://github.com/apache/hudi/issues/10181#issuecomment-1848905116
Also @ad1happy2go we are not using PG (Postgres as the source) and the DB is
YuagabyteDB however the CDC mechanism between YugabyteDB and PG is slightly
different. here we use neither
hudi-bot commented on PR #10153:
URL: https://github.com/apache/hudi/pull/10153#issuecomment-1848897427
## CI report:
* 4054985bed685d995f568217400a7f66bf591a72 Azure:
[FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21
76 matches
Mail list logo
