[GitHub] [hudi] novisfff commented on issue #6444: [SUPPORT] Timeline Service MarkerDirState thread safe issue

2022-08-18 Thread GitBox
novisfff commented on issue #6444: URL: https://github.com/apache/hudi/issues/6444#issuecomment-1220320665 > @novisfff Is this a new marker creation issue at the timeline server after #6383 is landed? yes, this may caused by `allMarkers` in `MarkerDirState` -- This is an automated

[GitHub] [hudi] junyuc25 commented on pull request #6431: Shutdown CloudWatch reporter when query completes

2022-08-18 Thread GitBox
junyuc25 commented on PR #6431: URL: https://github.com/apache/hudi/pull/6431#issuecomment-1220319146 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] yihua commented on issue #6444: [SUPPORT] Timeline Service MarkerDirState thread safe issue

2022-08-18 Thread GitBox
yihua commented on issue #6444: URL: https://github.com/apache/hudi/issues/6444#issuecomment-1220312682 @novisfff Is this a new marker creation issue at the timeline server after #6383 is landed? -- This is an automated message from the Apache Git Service. To respond to the message, pleas

[jira] [Created] (HUDI-4646) mvn package fails w/ flink streaming java maven aartifact not found

2022-08-18 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-4646: - Summary: mvn package fails w/ flink streaming java maven aartifact not found Key: HUDI-4646 URL: https://issues.apache.org/jira/browse/HUDI-4646 Project: Ap

[GitHub] [hudi] prasannarajaperumal merged pull request #6447: [DOCS] More improvements to spec

2022-08-18 Thread GitBox
prasannarajaperumal merged PR #6447: URL: https://github.com/apache/hudi/pull/6447 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@h

[hudi] branch asf-site updated: [DOCS] More improvements to spec - Removing the draft status - Added table/details around table versioning/compatibility - 1 missing point on reader expectations. -

2022-08-18 Thread prasanna
This is an automated email from the ASF dual-hosted git repository. prasanna pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 3c93f9c636 [DOCS] More improvements to spec

[jira] [Updated] (HUDI-1158) Optimizations in parallelized listing behaviour for markers and bootstrap source files

2022-08-18 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-1158: Epic Link: HUDI-1265 > Optimizations in parallelized listing behaviour for markers and bootstrap > source f

[jira] [Updated] (HUDI-1369) Bootstrap datasource jobs from hanging via spark-submit

2022-08-18 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-1369: Epic Link: HUDI-1265 > Bootstrap datasource jobs from hanging via spark-submit > ---

[GitHub] [hudi] codope commented on a diff in pull request #6438: [HUDI-4642] Adding support to hudi-cli to repair depcrated partition

2022-08-18 Thread GitBox
codope commented on code in PR #6438: URL: https://github.com/apache/hudi/pull/6438#discussion_r949841206 ## hudi-cli/src/main/java/org/apache/hudi/cli/commands/RepairsCommand.java: ## @@ -263,4 +264,30 @@ public String migratePartitionMeta( HoodieTableHeaderFields.HEAD

[GitHub] [hudi] vinothchandar opened a new pull request, #6447: [DOCS] More improvements to spec

2022-08-18 Thread GitBox
vinothchandar opened a new pull request, #6447: URL: https://github.com/apache/hudi/pull/6447 - Removing the draft status - Added table/details around table versioning/compatibility - 1 missing point on reader expectations. - Explain table configs and the hoodie.properties file!

[jira] [Updated] (HUDI-2071) Support Reading Bootstrap MOR RT Table In Spark DataSource Table

2022-08-18 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-2071: Epic Link: HUDI-1265 (was: HUDI-1297) > Support Reading Bootstrap MOR RT Table In Spark DataSource Table >

[jira] [Updated] (HUDI-3122) presto query failed for bootstrap tables

2022-08-18 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-3122: Epic Link: HUDI-1265 > presto query failed for bootstrap tables > >

[jira] [Updated] (HUDI-4642) Add hudi-cli support to repair deprecated partition

2022-08-18 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4642: -- Epic Link: HUDI-1388 > Add hudi-cli support to repair deprecated partition > ---

[jira] [Updated] (HUDI-1265) Improving Bootstrap with metadata table

2022-08-18 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-1265: Epic Name: Improving Bootstrap (was: Improve table bootstrap) > Improving Bootstrap with metadata table > -

[jira] [Updated] (HUDI-1265) Improving Bootstrap with metadata table

2022-08-18 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-1265: Summary: Improving Bootstrap with metadata table (was: Followup Tasks for Bootstrap) > Improving Bootstrap

[GitHub] [hudi] hudi-bot commented on pull request #6443: [HUDI-4643] MergeInto syntax WHEN MATCHED is optional but must be set

2022-08-18 Thread GitBox
hudi-bot commented on PR #6443: URL: https://github.com/apache/hudi/pull/6443#issuecomment-1220294045 ## CI report: * 8e5db16fd43d95d376bd58c11568bd3c70b47acf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1081

[jira] [Updated] (HUDI-1265) Followup Tasks for Bootstrap

2022-08-18 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-1265: Epic Name: Improve table bootstrap (was: Improve bootstrap) > Followup Tasks for Bootstrap > --

[GitHub] [hudi] prasannarajaperumal commented on a diff in pull request #6256: [RFC-51][HUDI-3478] Update RFC: CDC support

2022-08-18 Thread GitBox
prasannarajaperumal commented on code in PR #6256: URL: https://github.com/apache/hudi/pull/6256#discussion_r949833823 ## rfc/rfc-51/rfc-51.md: ## @@ -148,20 +152,27 @@ hudi_cdc_table/ Under a partition directory, the `.log` file with `CDCBlock` above will keep the changing

[GitHub] [hudi] hudi-bot commented on pull request #6445: [HUDI-4644] Change default flink profile to 1.15.x

2022-08-18 Thread GitBox
hudi-bot commented on PR #6445: URL: https://github.com/apache/hudi/pull/6445#issuecomment-1220290870 ## CI report: * 4251b68ce29a78a88f5355fb106bf5ca9444fec1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1081

[GitHub] [hudi] hudi-bot commented on pull request #6443: [HUDI-4643] MergeInto syntax WHEN MATCHED is optional but must be set

2022-08-18 Thread GitBox
hudi-bot commented on PR #6443: URL: https://github.com/apache/hudi/pull/6443#issuecomment-1220290852 ## CI report: * 8e5db16fd43d95d376bd58c11568bd3c70b47acf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1081

[jira] [Updated] (HUDI-992) For hive-style partitioned source data, partition columns synced with Hive will always have String type

2022-08-18 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-992: --- Epic Link: HUDI-1265 (was: HUDI-2519) > For hive-style partitioned source data, partition columns synced with

[jira] [Updated] (HUDI-4453) Support partition pruning for tables Bootstrapped from Source Hive Style partitioned tables

2022-08-18 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-4453: Epic Link: HUDI-1265 > Support partition pruning for tables Bootstrapped from Source Hive Style > partition

[jira] [Updated] (HUDI-1779) Fail to bootstrap/upsert a table which contains timestamp column

2022-08-18 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-1779: Epic Link: HUDI-1265 > Fail to bootstrap/upsert a table which contains timestamp column > --

[jira] [Updated] (HUDI-1265) Followup Tasks for Bootstrap

2022-08-18 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-1265: Fix Version/s: 0.13.0 > Followup Tasks for Bootstrap > > > Key:

[jira] [Updated] (HUDI-4125) Add integration tests around bootstrapped Hudi table

2022-08-18 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-4125: Epic Link: HUDI-1265 (was: HUDI-3303) > Add integration tests around bootstrapped Hudi table >

[jira] [Assigned] (HUDI-1265) Followup Tasks for Bootstrap

2022-08-18 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo reassigned HUDI-1265: --- Assignee: Ethan Guo (was: Udit Mehrotra) > Followup Tasks for Bootstrap > --

[jira] [Commented] (HUDI-4632) Remove the force active property for flink1.14 profile

2022-08-18 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581675#comment-17581675 ] Sagar Sumit commented on HUDI-4632: --- Fix under review https://github.com/apache/hudi/pul

[jira] [Reopened] (HUDI-4632) Remove the force active property for flink1.14 profile

2022-08-18 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit reopened HUDI-4632: --- Reopening due to [https://github.com/apache/hudi/issues/6422] Build fails for default profile after remov

[GitHub] [hudi] hudi-bot commented on pull request #6443: [HUDI-4643] MergeInto syntax WHEN MATCHED is optional but must be set

2022-08-18 Thread GitBox
hudi-bot commented on PR #6443: URL: https://github.com/apache/hudi/pull/6443#issuecomment-1220256737 ## CI report: * 8e5db16fd43d95d376bd58c11568bd3c70b47acf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1081

[GitHub] [hudi] hudi-bot commented on pull request #6419: [HUDI-2057] CTAS Generate An External Table When Create Managed Table

2022-08-18 Thread GitBox
hudi-bot commented on PR #6419: URL: https://github.com/apache/hudi/pull/6419#issuecomment-1220256551 ## CI report: * 8b8e05269c88a057840b743cc60f9163b2348db2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1081

[GitHub] [hudi] hudi-bot commented on pull request #6419: [HUDI-2057] CTAS Generate An External Table When Create Managed Table

2022-08-18 Thread GitBox
hudi-bot commented on PR #6419: URL: https://github.com/apache/hudi/pull/6419#issuecomment-1220251702 ## CI report: * 8b8e05269c88a057840b743cc60f9163b2348db2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1081

[GitHub] [hudi] hudi-bot commented on pull request #6446: [HUDI-4645] Flink batch incremental changelog query

2022-08-18 Thread GitBox
hudi-bot commented on PR #6446: URL: https://github.com/apache/hudi/pull/6446#issuecomment-1220249418 ## CI report: * 579b8e20441d54b2eb672218f8e7dbd1d0a1ebaf Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1082

[GitHub] [hudi] hudi-bot commented on pull request #6442: [HUDI-4449] Support DataSourceV2 Read for Spark3.2

2022-08-18 Thread GitBox
hudi-bot commented on PR #6442: URL: https://github.com/apache/hudi/pull/6442#issuecomment-1220249387 ## CI report: * ecc3e06795ac1f4c6439abb9fa53b666d2f5a8eb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1081

[GitHub] [hudi] nsivabalan commented on issue #5875: [SUPPORT] Hoodie Delta streamer Job with Kafka Source fetching the same offset again and again Commiting the same offset again and again

2022-08-18 Thread GitBox
nsivabalan commented on issue #5875: URL: https://github.com/apache/hudi/issues/5875#issuecomment-1220238494 as I pointed out in the other github issue, if kafka checkpoint type is timestamp, we don't support auto resetting to LATEST yet. -- This is an automated message from the Apache G

[GitHub] [hudi] nsivabalan commented on issue #5822: Hudi Clustering not working

2022-08-18 Thread GitBox
nsivabalan commented on issue #5822: URL: https://github.com/apache/hudi/issues/5822#issuecomment-1220237957 thanks!. curious to know what was missing or how did you resolve. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] nsivabalan closed issue #5822: Hudi Clustering not working

2022-08-18 Thread GitBox
nsivabalan closed issue #5822: Hudi Clustering not working URL: https://github.com/apache/hudi/issues/5822 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mai

[GitHub] [hudi] hudi-bot commented on pull request #6431: Shutdown CloudWatch reporter when query completes

2022-08-18 Thread GitBox
hudi-bot commented on PR #6431: URL: https://github.com/apache/hudi/pull/6431#issuecomment-1220222643 ## CI report: * Unknown: [CANCELED](TBD) * 435e3afbae2f62db1e661ad46aabb50df421049c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039

[GitHub] [hudi] hudi-bot commented on pull request #6446: [HUDI-4645] Flink batch incremental changelog query

2022-08-18 Thread GitBox
hudi-bot commented on PR #6446: URL: https://github.com/apache/hudi/pull/6446#issuecomment-1220222675 ## CI report: * 579b8e20441d54b2eb672218f8e7dbd1d0a1ebaf UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[jira] [Updated] (HUDI-4645) Flink batch incremental changelog query

2022-08-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4645: - Labels: pull-request-available (was: ) > Flink batch incremental changelog query > -

[GitHub] [hudi] flashJd opened a new pull request, #6446: [HUDI-4645] Flink batch incremental changelog query

2022-08-18 Thread GitBox
flashJd opened a new pull request, #6446: URL: https://github.com/apache/hudi/pull/6446 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any performance

[GitHub] [hudi] hudi-bot commented on pull request #6431: Shutdown CloudWatch reporter when query completes

2022-08-18 Thread GitBox
hudi-bot commented on PR #6431: URL: https://github.com/apache/hudi/pull/6431#issuecomment-1220221073 ## CI report: * Unknown: [CANCELED](TBD) * 435e3afbae2f62db1e661ad46aabb50df421049c UNKNOWN Bot commands @hudi-bot supports the following commands: - `

[jira] [Created] (HUDI-4645) Flink batch incremental changelog query

2022-08-18 Thread yonghua jian (Jira)
yonghua jian created HUDI-4645: -- Summary: Flink batch incremental changelog query Key: HUDI-4645 URL: https://issues.apache.org/jira/browse/HUDI-4645 Project: Apache Hudi Issue Type: Improvemen

[GitHub] [hudi] hudi-bot commented on pull request #6445: [HUDI-4644] Change default flink profile to 1.15.x

2022-08-18 Thread GitBox
hudi-bot commented on PR #6445: URL: https://github.com/apache/hudi/pull/6445#issuecomment-1220221113 ## CI report: * 4251b68ce29a78a88f5355fb106bf5ca9444fec1 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1081

[GitHub] [hudi] junyuc25 commented on pull request #6431: Shutdown CloudWatch reporter when query completes

2022-08-18 Thread GitBox
junyuc25 commented on PR #6431: URL: https://github.com/apache/hudi/pull/6431#issuecomment-1220219273 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] hudi-bot commented on pull request #6445: [HUDI-4644] Change default flink profile to 1.15.x

2022-08-18 Thread GitBox
hudi-bot commented on PR #6445: URL: https://github.com/apache/hudi/pull/6445#issuecomment-1220219205 ## CI report: * 4251b68ce29a78a88f5355fb106bf5ca9444fec1 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #6443: [HUDI-4643] MergeInto syntax WHEN MATCHED is optional but must be set

2022-08-18 Thread GitBox
hudi-bot commented on PR #6443: URL: https://github.com/apache/hudi/pull/6443#issuecomment-1220217068 ## CI report: * 8e5db16fd43d95d376bd58c11568bd3c70b47acf Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1081

[GitHub] [hudi] hudi-bot commented on pull request #6431: Shutdown CloudWatch reporter when query completes

2022-08-18 Thread GitBox
hudi-bot commented on PR #6431: URL: https://github.com/apache/hudi/pull/6431#issuecomment-1220217036 ## CI report: * 0e37748dafc19e6764a2cad903d8938ac3266961 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1081

[GitHub] [hudi] hudi-bot commented on pull request #6419: [HUDI-2057] CTAS Generate An External Table When Create Managed Table

2022-08-18 Thread GitBox
hudi-bot commented on PR #6419: URL: https://github.com/apache/hudi/pull/6419#issuecomment-1220217014 ## CI report: * 8b8e05269c88a057840b743cc60f9163b2348db2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1081

[GitHub] [hudi] amit-ranjan-de commented on issue #5916: [SUPPORT] `show fsview latest` throwing IllegalStateException...pending compactions for merge_on_read table

2022-08-18 Thread GitBox
amit-ranjan-de commented on issue #5916: URL: https://github.com/apache/hudi/issues/5916#issuecomment-1220212653 Hi @minihippo, in my case I don't have the `20220615161228.compaction` or `20220615172826.compaction` files. -- This is an automated message from the Apache Git Service. To res

[GitHub] [hudi] danny0405 commented on pull request #6437: [HUDI-4632] Map Flink artifact ids correctly as per scala dependence

2022-08-18 Thread GitBox
danny0405 commented on PR #6437: URL: https://github.com/apache/hudi/pull/6437#issuecomment-1220208442 Thanks, i have fired a fix in https://github.com/apache/hudi/pull/6445/files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[jira] [Updated] (HUDI-4644) Change default flink profile to 1.15.x

2022-08-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4644: - Labels: pull-request-available (was: ) > Change default flink profile to 1.15.x > ---

[GitHub] [hudi] danny0405 opened a new pull request, #6445: [HUDI-4644] Change default flink profile to 1.15.x

2022-08-18 Thread GitBox
danny0405 opened a new pull request, #6445: URL: https://github.com/apache/hudi/pull/6445 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any performanc

[jira] [Created] (HUDI-4644) Change default flink profile to 1.15.x

2022-08-18 Thread Danny Chen (Jira)
Danny Chen created HUDI-4644: Summary: Change default flink profile to 1.15.x Key: HUDI-4644 URL: https://issues.apache.org/jira/browse/HUDI-4644 Project: Apache Hudi Issue Type: Task C

[GitHub] [hudi] novisfff opened a new issue, #6444: [SUPPORT] Timeline Service MarkerDirState thread safe issue

2022-08-18 Thread GitBox
novisfff opened a new issue, #6444: URL: https://github.com/apache/hudi/issues/6444 **_Tips before filing an issue_** - Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)? - Join the mailing list to engage in conversations and get faster support at dev-subscr.

[jira] [Closed] (HUDI-4013) Document all manual tests done as part of 0.11 release certification

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-4013. Resolution: Won't Do > Document all manual tests done as part of 0.11 release certification > -

[jira] [Updated] (HUDI-4013) Document all manual tests done as part of 0.11 release certification

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4013: - Fix Version/s: (was: 0.12.1) > Document all manual tests done as part of 0.11 release certification >

[jira] [Closed] (HUDI-4272) Auto detect partitions without PartitionValueExtractor

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-4272. Resolution: Won't Do Abandoned due to change of plan: using config inference instead > Auto detect partitio

[jira] [Updated] (HUDI-4013) Document all manual tests done as part of 0.11 release certification

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4013: - Issue Type: Task (was: Improvement) > Document all manual tests done as part of 0.11 release certificatio

[jira] [Updated] (HUDI-4012) Evaluate support for spark2 and scala12

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4012: - Issue Type: Task (was: Improvement) > Evaluate support for spark2 and scala12 > -

[GitHub] [hudi] dongkelun commented on issue #6410: [SUPPORT] MergeInto syntax WHEN MATCHED is not optional but must be set.

2022-08-18 Thread GitBox
dongkelun commented on issue #6410: URL: https://github.com/apache/hudi/issues/6410#issuecomment-1220186095 I created a PR to try to solve this problem:[https://github.com/apache/hudi/pull/6443](https://github.com/apache/hudi/pull/6443) -- This is an automated message from the Apache Git

[jira] [Updated] (HUDI-4012) Evaluate support for spark2 and scala12

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4012: - Fix Version/s: 0.13.0 (was: 0.12.1) > Evaluate support for spark2 and scala12 > ---

[jira] [Updated] (HUDI-4272) Auto detect partitions without PartitionValueExtractor

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4272: - Fix Version/s: (was: 0.12.1) > Auto detect partitions without PartitionValueExtractor > --

[GitHub] [hudi] hudi-bot commented on pull request #6443: [HUDI-4643] MergeInto syntax WHEN MATCHED is optional but must be set

2022-08-18 Thread GitBox
hudi-bot commented on PR #6443: URL: https://github.com/apache/hudi/pull/6443#issuecomment-1220185753 ## CI report: * 8e5db16fd43d95d376bd58c11568bd3c70b47acf UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[jira] [Updated] (HUDI-4486) validate the coming configs and table name when create an external table

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4486: - Fix Version/s: 0.13.0 (was: 0.12.1) > validate the coming configs and table name wh

[jira] [Updated] (HUDI-4457) Make sure IT docker test return code non-zero when failed

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4457: - Priority: Critical (was: Major) > Make sure IT docker test return code non-zero when failed > ---

[jira] [Updated] (HUDI-3787) Implement redis index

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3787: - Fix Version/s: 0.13.0 (was: 0.12.1) > Implement redis index > -

[jira] [Updated] (HUDI-4483) Fix checkstyle on scala code and integ-test module

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4483: - Fix Version/s: 0.13.0 (was: 0.12.1) > Fix checkstyle on scala code and integ-test m

[jira] [Updated] (HUDI-3545) Make HoodieAvroWriteSupport class configurable

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3545: - Fix Version/s: 0.13.0 (was: 0.12.1) > Make HoodieAvroWriteSupport class configurabl

[jira] [Updated] (HUDI-4643) MergeInto syntax WHEN MATCHED is optional but must be set

2022-08-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4643: - Labels: pull-request-available (was: ) > MergeInto syntax WHEN MATCHED is optional but must be se

[jira] [Updated] (HUDI-3554) Support Custom Naming of RO and RT tables for Hudi MOR table

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3554: - Fix Version/s: 0.13.0 (was: 0.12.1) > Support Custom Naming of RO and RT tables for

[jira] [Updated] (HUDI-3555) re-use spark config for parquet timestamp format instead of having our own config

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3555: - Fix Version/s: 0.13.0 (was: 0.12.1) > re-use spark config for parquet timestamp for

[GitHub] [hudi] dongkelun opened a new pull request, #6443: [HUDI-4643] MergeInto syntax WHEN MATCHED is optional but must be set

2022-08-18 Thread GitBox
dongkelun opened a new pull request, #6443: URL: https://github.com/apache/hudi/pull/6443 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any performanc

[jira] [Updated] (HUDI-4575) Initial Kafka Global Offsets in Hudi Kafka Sink Connector

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4575: - Fix Version/s: 0.13.0 (was: 0.12.1) > Initial Kafka Global Offsets in Hudi Kafka S

[GitHub] [hudi] KnightChess commented on issue #6410: [SUPPORT] MergeInto syntax WHEN MATCHED is not optional but must be set.

2022-08-18 Thread GitBox
KnightChess commented on issue #6410: URL: https://github.com/apache/hudi/issues/6410#issuecomment-1220182156 yeah, look like `WHEN MATCHED` is required in the implementation of hudi code -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[jira] [Updated] (HUDI-3531) Review and shade transitive dependencies in hudi bundle jar

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3531: - Priority: Critical (was: Major) > Review and shade transitive dependencies in hudi bundle jar > -

[jira] [Updated] (HUDI-3531) Review and shade transitive dependencies in hudi bundle jar

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3531: - Fix Version/s: 0.13.0 (was: 0.12.1) > Review and shade transitive dependencies in h

[jira] [Updated] (HUDI-3599) Not atomicity commit could cause streaming read loss data

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3599: - Priority: Critical (was: Major) > Not atomicity commit could cause streaming read loss data > ---

[jira] [Updated] (HUDI-4643) MergeInto syntax WHEN MATCHED is optional but must be set

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4643: - Fix Version/s: 0.13.0 (was: 0.12.1) > MergeInto syntax WHEN MATCHED is optional but

[jira] [Updated] (HUDI-3314) support merge into with no-pk condition

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3314: - Fix Version/s: 0.13.0 (was: 0.12.1) > support merge into with no-pk condition > ---

[jira] [Assigned] (HUDI-3545) Make HoodieAvroWriteSupport class configurable

2022-08-18 Thread Surya Prasanna Yalla (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surya Prasanna Yalla reassigned HUDI-3545: -- Assignee: Surya Prasanna Yalla > Make HoodieAvroWriteSupport class configurable

[GitHub] [hudi] hudi-bot commented on pull request #6442: [HUDI-4449] Support DataSourceV2 Read for Spark3.2

2022-08-18 Thread GitBox
hudi-bot commented on PR #6442: URL: https://github.com/apache/hudi/pull/6442#issuecomment-1220179204 ## CI report: * ecc3e06795ac1f4c6439abb9fa53b666d2f5a8eb Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1081

[GitHub] [hudi] KnightChess commented on issue #6405: [SUPPORT] Using SparkSQL DDL To Create External Error.

2022-08-18 Thread GitBox
KnightChess commented on issue #6405: URL: https://github.com/apache/hudi/issues/6405#issuecomment-1220179037 skip step 1 from then `Stacktrace`, look like something qe in config, make sure you have set config `spark.sql.extensions` which value is `org.apache.spark.sql.hudi.HoodieSparkSe

[jira] [Updated] (HUDI-3416) Incremental read using v2 datasource

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3416: - Fix Version/s: 0.13.0 (was: 0.12.1) > Incremental read using v2 datasource > --

[jira] [Updated] (HUDI-3527) Move metrics classes to hudi-metrics module

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3527: - Fix Version/s: 0.13.0 (was: 0.12.1) > Move metrics classes to hudi-metrics module >

[jira] [Updated] (HUDI-3742) Enable parquet enableVectorizedReader for spark incremental read to prevent pef regression

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3742: - Priority: Critical (was: Major) > Enable parquet enableVectorizedReader for spark incremental read to pr

[jira] [Updated] (HUDI-3657) Unbound the restriction that clean retain commits must be smaller than archive minimum commits

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3657: - Fix Version/s: 0.13.0 (was: 0.12.1) > Unbound the restriction that clean retain com

[jira] [Updated] (HUDI-3856) Upgrade maven surefire to M5

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3856: - Fix Version/s: 0.13.0 (was: 0.12.1) > Upgrade maven surefire to M5 > --

[jira] [Updated] (HUDI-4026) Add support for spark streaming writes to integ test framework

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4026: - Fix Version/s: 0.13.0 (was: 0.12.1) > Add support for spark streaming writes to int

[GitHub] [hudi] danny0405 commented on a diff in pull request #6437: [HUDI-4632] Map Flink artifact ids correctly as per scala dependence

2022-08-18 Thread GitBox
danny0405 commented on code in PR #6437: URL: https://github.com/apache/hudi/pull/6437#discussion_r949747182 ## pom.xml: ## @@ -1823,7 +1823,9 @@ flink1.15 ${flink1.15.version} +flink-runtime flink-table-runtime + flink-table-pla

[jira] [Updated] (HUDI-4248) Upgrade Apache Avro version for hudi-flink

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4248: - Component/s: flink > Upgrade Apache Avro version for hudi-flink >

[jira] [Closed] (HUDI-4317) hudi utilities bundle fails/skips to build w/ azure CI and eventually test fails

2022-08-18 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-4317. - Fix Version/s: 0.12.0 (was: 0.12.1) Resolution: Fixed > hudi

[jira] [Updated] (HUDI-4289) Flink sql write to hudi,set the write operation to upsert,There are duplicate data in the hudi partition table.

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4289: - Fix Version/s: 0.13.0 (was: 0.12.1) > Flink sql write to hudi,set the write operati

[jira] [Updated] (HUDI-4289) Flink sql write to hudi,set the write operation to upsert,There are duplicate data in the hudi partition table.

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4289: - Component/s: flink-sql > Flink sql write to hudi,set the write operation to upsert,There are duplicate >

[jira] [Updated] (HUDI-4317) hudi utilities bundle fails/skips to build w/ azure CI and eventually test fails

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4317: - Fix Version/s: 0.12.1 > hudi utilities bundle fails/skips to build w/ azure CI and eventually test > fail

[jira] [Updated] (HUDI-4317) hudi utilities bundle fails/skips to build w/ azure CI and eventually test fails

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4317: - Priority: Critical (was: Major) > hudi utilities bundle fails/skips to build w/ azure CI and eventually t

[jira] [Updated] (HUDI-4283) The memory cost in HoodieMergeHandle is increasing as the size of base file

2022-08-18 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4283: -- Priority: Critical (was: Major) > The memory cost in HoodieMergeHandle is increasing as

[jira] [Updated] (HUDI-4289) Flink sql write to hudi,set the write operation to upsert,There are duplicate data in the hudi partition table.

2022-08-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4289: - Component/s: flink > Flink sql write to hudi,set the write operation to upsert,There are duplicate > data

[jira] [Updated] (HUDI-4283) The memory cost in HoodieMergeHandle is increasing as the size of base file

2022-08-18 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4283: -- Fix Version/s: 0.12.1 > The memory cost in HoodieMergeHandle is increasing as the size o

[jira] [Updated] (HUDI-4289) Flink sql write to hudi,set the write operation to upsert,There are duplicate data in the hudi partition table.

2022-08-18 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4289: -- Fix Version/s: 0.12.1 > Flink sql write to hudi,set the write operation to upsert,There

[jira] [Updated] (HUDI-4282) Throws IOException in method HoodieLogFileReader.isBlockCorrupted()

2022-08-18 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4282: -- Priority: Critical (was: Major) > Throws IOException in method HoodieLogFileReader.isBl

  1   2   3   4   >