[GitHub] [hudi] hudi-bot commented on pull request #7039: [HUDI-5080] Fixing unpersist to consider only rdds pertaining to current write operation

hudi-bot commented on PR #7039: URL: https://github.com/apache/hudi/pull/7039#issuecomment-1293071587 ## CI report: * 9ffc15c02a719adff1492b6d623873fc9ac8040b Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=124

[GitHub] [hudi] hudi-bot commented on pull request #7074: [HUDI-5101] Adding spark-structured streaming test support via spark-submit job

hudi-bot commented on PR #7074: URL: https://github.com/apache/hudi/pull/7074#issuecomment-1293067040 ## CI report: * 51dd40cb8d37331fb7737d707958e26ea57ac5b2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1261

[GitHub] [hudi] hudi-bot commented on pull request #7039: [HUDI-5080] Fixing unpersist to consider only rdds pertaining to current write operation

hudi-bot commented on PR #7039: URL: https://github.com/apache/hudi/pull/7039#issuecomment-1293066924 ## CI report: * 9ffc15c02a719adff1492b6d623873fc9ac8040b Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=124

[GitHub] [hudi] hudi-bot commented on pull request #7011: [HUDI-5102] source operator(monitor and reader) support user uid

hudi-bot commented on PR #7011: URL: https://github.com/apache/hudi/pull/7011#issuecomment-1293062040 ## CI report: * 7268dd60d09a60ac84761a7e177306a9d2b3c576 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1261

[GitHub] [hudi] waywtdcc commented on a diff in pull request #7056: [HUDI-5088]Fix bug:Failed to synchronize the hive metadata of the Flink table

waywtdcc commented on code in PR #7056: URL: https://github.com/apache/hudi/pull/7056#discussion_r1006430155 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/catalog/HiveSchemaUtils.java: ## @@ -177,10 +180,19 @@ private static DataType toFlinkPrimitiveTy

[jira] [Updated] (HUDI-5088) Failed to synchronize the hive metadata of the Flink table

[ https://issues.apache.org/jira/browse/HUDI-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] waywtdcc updated HUDI-5088: --- Description: The flash table failed to synchronize the hive metadata. After using the flash catalog and speci

[jira] [Updated] (HUDI-5088) Failed to synchronize the hive metadata of the Flink table

[ https://issues.apache.org/jira/browse/HUDI-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] waywtdcc updated HUDI-5088: --- Attachment: image-2022-10-27-13-51-06-216.png > Failed to synchronize the hive metadata of the Flink table > -

[GitHub] [hudi] waywtdcc commented on a diff in pull request #7056: [HUDI-5088]Fix bug:Failed to synchronize the hive metadata of the Flink table

waywtdcc commented on code in PR #7056: URL: https://github.com/apache/hudi/pull/7056#discussion_r1006426816 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/catalog/HiveSchemaUtils.java: ## @@ -177,10 +180,19 @@ private static DataType toFlinkPrimitiveTy

[GitHub] [hudi] waywtdcc commented on a diff in pull request #7075: [HUDI-5100][flink]Support writing tasks independently in the flink batch mode

waywtdcc commented on code in PR #7075: URL: https://github.com/apache/hudi/pull/7075#discussion_r1006419928 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/meta/CkpMetadata.java: ## @@ -132,6 +132,20 @@ private void clean(String newInstant) { } }

[GitHub] [hudi] waywtdcc commented on a diff in pull request #7075: [HUDI-5100][flink]Support writing tasks independently in the flink batch mode

waywtdcc commented on code in PR #7075: URL: https://github.com/apache/hudi/pull/7075#discussion_r1006419928 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/meta/CkpMetadata.java: ## @@ -132,6 +132,20 @@ private void clean(String newInstant) { } }

[GitHub] [hudi] waywtdcc commented on a diff in pull request #7075: [HUDI-5100][flink]Support writing tasks independently in the flink batch mode

waywtdcc commented on code in PR #7075: URL: https://github.com/apache/hudi/pull/7075#discussion_r1006418655 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/meta/CkpMetadata.java: ## @@ -132,6 +132,20 @@ private void clean(String newInstant) { } }

[GitHub] [hudi] hudi-bot commented on pull request #7068: [HUDI-5096] boolean params is broken in HiveSyncTool

hudi-bot commented on PR #7068: URL: https://github.com/apache/hudi/pull/7068#issuecomment-1293014882 ## CI report: * 3c7dcc118207c8245d9e716d2a779f14cba6fd2c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1259

[GitHub] [hudi] hudi-bot commented on pull request #7026: [MINOR] Properly registering target classes w/ Kryo

hudi-bot commented on PR #7026: URL: https://github.com/apache/hudi/pull/7026#issuecomment-1293014766 ## CI report: * d6461234a99c761cd0b46553939d7140052d9f71 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1257

[GitHub] [hudi] hudi-bot commented on pull request #7068: [HUDI-5096] boolean params is broken in HiveSyncTool

hudi-bot commented on PR #7068: URL: https://github.com/apache/hudi/pull/7068#issuecomment-1293011799 ## CI report: * 3c7dcc118207c8245d9e716d2a779f14cba6fd2c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1259

[GitHub] [hudi] hudi-bot commented on pull request #7026: [MINOR] Properly registering target classes w/ Kryo

hudi-bot commented on PR #7026: URL: https://github.com/apache/hudi/pull/7026#issuecomment-1293011697 ## CI report: * d6461234a99c761cd0b46553939d7140052d9f71 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1257

[GitHub] [hudi] zhangyue19921010 commented on pull request #5416: [HUDI-3963] Use Lock-Free Message Queue Disruptor Improving Hoodie Writing Efficiency

zhangyue19921010 commented on PR #5416: URL: https://github.com/apache/hudi/pull/5416#issuecomment-1293004257 > @zhangyue19921010 please rebase on the latest master. Test failures you observe have already been addressed on master. Hi [alexeykudinkin](https://github.com/alexeykudinkin)

[jira] [Updated] (HUDI-5102) source operator(monitor and reader) support user uid

[ https://issues.apache.org/jira/browse/HUDI-5102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenzhiming updated HUDI-5102: -- Status: Open (was: In Progress) > source operator(monitor and reader) support user uid > --

[jira] [Updated] (HUDI-5102) source operator(monitor and reader) support user uid

[ https://issues.apache.org/jira/browse/HUDI-5102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenzhiming updated HUDI-5102: -- Status: In Progress (was: Open) > source operator(monitor and reader) support user uid > --

[jira] [Updated] (HUDI-5102) source operator(monitor and reader) support user uid

[ https://issues.apache.org/jira/browse/HUDI-5102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenzhiming updated HUDI-5102: -- Component/s: flink-sql Labels: pull-request-available (was: ) > source operator(monitor and re

[jira] [Updated] (HUDI-5102) source operator(monitor and reader) support user uid

[ https://issues.apache.org/jira/browse/HUDI-5102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenzhiming updated HUDI-5102: -- Description: refer to org.apache.hudi.sink.utils.Pipelines#hoodieStreamWrite , add user uid for StreamR

[jira] [Updated] (HUDI-5102) source operator(monitor and reader) support user uid

[ https://issues.apache.org/jira/browse/HUDI-5102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenzhiming updated HUDI-5102: -- Fix Version/s: 0.12.1 > source operator(monitor and reader) support user uid > -

[jira] [Created] (HUDI-5102) source operator(monitor and reader) support user uid

chenzhiming created HUDI-5102: - Summary: source operator(monitor and reader) support user uid Key: HUDI-5102 URL: https://issues.apache.org/jira/browse/HUDI-5102 Project: Apache Hudi Issue Type:

[GitHub] [hudi] nsivabalan commented on issue #6907: [SUPPORT] hoodie commit time format change

nsivabalan commented on issue #6907: URL: https://github.com/apache/hudi/issues/6907#issuecomment-1292967917 I discussed w/ few other hudi experts. We feel this has to be addressed at app layer where commit times are casted to timestamp. we don't have plans to support sec level granularity.

[GitHub] [hudi] nsivabalan closed issue #6907: [SUPPORT] hoodie commit time format change

nsivabalan closed issue #6907: [SUPPORT] hoodie commit time format change URL: https://github.com/apache/hudi/issues/6907 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

[GitHub] [hudi] danny0405 commented on a diff in pull request #7075: [HUDI-5100][flink]Support writing tasks independently in the flink batch mode

danny0405 commented on code in PR #7075: URL: https://github.com/apache/hudi/pull/7075#discussion_r1006392805 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/meta/CkpMetadata.java: ## @@ -132,6 +132,20 @@ private void clean(String newInstant) { } }

[GitHub] [hudi] chenshzh commented on a diff in pull request #7017: [HUDI-5066] Support flink hoodie source metaclient cache

chenshzh commented on code in PR #7017: URL: https://github.com/apache/hudi/pull/7017#discussion_r1006389908 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableSource.java: ## @@ -162,7 +163,7 @@ public HoodieTableSource( this.limit = limit =

[GitHub] [hudi] chenshzh commented on a diff in pull request #7017: [HUDI-5066] Support flink hoodie source metaclient cache

chenshzh commented on code in PR #7017: URL: https://github.com/apache/hudi/pull/7017#discussion_r1006389908 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableSource.java: ## @@ -162,7 +163,7 @@ public HoodieTableSource( this.limit = limit =

[GitHub] [hudi] hudi-bot commented on pull request #6824: [HUDI-4946] fix merge into with no preCombineField has dup row by onl…

hudi-bot commented on PR #6824: URL: https://github.com/apache/hudi/pull/6824#issuecomment-1292944401 ## CI report: * 4340e3a6cff3dde74c22911eafadfe346b95f8cd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1259

[GitHub] [hudi] hudi-bot commented on pull request #7075: [HUDI-5100][flink]Support writing tasks independently in the flink batch mode

hudi-bot commented on PR #7075: URL: https://github.com/apache/hudi/pull/7075#issuecomment-1292941916 ## CI report: * 4e7c3106700d9100a3d91b90995260e6e6fabee2 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1261

[GitHub] [hudi] hudi-bot commented on pull request #6824: [HUDI-4946] fix merge into with no preCombineField has dup row by onl…

hudi-bot commented on PR #6824: URL: https://github.com/apache/hudi/pull/6824#issuecomment-1292941556 ## CI report: * 4340e3a6cff3dde74c22911eafadfe346b95f8cd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1259

[GitHub] [hudi] danny0405 commented on issue #7073: [BUG] Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

danny0405 commented on issue #7073: URL: https://github.com/apache/hudi/issues/7073#issuecomment-1292940168 Hello, why close it then ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] hudi-bot commented on pull request #7075: [HUDI-5100][flink]Support writing tasks independently in the flink batch mode

hudi-bot commented on PR #7075: URL: https://github.com/apache/hudi/pull/7075#issuecomment-1292938809 ## CI report: * 4e7c3106700d9100a3d91b90995260e6e6fabee2 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #6448: [HUDI-4647] Keep the hive sync settings in spark sql consistent

hudi-bot commented on PR #6448: URL: https://github.com/apache/hudi/pull/6448#issuecomment-1292938328 ## CI report: * 36ea5a52752d98eabe02c1b937d77c12e87ababf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1260

[GitHub] [hudi] chenshzh commented on a diff in pull request #5991: [HUDI-4329] Add separate control for Flink compaction operation sync/async mode

chenshzh commented on code in PR #5991: URL: https://github.com/apache/hudi/pull/5991#discussion_r1006371610 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/compact/CompactFunction.java: ## @@ -74,14 +74,14 @@ public class CompactFunction extends ProcessF

[GitHub] [hudi] waywtdcc closed issue #7073: [BUG] Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

waywtdcc closed issue #7073: [BUG] Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written URL: https://github.com/apache/hudi/issues/7073 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[GitHub] [hudi] danny0405 commented on a diff in pull request #7056: [HUDI-5088]Fix bug:Failed to synchronize the hive metadata of the Flink table

danny0405 commented on code in PR #7056: URL: https://github.com/apache/hudi/pull/7056#discussion_r1006366625 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/catalog/HiveSchemaUtils.java: ## @@ -177,10 +180,19 @@ private static DataType toFlinkPrimitiveT

[GitHub] [hudi] danny0405 commented on pull request #7011: source operator(monitor and reader) support user uid

danny0405 commented on PR #7011: URL: https://github.com/apache/hudi/pull/7011#issuecomment-1292916539 [fix_op_uid.patch.zip](https://github.com/apache/hudi/files/9875471/fix_op_uid.patch.zip) Thanks for the contribution, i have reviewed and applied a patch, can you log a JIRA issue he

[GitHub] [hudi] waywtdcc commented on a diff in pull request #7056: [HUDI-5088]Fix bug:Failed to synchronize the hive metadata of the Flink table

waywtdcc commented on code in PR #7056: URL: https://github.com/apache/hudi/pull/7056#discussion_r1006361456 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/catalog/HiveSchemaUtils.java: ## @@ -177,10 +180,19 @@ private static DataType toFlinkPrimitiveTy

[jira] [Updated] (HUDI-5100) Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

[ https://issues.apache.org/jira/browse/HUDI-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] waywtdcc updated HUDI-5100: --- Description: Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be

[jira] [Updated] (HUDI-5100) Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

[ https://issues.apache.org/jira/browse/HUDI-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] waywtdcc updated HUDI-5100: --- Description: Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be

[jira] [Updated] (HUDI-5100) Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

[ https://issues.apache.org/jira/browse/HUDI-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] waywtdcc updated HUDI-5100: --- Description: Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be

[GitHub] [hudi] hudi-bot commented on pull request #6952: [HUDI-5035] Remove usage of deprecated HoodieTimer constructor

hudi-bot commented on PR #6952: URL: https://github.com/apache/hudi/pull/6952#issuecomment-1292902175 ## CI report: * c9cba1c13b3f54438dfa24039571e259aaa6af82 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1260

[jira] [Updated] (HUDI-5100) Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

[ https://issues.apache.org/jira/browse/HUDI-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5100: - Labels: pull-request-available (was: ) > Flink batch is written to hudi. When the number of slots

[GitHub] [hudi] waywtdcc opened a new pull request, #7075: [HUDI-5100][flink]Support writing tasks independently in the flink batch mode

waywtdcc opened a new pull request, #7075: URL: https://github.com/apache/hudi/pull/7075 …tch mode ### Change Logs Support writing tasks independently in the flink batch mode, jira:https://issues.apache.org/jira/browse/HUDI-5100 ### Impact Support writing tasks in

[GitHub] [hudi] hudi-bot commented on pull request #7074: [HUDI-5101] Adding spark-structured streaming test support via spark-submit job

hudi-bot commented on PR #7074: URL: https://github.com/apache/hudi/pull/7074#issuecomment-1292899601 ## CI report: * 51dd40cb8d37331fb7737d707958e26ea57ac5b2 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1261

[GitHub] [hudi] hudi-bot commented on pull request #7071: [HUDI-4982] Upgrade Bundle Testing

hudi-bot commented on PR #7071: URL: https://github.com/apache/hudi/pull/7071#issuecomment-1292899575 ## CI report: * 46a96de62e5f6df954a024ecaf7fa7c30963a146 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1260

[GitHub] [hudi] hudi-bot commented on pull request #7011: source operator(monitor and reader) support user uid

hudi-bot commented on PR #7011: URL: https://github.com/apache/hudi/pull/7011#issuecomment-1292899464 ## CI report: * 4e780b9f2024918093bedfe9aaa9dd9687e5dd0e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1238

[GitHub] [hudi] hudi-bot commented on pull request #7074: [HUDI-5101] Adding spark-structured streaming test support via spark-submit job

hudi-bot commented on PR #7074: URL: https://github.com/apache/hudi/pull/7074#issuecomment-1292896535 ## CI report: * 51dd40cb8d37331fb7737d707958e26ea57ac5b2 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7011: source operator(monitor and reader) support user uid

hudi-bot commented on PR #7011: URL: https://github.com/apache/hudi/pull/7011#issuecomment-1292896402 ## CI report: * 4e780b9f2024918093bedfe9aaa9dd9687e5dd0e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1238

[jira] [Updated] (HUDI-5101) Adding spark structured streaming tests to integ tests

[ https://issues.apache.org/jira/browse/HUDI-5101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5101: - Labels: pull-request-available (was: ) > Adding spark structured streaming tests to integ tests >

[jira] [Updated] (HUDI-5101) Adding spark structured streaming tests to integ tests

[ https://issues.apache.org/jira/browse/HUDI-5101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5101: -- Status: Patch Available (was: In Progress) > Adding spark structured streaming tests to

[jira] [Updated] (HUDI-5101) Adding spark structured streaming tests to integ tests

[ https://issues.apache.org/jira/browse/HUDI-5101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5101: -- Status: In Progress (was: Open) > Adding spark structured streaming tests to integ test

[GitHub] [hudi] nsivabalan opened a new pull request, #7074: [HUDI-5101] Adding spark-structured streaming test support via spark-submit job

nsivabalan opened a new pull request, #7074: URL: https://github.com/apache/hudi/pull/7074 ### Change Logs Adding support to test spark structured streaming job to integ test. Added a spark submit job for ease of testing w/ EKS as well. ### Impact We should be able to t

[GitHub] [hudi] waywtdcc opened a new issue, #7073: [BUG] Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

waywtdcc opened a new issue, #7073: URL: https://github.com/apache/hudi/issues/7073 jira: https://issues.apache.org/jira/browse/HUDI-5100 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[jira] [Updated] (HUDI-5101) Adding spark structured streaming tests to integ tests

[ https://issues.apache.org/jira/browse/HUDI-5101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5101: -- Fix Version/s: 0.12.2 > Adding spark structured streaming tests to integ tests > ---

[jira] [Updated] (HUDI-5101) Adding spark structured streaming tests to integ tests

[ https://issues.apache.org/jira/browse/HUDI-5101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5101: -- Story Points: 3 > Adding spark structured streaming tests to integ tests > -

[jira] [Created] (HUDI-5101) Adding spark structured streaming tests to integ tests

sivabalan narayanan created HUDI-5101: - Summary: Adding spark structured streaming tests to integ tests Key: HUDI-5101 URL: https://issues.apache.org/jira/browse/HUDI-5101 Project: Apache Hudi

[jira] [Updated] (HUDI-5101) Adding spark structured streaming tests to integ tests

[ https://issues.apache.org/jira/browse/HUDI-5101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5101: -- Sprint: 2022/10/18 > Adding spark structured streaming tests to integ tests > --

[jira] [Assigned] (HUDI-5101) Adding spark structured streaming tests to integ tests

[ https://issues.apache.org/jira/browse/HUDI-5101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-5101: - Assignee: sivabalan narayanan > Adding spark structured streaming tests to integ

[GitHub] [hudi] danny0405 commented on a diff in pull request #7017: [HUDI-5066] Support flink hoodie source metaclient cache

danny0405 commented on code in PR #7017: URL: https://github.com/apache/hudi/pull/7017#discussion_r1006349333 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableSource.java: ## @@ -162,7 +163,7 @@ public HoodieTableSource( this.limit = limit

[GitHub] [hudi] 1032851561 commented on a diff in pull request #7011: source operator(monitor and reader) support user uid

1032851561 commented on code in PR #7011: URL: https://github.com/apache/hudi/pull/7011#discussion_r1006346027 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableSource.java: ## @@ -188,9 +188,11 @@ public DataStream produceDataStream(StreamExecu

[jira] [Updated] (HUDI-5100) Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

[ https://issues.apache.org/jira/browse/HUDI-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] waywtdcc updated HUDI-5100: --- Attachment: image-2022-10-27-10-29-17-593.png > Flink batch is written to hudi. When the number of slots is le

[jira] [Updated] (HUDI-5100) Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

[ https://issues.apache.org/jira/browse/HUDI-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] waywtdcc updated HUDI-5100: --- Description: Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be

[jira] [Updated] (HUDI-5100) Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

[ https://issues.apache.org/jira/browse/HUDI-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] waywtdcc updated HUDI-5100: --- Attachment: image-2022-10-27-10-29-10-350.png > Flink batch is written to hudi. When the number of slots is le

[jira] [Updated] (HUDI-5100) Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

[ https://issues.apache.org/jira/browse/HUDI-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] waywtdcc updated HUDI-5100: --- Attachment: (was: image-2022-10-27-10-29-10-350.png) > Flink batch is written to hudi. When the number of

[jira] [Updated] (HUDI-5100) Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

[ https://issues.apache.org/jira/browse/HUDI-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] waywtdcc updated HUDI-5100: --- Description: Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be

[GitHub] [hudi] danny0405 commented on issue #6984: [SUPPORT] hudi metrics with flink so little

danny0405 commented on issue #6984: URL: https://github.com/apache/hudi/issues/6984#issuecomment-1292880663 Yeah, we are pushing forward the metrics integration for flink though, hope we can strength it in release 0.13.0. -- This is an automated message from the Apache Git Service. To res

[jira] [Updated] (HUDI-5100) Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

[ https://issues.apache.org/jira/browse/HUDI-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] waywtdcc updated HUDI-5100: --- Attachment: image-2022-10-27-10-24-14-374.png > Flink batch is written to hudi. When the number of slots is le

[GitHub] [hudi] danny0405 commented on a diff in pull request #5991: [HUDI-4329] Add separate control for Flink compaction operation sync/async mode

danny0405 commented on code in PR #5991: URL: https://github.com/apache/hudi/pull/5991#discussion_r1006343669 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/compact/CompactFunction.java: ## @@ -74,14 +74,14 @@ public class CompactFunction extends Process

[GitHub] [hudi] danny0405 commented on a diff in pull request #7056: [HUDI-5088]Fix bug:Failed to synchronize the hive metadata of the Flink table

danny0405 commented on code in PR #7056: URL: https://github.com/apache/hudi/pull/7056#discussion_r1006339111 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/catalog/HiveSchemaUtils.java: ## @@ -177,10 +180,19 @@ private static DataType toFlinkPrimitiveT

[GitHub] [hudi] navbalaraman commented on issue #7060: Error when upgrading to hudi 0.12.0 from 0.9.0

navbalaraman commented on issue #7060: URL: https://github.com/apache/hudi/issues/7060#issuecomment-1292870879 Thanks @namuny I updated the pom reference as below but still getting the same error. org.apache.hudi hudi-spark3.1-bundle_2.12 0.12.0

[GitHub] [hudi] danny0405 commented on a diff in pull request #7056: [HUDI-5088]Fix bug:Failed to synchronize the hive metadata of the Flink table

danny0405 commented on code in PR #7056: URL: https://github.com/apache/hudi/pull/7056#discussion_r1006335372 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/catalog/HiveSchemaUtils.java: ## @@ -177,10 +180,19 @@ private static DataType toFlinkPrimitiveT

[GitHub] [hudi] danny0405 commented on a diff in pull request #7011: source operator(monitor and reader) support user uid

danny0405 commented on code in PR #7011: URL: https://github.com/apache/hudi/pull/7011#discussion_r1006333162 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableSource.java: ## @@ -188,9 +188,11 @@ public DataStream produceDataStream(StreamExecut

[GitHub] [hudi] danny0405 commented on pull request #7065: [HUDI-5090] throw runtime Exception when flink streming job checkpoint abort

danny0405 commented on PR #7065: URL: https://github.com/apache/hudi/pull/7065#issuecomment-1292864207 > #6796 Yeah, let's move to https://github.com/apache/hudi/pull/6796 and go on with the discussion. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hudi] hudi-bot commented on pull request #7052: [MINOR] Make sure Dictionary Encoding in Parquet enabled by default

hudi-bot commented on PR #7052: URL: https://github.com/apache/hudi/pull/7052#issuecomment-1292853249 ## CI report: * 2e5cc087542cd1ae853b3fb477cc9358e7bde36a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1254

[GitHub] [hudi] hudi-bot commented on pull request #6448: [HUDI-4647] Keep the hive sync settings in spark sql consistent

hudi-bot commented on PR #6448: URL: https://github.com/apache/hudi/pull/6448#issuecomment-1292852742 ## CI report: * 1767369a28a245c7d28593aee7eb4f51459dbbf8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1259

[GitHub] [hudi] hudi-bot commented on pull request #7052: [MINOR] Make sure Dictionary Encoding in Parquet enabled by default

hudi-bot commented on PR #7052: URL: https://github.com/apache/hudi/pull/7052#issuecomment-1292850313 ## CI report: * 2e5cc087542cd1ae853b3fb477cc9358e7bde36a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1254

[GitHub] [hudi] hudi-bot commented on pull request #6448: [HUDI-4647] Keep the hive sync settings in spark sql consistent

hudi-bot commented on PR #6448: URL: https://github.com/apache/hudi/pull/6448#issuecomment-1292849538 ## CI report: * 1767369a28a245c7d28593aee7eb4f51459dbbf8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1259

[GitHub] [hudi] hudi-bot commented on pull request #7070: [HUDI-5099] Update stock data to be more useful for testing

hudi-bot commented on PR #7070: URL: https://github.com/apache/hudi/pull/7070#issuecomment-1292847122 ## CI report: * 884f40a07323c18df77fe7461ed71a909a632811 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1260

[GitHub] [hudi] hudi-bot commented on pull request #6227: [HUDI-4496] Fixing Orc support broken for Spark 3.x and more

hudi-bot commented on PR #6227: URL: https://github.com/apache/hudi/pull/6227#issuecomment-1292846461 ## CI report: * 05eb8988fb1e9b462e5c2dbc1db32b2a56b8edf0 UNKNOWN * 897aac606d8f112dab9e01e59b2271c5ecb5f9a4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] alexeykudinkin commented on issue #6137: [SUPPORT] Hudi 0.10.1 throws NoSuchMethodError: org.apache.spark.sql.execution.datasources.FileStatusCache.putLeafFiles(Lorg/apache/hadoop/fs/P

alexeykudinkin commented on issue #6137: URL: https://github.com/apache/hudi/issues/6137#issuecomment-1292844493 We have created HUDI-5092 to track this one -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[jira] [Updated] (HUDI-5100) Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

[ https://issues.apache.org/jira/browse/HUDI-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] waywtdcc updated HUDI-5100: --- Issue Type: Improvement (was: Bug) > Flink batch is written to hudi. When the number of slots is less than >

[jira] [Closed] (HUDI-1368) Merge On Read Snapshot Reader not working for Databricks on ADLS Gen2

[ https://issues.apache.org/jira/browse/HUDI-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin closed HUDI-1368. - Resolution: Duplicate > Merge On Read Snapshot Reader not working for Databricks on ADLS Gen2 > --

[jira] [Updated] (HUDI-5100) Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

[ https://issues.apache.org/jira/browse/HUDI-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] waywtdcc updated HUDI-5100: --- Affects Version/s: (was: 0.12.1) > Flink batch is written to hudi. When the number of slots is less than

[jira] [Updated] (HUDI-5100) Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

[ https://issues.apache.org/jira/browse/HUDI-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] waywtdcc updated HUDI-5100: --- Issue Type: Bug (was: New Feature) > Flink batch is written to hudi. When the number of slots is less than >

[jira] [Updated] (HUDI-5100) Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

[ https://issues.apache.org/jira/browse/HUDI-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] waywtdcc updated HUDI-5100: --- Affects Version/s: 0.12.1 > Flink batch is written to hudi. When the number of slots is less than > write.tas

[jira] [Assigned] (HUDI-5100) Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

[ https://issues.apache.org/jira/browse/HUDI-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] waywtdcc reassigned HUDI-5100: -- Assignee: waywtdcc > Flink batch is written to hudi. When the number of slots is less than > write.tas

[jira] [Resolved] (HUDI-5100) Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

[ https://issues.apache.org/jira/browse/HUDI-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] waywtdcc resolved HUDI-5100. > Flink batch is written to hudi. When the number of slots is less than > write.tasks, it cannot be written > -

[jira] [Reopened] (HUDI-5100) Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

[ https://issues.apache.org/jira/browse/HUDI-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] waywtdcc reopened HUDI-5100: > Flink batch is written to hudi. When the number of slots is less than > write.tasks, it cannot be written > -

[jira] [Assigned] (HUDI-5100) Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

[ https://issues.apache.org/jira/browse/HUDI-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] waywtdcc reassigned HUDI-5100: -- Assignee: (was: waywtdcc) > Flink batch is written to hudi. When the number of slots is less than

[jira] [Updated] (HUDI-5100) Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

[ https://issues.apache.org/jira/browse/HUDI-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] waywtdcc updated HUDI-5100: --- Issue Type: New Feature (was: Improvement) > Flink batch is written to hudi. When the number of slots is less

[jira] [Updated] (HUDI-5092) Querying Hudi table throws NoSuchMethodError in Databricks runtime

[ https://issues.apache.org/jira/browse/HUDI-5092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-5092: -- Description: Originally reported by the user: [https://github.com/apache/hudi/issues/6137]  

[jira] [Assigned] (HUDI-5092) Querying Hudi table throws NoSuchMethodError in Databricks runtime

[ https://issues.apache.org/jira/browse/HUDI-5092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin reassigned HUDI-5092: - Assignee: Alexey Kudinkin > Querying Hudi table throws NoSuchMethodError in Databricks ru

[jira] [Updated] (HUDI-5100) Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

[ https://issues.apache.org/jira/browse/HUDI-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] waywtdcc updated HUDI-5100: --- Fix Version/s: 0.13.0 > Flink batch is written to hudi. When the number of slots is less than > write.tasks,

[jira] [Updated] (HUDI-5100) Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

[ https://issues.apache.org/jira/browse/HUDI-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] waywtdcc updated HUDI-5100: --- Component/s: flink-sql Issue Type: Improvement (was: New Feature) > Flink batch is written to hudi. When

[jira] [Created] (HUDI-5100) Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

waywtdcc created HUDI-5100: -- Summary: Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written Key: HUDI-5100 URL: https://issues.apache.org/jira/browse/HUDI-5100

[jira] [Updated] (HUDI-5092) Querying Hudi table throws NoSuchMethodError in Databricks runtime

[ https://issues.apache.org/jira/browse/HUDI-5092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-5092: -- Description: Originally reported by the user: [https://github.com/apache/hudi/issues/6137]  

[jira] [Updated] (HUDI-5092) Querying Hudi table throws NoSuchMethodError in Databricks runtime

[ https://issues.apache.org/jira/browse/HUDI-5092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-5092: -- Priority: Blocker (was: Critical) > Querying Hudi table throws NoSuchMethodError in Databricks

[jira] [Updated] (HUDI-5092) Querying Hudi table throws NoSuchMethodError in Databricks runtime

[ https://issues.apache.org/jira/browse/HUDI-5092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-5092: -- Component/s: spark > Querying Hudi table throws NoSuchMethodError in Databricks runtime > -

[jira] [Updated] (HUDI-5092) Querying Hudi table throws NoSuchMethodError in Databricks runtime

[ https://issues.apache.org/jira/browse/HUDI-5092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-5092: -- Fix Version/s: 0.12.2 > Querying Hudi table throws NoSuchMethodError in Databricks runtime > --

  1   2   3   >