[GitHub] [hudi] hudi-bot commented on pull request #6616: Add Postgres Schema Name to Postgres Debezium Source

2022-09-07 Thread GitBox
hudi-bot commented on PR #6616: URL: https://github.com/apache/hudi/pull/6616#issuecomment-1240039218 ## CI report: * 8176e809b4f329e0cfbff75484b3595c69970207 Azure:

[GitHub] [hudi] xushiyan commented on a diff in pull request #6256: [RFC-51][HUDI-3478] Update RFC: CDC support

2022-09-07 Thread GitBox
xushiyan commented on code in PR #6256: URL: https://github.com/apache/hudi/pull/6256#discussion_r965393070 ## rfc/rfc-51/rfc-51.md: ## @@ -215,18 +245,31 @@ Note: - Only instants that are active can be queried in a CDC scenario. - `CDCReader` manages all the things on CDC,

[GitHub] [hudi] hudi-bot commented on pull request #6628: [HUDI-4806] Use Avro version from the root pom for Flink bundle

2022-09-07 Thread GitBox
hudi-bot commented on PR #6628: URL: https://github.com/apache/hudi/pull/6628#issuecomment-1240035433 ## CI report: * 2504fd6b17a7a3fb2a77f755d7fe6b6c7f83c96f UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #6616: Add Postgres Schema Name to Postgres Debezium Source

2022-09-07 Thread GitBox
hudi-bot commented on PR #6616: URL: https://github.com/apache/hudi/pull/6616#issuecomment-1240035379 ## CI report: * 8176e809b4f329e0cfbff75484b3595c69970207 Azure:

[GitHub] [hudi] yuzhaojing commented on a diff in pull request #4309: [HUDI-3016][RFC-43] Proposal to implement Table Management Service

2022-09-07 Thread GitBox
yuzhaojing commented on code in PR #4309: URL: https://github.com/apache/hudi/pull/4309#discussion_r965390402 ## rfc/rfc-43/rfc-43.md: ## @@ -0,0 +1,316 @@ + + +# RFC-43: Implement Table Management ServiceTable Management Service for Hudi + +## Proposers + +- @yuzhaojing + +##

[jira] [Updated] (HUDI-4806) Use Avro version from root pom file for Flink bundle

2022-09-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4806: - Labels: pull-request-available (was: ) > Use Avro version from root pom file for Flink bundle >

[GitHub] [hudi] xushiyan commented on a diff in pull request #6256: [RFC-51][HUDI-3478] Update RFC: CDC support

2022-09-07 Thread GitBox
xushiyan commented on code in PR #6256: URL: https://github.com/apache/hudi/pull/6256#discussion_r965380973 ## rfc/rfc-51/rfc-51.md: ## @@ -148,20 +152,46 @@ hudi_cdc_table/ Under a partition directory, the `.log` file with `CDCBlock` above will keep the changing data we

[GitHub] [hudi] CTTY opened a new pull request, #6628: [HUDI-4806] Use Avro version from the root pom for Flink bundle

2022-09-07 Thread GitBox
CTTY opened a new pull request, #6628: URL: https://github.com/apache/hudi/pull/6628 ### Change Logs Make Avro version consistent across Hudi and make sure flink bundle is usable even when user is building against spark3 profile ### Impact _Describe any public API or

[jira] [Created] (HUDI-4806) Use Avro version from root pom file for Flink bundle

2022-09-07 Thread Shawn Chang (Jira)
Shawn Chang created HUDI-4806: - Summary: Use Avro version from root pom file for Flink bundle Key: HUDI-4806 URL: https://issues.apache.org/jira/browse/HUDI-4806 Project: Apache Hudi Issue Type:

[GitHub] [hudi] xushiyan commented on a diff in pull request #6256: [RFC-51][HUDI-3478] Update RFC: CDC support

2022-09-07 Thread GitBox
xushiyan commented on code in PR #6256: URL: https://github.com/apache/hudi/pull/6256#discussion_r965380973 ## rfc/rfc-51/rfc-51.md: ## @@ -148,20 +152,46 @@ hudi_cdc_table/ Under a partition directory, the `.log` file with `CDCBlock` above will keep the changing data we

[GitHub] [hudi] yuzhaojing commented on a diff in pull request #4309: [HUDI-3016][RFC-43] Proposal to implement Table Management Service

2022-09-07 Thread GitBox
yuzhaojing commented on code in PR #4309: URL: https://github.com/apache/hudi/pull/4309#discussion_r965378497 ## rfc/rfc-43/rfc-43.md: ## @@ -0,0 +1,316 @@ + + +# RFC-43: Implement Table Management ServiceTable Management Service for Hudi + +## Proposers + +- @yuzhaojing + +##

[GitHub] [hudi] yuzhaojing commented on a diff in pull request #4309: [HUDI-3016][RFC-43] Proposal to implement Table Management Service

2022-09-07 Thread GitBox
yuzhaojing commented on code in PR #4309: URL: https://github.com/apache/hudi/pull/4309#discussion_r965378497 ## rfc/rfc-43/rfc-43.md: ## @@ -0,0 +1,316 @@ + + +# RFC-43: Implement Table Management ServiceTable Management Service for Hudi + +## Proposers + +- @yuzhaojing + +##

[GitHub] [hudi] yuzhaojing commented on a diff in pull request #4309: [HUDI-3016][RFC-43] Proposal to implement Table Management Service

2022-09-07 Thread GitBox
yuzhaojing commented on code in PR #4309: URL: https://github.com/apache/hudi/pull/4309#discussion_r965378032 ## rfc/rfc-43/rfc-43.md: ## @@ -0,0 +1,316 @@ + + +# RFC-43: Implement Table Management ServiceTable Management Service for Hudi + +## Proposers + +- @yuzhaojing + +##

[GitHub] [hudi] yuzhaojing commented on a diff in pull request #4309: [HUDI-3016][RFC-43] Proposal to implement Table Management Service

2022-09-07 Thread GitBox
yuzhaojing commented on code in PR #4309: URL: https://github.com/apache/hudi/pull/4309#discussion_r965376886 ## rfc/rfc-43/rfc-43.md: ## @@ -0,0 +1,316 @@ + + +# RFC-43: Implement Table Management ServiceTable Management Service for Hudi + +## Proposers + +- @yuzhaojing + +##

[GitHub] [hudi] xushiyan commented on a diff in pull request #6256: [RFC-51][HUDI-3478] Update RFC: CDC support

2022-09-07 Thread GitBox
xushiyan commented on code in PR #6256: URL: https://github.com/apache/hudi/pull/6256#discussion_r965376782 ## rfc/rfc-51/rfc-51.md: ## @@ -64,69 +65,72 @@ We follow the debezium output format: four columns as shown below Note: the illustration here ignores all the Hudi

[jira] [Closed] (HUDI-4793) Fix ScalaTest not respecting Log4j2 configs

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-4793. Resolution: Fixed > Fix ScalaTest not respecting Log4j2 configs >

[GitHub] [hudi] hudi-bot commented on pull request #6624: [HUDI-4518] add unit for reentrant lock in diff lockProvider

2022-09-07 Thread GitBox
hudi-bot commented on PR #6624: URL: https://github.com/apache/hudi/pull/6624#issuecomment-1239988100 ## CI report: * 5bc514fe8aa680df5066dc0d1bcad3fc950afdf8 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6615: [HUDI-4758] Add validations to java spark examples

2022-09-07 Thread GitBox
hudi-bot commented on PR #6615: URL: https://github.com/apache/hudi/pull/6615#issuecomment-1239988078 ## CI report: * 61214015c3aed029c00882f121e6ec0333767e7f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6615: [HUDI-4758] Add validations to java spark examples

2022-09-07 Thread GitBox
hudi-bot commented on PR #6615: URL: https://github.com/apache/hudi/pull/6615#issuecomment-1239984954 ## CI report: * 61214015c3aed029c00882f121e6ec0333767e7f Azure:

[jira] [Updated] (HUDI-4369) Hudi Kafka Connect Sink writing to GCS bucket

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4369: - Sprint: (was: 2022/09/05) > Hudi Kafka Connect Sink writing to GCS bucket >

[jira] [Updated] (HUDI-3961) Encounter NoClassDefFoundError when using Spark 3.1 bundle and utilities slim bundle

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3961: - Reviewers: Raymond Xu > Encounter NoClassDefFoundError when using Spark 3.1 bundle and utilities slim >

[jira] [Updated] (HUDI-4542) Flink streaming query fails with ClassNotFoundException

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4542: - Sprint: (was: 2022/09/05) > Flink streaming query fails with ClassNotFoundException >

[jira] [Assigned] (HUDI-4762) Hive sync update schema removes columns

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-4762: Assignee: Raymond Xu > Hive sync update schema removes columns >

[jira] [Assigned] (HUDI-4762) Hive sync update schema removes columns

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-4762: Assignee: nicolas paris (was: Raymond Xu) > Hive sync update schema removes columns >

[GitHub] [hudi] hudi-bot commented on pull request #6619: [HUDI-4796] MetricsReporter stop bug

2022-09-07 Thread GitBox
hudi-bot commented on PR #6619: URL: https://github.com/apache/hudi/pull/6619#issuecomment-1239981498 ## CI report: * 138acf4a157d61a6e5e42b0e86b270ae500d60a1 Azure:

[jira] [Updated] (HUDI-4539) Make Hudi's CLI API consistent

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4539: - Sprint: (was: 2022/09/05) > Make Hudi's CLI API consistent > -- > >

[jira] [Updated] (HUDI-4787) ITTestHoodieSanity#testRunHoodieJavaAppOnMultiPartitionKeysMORTable streaming test

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4787: - Sprint: 2022/09/19 (was: 2022/09/05) >

[jira] [Updated] (HUDI-4457) Make sure IT docker test return code non-zero when failed

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4457: - Sprint: 2022/09/19 (was: 2022/09/05) > Make sure IT docker test return code non-zero when failed >

[GitHub] [hudi] rahil-c commented on issue #6552: [SUPPORT] AWSDmsAvroPayload does not work correctly with any version above 0.10.0

2022-09-07 Thread GitBox
rahil-c commented on issue #6552: URL: https://github.com/apache/hudi/issues/6552#issuecomment-1239972575 Currently investigating this issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[hudi] 01/01: [DOCS] Fix image references in recently added blogs

2022-09-07 Thread bhavanisudha
This is an automated email from the ASF dual-hosted git repository. bhavanisudha pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git commit 69ef43cd38b243c05404c8592a8bf4de12e102b8 Author: Bhavani Sudha Saktheeswaran

[hudi] branch asf-site updated (be60522e64 -> 69ef43cd38)

2022-09-07 Thread bhavanisudha
This is an automated email from the ASF dual-hosted git repository. bhavanisudha pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git discard be60522e64 [DOCS] Asf site update flink option 'read.tasks & write.tasks' description (#6614) discard

[hudi] branch asf-site updated: [DOCS] Asf site update flink option 'read.tasks & write.tasks' description (#6614)

2022-09-07 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new be60522e64 [DOCS] Asf site update flink option

[GitHub] [hudi] yihua merged pull request #6614: [DOCS] Asf site update flink option 'read.tasks & write.tasks' description

2022-09-07 Thread GitBox
yihua merged PR #6614: URL: https://github.com/apache/hudi/pull/6614 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] yihua commented on a diff in pull request #6614: [doc]Asf site update flink option 'read.tasks & write.tasks' description

2022-09-07 Thread GitBox
yihua commented on code in PR #6614: URL: https://github.com/apache/hudi/pull/6614#discussion_r965336932 ## website/docs/configurations.md: ## @@ -978,8 +978,8 @@ Actual value obtained by invoking .toString(), default '' --- > write.tasks -> Parallelism of tasks that

[hudi] branch asf-site updated: [DOCS] Add blogs to Hudi website (#6627)

2022-09-07 Thread bhavanisudha
This is an automated email from the ASF dual-hosted git repository. bhavanisudha pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new a10ff34637 [DOCS] Add blogs to Hudi

[GitHub] [hudi] bhasudha merged pull request #6627: [DOCS] Add blogs to Hudi website

2022-09-07 Thread GitBox
bhasudha merged PR #6627: URL: https://github.com/apache/hudi/pull/6627 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] hudi-bot commented on pull request #6625: [HUDI-4799] improve analyzer exception tip when can not resolve expre…

2022-09-07 Thread GitBox
hudi-bot commented on PR #6625: URL: https://github.com/apache/hudi/pull/6625#issuecomment-1239915170 ## CI report: * 5f385a174df1fa344b87a3a4ada3f3f6d61f1d76 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6384: [HUDI-4613] Avoid the use of regex expressions when call hoodieFileGroup#addLogFile function

2022-09-07 Thread GitBox
hudi-bot commented on PR #6384: URL: https://github.com/apache/hudi/pull/6384#issuecomment-1239914461 ## CI report: * 4bc1babc9814102ce767cccbc5c1bd78447cc5a2 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6548: [HUDI-4749] Fixing full cleaning to leverage metadata table

2022-09-07 Thread GitBox
hudi-bot commented on PR #6548: URL: https://github.com/apache/hudi/pull/6548#issuecomment-1239834779 ## CI report: * 67cd8a64f22c3ed2b57b2bd97723874d5ea3ae54 Azure:

[GitHub] [hudi] yihua commented on a diff in pull request #6550: [HUDI-4691] Cleaning up duplicated classes in Spark 3.3 module

2022-09-07 Thread GitBox
yihua commented on code in PR #6550: URL: https://github.com/apache/hudi/pull/6550#discussion_r965238401 ## pom.xml: ## @@ -377,9 +377,17 @@ org.sl4fj:slf4j-jcl log4j:log4j ch.qos.logback:logback-classic +

[GitHub] [hudi] hudi-bot commented on pull request #5091: [HUDI-3453] Fix HoodieBackedTableMetadata concurrent reading issue

2022-09-07 Thread GitBox
hudi-bot commented on PR #5091: URL: https://github.com/apache/hudi/pull/5091#issuecomment-1239833315 ## CI report: * c0dc922eec0ffe4c93f250dcf91dd313713057db Azure:

[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #5091: [HUDI-3453] Fix HoodieBackedTableMetadata concurrent reading issue

2022-09-07 Thread GitBox
alexeykudinkin commented on code in PR #5091: URL: https://github.com/apache/hudi/pull/5091#discussion_r965235941 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java: ## @@ -231,7 +232,7 @@ public List>>> getRecord throw new

[GitHub] [hudi] hudi-bot commented on pull request #6620: [HUDI-4797] fix merge into table for source table with different column order

2022-09-07 Thread GitBox
hudi-bot commented on PR #6620: URL: https://github.com/apache/hudi/pull/6620#issuecomment-123982 ## CI report: * 4e4f5cd356e0c7022b998e46b155eed0c35eb226 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6548: [HUDI-4749] Fixing full cleaning to leverage metadata table

2022-09-07 Thread GitBox
hudi-bot commented on PR #6548: URL: https://github.com/apache/hudi/pull/6548#issuecomment-1239829803 ## CI report: * 67cd8a64f22c3ed2b57b2bd97723874d5ea3ae54 Azure:

[GitHub] [hudi] nsivabalan commented on issue #6601: [SUPPORT] "default" folder not outputted by Hudi for non-partitioned tables when used with Spark

2022-09-07 Thread GitBox
nsivabalan commented on issue #6601: URL: https://github.com/apache/hudi/issues/6601#issuecomment-1239806342 Using custom key gen, but setting empty value for partition path is a wrong usage may be. Can you try fixing it and give it a try. Reason why hudi could write to default folder

[GitHub] [hudi] nsivabalan commented on issue #6601: [SUPPORT] "default" folder not outputted by Hudi for non-partitioned tables when used with Spark

2022-09-07 Thread GitBox
nsivabalan commented on issue #6601: URL: https://github.com/apache/hudi/issues/6601#issuecomment-1239804231 hey I am bit confused. you claim that you are interested in non partitioned tables. but I see you are using CustomKeyGenerator. I would expect you to use NonPartitionedKeyGenerator

[GitHub] [hudi] bhasudha commented on pull request #6627: [DOCS] Add blogs to Hudi website

2022-09-07 Thread GitBox
bhasudha commented on PR #6627: URL: https://github.com/apache/hudi/pull/6627#issuecomment-1239792139 https://user-images.githubusercontent.com/2179254/188961672-e42d57d4-48ed-495b-ad42-a52e819d1f2f.png;> The hudi logo images should go away when the images are referenced in the next

[GitHub] [hudi] nsivabalan commented on issue #5984: [SUPPORT] Error on GlobalSortPartitioner using 0.9.0

2022-09-07 Thread GitBox
nsivabalan commented on issue #5984: URL: https://github.com/apache/hudi/issues/5984#issuecomment-1239791049 @rubenssoto : hey man. we might need more info about write configs used. whether are you using bulk_insert to write to hudi. or is it happening w/ clustering. would appreciate if

[GitHub] [hudi] bhasudha opened a new pull request, #6627: [DOCS] Add blogs to Hudi website

2022-09-07 Thread GitBox
bhasudha opened a new pull request, #6627: URL: https://github.com/apache/hudi/pull/6627 ### Change Logs Added reference to blogs in Hudi website. Have added the images separately. Once the images are merged I will follow up with a separate commit to refer the images in the blog.

[GitHub] [hudi] hudi-bot commented on pull request #6575: [HUDI-4754] Add compliance check in github actions

2022-09-07 Thread GitBox
hudi-bot commented on PR #6575: URL: https://github.com/apache/hudi/pull/6575#issuecomment-1239781330 ## CI report: * 1600e31836157c8d05e3bc8b9e08e1717471f1a6 UNKNOWN * 4d02f2c64a5fc4b89889677ee639a20b53cec26a UNKNOWN * 48147d19c835e7868102fd2d083659e6ee2ac343 UNKNOWN *

[GitHub] [hudi] hudi-bot commented on pull request #6575: [HUDI-4754] Add compliance check in github actions

2022-09-07 Thread GitBox
hudi-bot commented on PR #6575: URL: https://github.com/apache/hudi/pull/6575#issuecomment-1239776908 ## CI report: * 1600e31836157c8d05e3bc8b9e08e1717471f1a6 UNKNOWN * 4d02f2c64a5fc4b89889677ee639a20b53cec26a UNKNOWN * 48147d19c835e7868102fd2d083659e6ee2ac343 UNKNOWN *

[GitHub] [hudi] nsivabalan commented on a diff in pull request #6615: [HUDI-4758] Add validations to java spark examples

2022-09-07 Thread GitBox
nsivabalan commented on code in PR #6615: URL: https://github.com/apache/hudi/pull/6615#discussion_r965133110 ## hudi-examples/hudi-examples-spark/src/main/java/org/apache/hudi/examples/quickstart/HoodieSparkQuickstart.java: ## @@ -65,30 +66,42 @@ public static void

[GitHub] [hudi] hudi-bot commented on pull request #6608: [HUDI-4752] Add dedup support for MOR table in cli

2022-09-07 Thread GitBox
hudi-bot commented on PR #6608: URL: https://github.com/apache/hudi/pull/6608#issuecomment-1239722010 ## CI report: * 3ace8fd54f3aceec456b471e750c1aa1e04fa8f7 UNKNOWN * 340548ebdb9d36734f1349fa91af3cc65bb4963a Azure:

[GitHub] [hudi] suryaprasanna commented on pull request #5958: [HUDI-3900] [UBER] Support log compaction action for MOR tables

2022-09-07 Thread GitBox
suryaprasanna commented on PR #5958: URL: https://github.com/apache/hudi/pull/5958#issuecomment-1239717751 @prasannarajaperumal My apologies, I was off until today due to some personal reasons. Will work on this PR now and get it merged. -- This is an automated message from the

[GitHub] [hudi] hudi-bot commented on pull request #6520: [HUDI-4726] Incremental input splits result is not as expected when f…

2022-09-07 Thread GitBox
hudi-bot commented on PR #6520: URL: https://github.com/apache/hudi/pull/6520#issuecomment-1239712656 ## CI report: * e55d28bdafa64d4a5180fd46191a420e702a58dc UNKNOWN * 3b63f17ba96b7514d84252b28f104a666fdb012d Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6520: [HUDI-4726] Incremental input splits result is not as expected when f…

2022-09-07 Thread GitBox
hudi-bot commented on PR #6520: URL: https://github.com/apache/hudi/pull/6520#issuecomment-1239707811 ## CI report: * e55d28bdafa64d4a5180fd46191a420e702a58dc UNKNOWN * 3b63f17ba96b7514d84252b28f104a666fdb012d Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5091: [HUDI-3453] Fix HoodieBackedTableMetadata concurrent reading issue

2022-09-07 Thread GitBox
hudi-bot commented on PR #5091: URL: https://github.com/apache/hudi/pull/5091#issuecomment-1239706367 ## CI report: * 13507fc5edcfad81e559607c3e36b9e56eb6d09a Azure:

[GitHub] [hudi] yihua commented on issue #6623: [SUPPORT] java.lang.ClassNotFoundException: Class org.apache.hadoop.hbase.client.ClusterStatusListener$MulticastListener with HBase Index

2022-09-07 Thread GitBox
yihua commented on issue #6623: URL: https://github.com/apache/hudi/issues/6623#issuecomment-1239698510 cc @umehrot2 @rahil-c @CTTY -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] yihua commented on issue #6623: [SUPPORT] java.lang.ClassNotFoundException: Class org.apache.hadoop.hbase.client.ClusterStatusListener$MulticastListener with HBase Index

2022-09-07 Thread GitBox
yihua commented on issue #6623: URL: https://github.com/apache/hudi/issues/6623#issuecomment-1239696874 @praveenkmr have you tried spark-submit or spark-shell with the suggested workaround in https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-hudi-considerations.html? ```

[jira] [Updated] (HUDI-2369) Blog on bulk insert sort modes

2022-09-07 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2369: -- Sprint: 2022/09/19 (was: 2022/09/05) > Blog on bulk insert sort modes >

[GitHub] [hudi] arunb2w opened a new issue, #6626: [SUPPORT] HUDI merge into via spark sql not working

2022-09-07 Thread GitBox
arunb2w opened a new issue, #6626: URL: https://github.com/apache/hudi/issues/6626 Getting error pyspark.sql.utils.AnalysisException: Invalidate Merge-On condition: when running the below code Hudi Version: 0.10 Spark version: 3.1.2 **Sample code** ```

[jira] [Updated] (HUDI-4800) Community support - Raymond

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4800: - Story Points: 8 (was: 5) > Community support - Raymond > --- > >

[jira] [Updated] (HUDI-3304) support partial update on mor table

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3304: - Sprint: (was: 2022/09/05) > support partial update on mor table >

[jira] [Updated] (HUDI-4803) Community support - Sagar

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4803: - Story Points: 9 (was: 5) > Community support - Sagar > - > >

[jira] [Updated] (HUDI-2768) Enable async timeline server by default

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2768: - Sprint: (was: 2022/09/05) > Enable async timeline server by default >

[jira] [Assigned] (HUDI-4342) Improve handling of 5xx in timeline server

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-4342: Assignee: Sagar Sumit (was: Ethan Guo) > Improve handling of 5xx in timeline server >

[jira] [Assigned] (HUDI-3648) Failed to execute rollback due to HoodieIOException: Could not delete instant

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-3648: Assignee: Sagar Sumit (was: Ethan Guo) > Failed to execute rollback due to HoodieIOException:

[jira] [Assigned] (HUDI-4256) Bulk insert of a large dataset with S3 fails w/ timeline server based markers

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-4256: Assignee: Sagar Sumit (was: Ethan Guo) > Bulk insert of a large dataset with S3 fails w/ timeline

[jira] [Updated] (HUDI-4586) Address S3 timeouts in Bloom Index with metadata table

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4586: - Story Points: 3 (was: 1) > Address S3 timeouts in Bloom Index with metadata table >

[jira] [Updated] (HUDI-4341) HoodieHFileReader is not compatible with Hadoop 3

2022-09-07 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-4341: Sprint: 2022/08/22, 2022/09/19 (was: 2022/08/22, 2022/09/05) > HoodieHFileReader is not compatible with

[jira] [Updated] (HUDI-4805) Update docs for workaround to make HBase working with HDFS on Hadoop 3

2022-09-07 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-4805: Sprint: 2022/09/05 > Update docs for workaround to make HBase working with HDFS on Hadoop 3 >

[jira] [Updated] (HUDI-4805) Update docs for workaround to make HBase working with HDFS on Hadoop 3

2022-09-07 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-4805: Fix Version/s: 0.12.1 > Update docs for workaround to make HBase working with HDFS on Hadoop 3 >

[jira] [Assigned] (HUDI-4805) Update docs for workaround to make HBase working with HDFS on Hadoop 3

2022-09-07 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo reassigned HUDI-4805: --- Assignee: Ethan Guo > Update docs for workaround to make HBase working with HDFS on Hadoop 3 >

[jira] [Updated] (HUDI-4805) Update docs for workaround to make HBase working with HDFS on Hadoop 3

2022-09-07 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-4805: Story Points: 1 Issue Type: Improvement (was: Bug) > Update docs for workaround to make HBase

[jira] [Updated] (HUDI-2580) Ability to clean up dangling data files using hudi-cli

2022-09-07 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-2580: -- Sprint: 2022/09/19 (was: 2022/09/05) > Ability to clean up dangling data files using hudi-cli >

[jira] [Updated] (HUDI-3819) upgrade spring cve-2022-22965

2022-09-07 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3819: -- Sprint: 2022/09/19 (was: 2022/09/05) > upgrade spring cve-2022-22965 > - >

[jira] [Updated] (HUDI-4789) Convert FileSystem usage in hudi connector to use TrinoFileSystem interface

2022-09-07 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4789: -- Sprint: 2022/09/19 (was: 2022/09/05) > Convert FileSystem usage in hudi connector to use

[jira] [Updated] (HUDI-1413) Need binary release of Hudi to distribute tools like hudi-cli.sh and hudi-sync

2022-09-07 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-1413: -- Sprint: 2022/09/19 (was: 2022/09/05) > Need binary release of Hudi to distribute tools like

[jira] [Created] (HUDI-4805) Update docs for workaround to make HBase working with HDFS on Hadoop 3

2022-09-07 Thread Ethan Guo (Jira)
Ethan Guo created HUDI-4805: --- Summary: Update docs for workaround to make HBase working with HDFS on Hadoop 3 Key: HUDI-4805 URL: https://issues.apache.org/jira/browse/HUDI-4805 Project: Apache Hudi

[jira] [Updated] (HUDI-4666) Investigate Hudi CLI out of box support

2022-09-07 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4666: -- Sprint: 2022/09/19 (was: 2022/09/05) > Investigate Hudi CLI out of box support >

[jira] [Updated] (HUDI-3626) Refactor TableSchemaResolver to remove `includeMetadataFields` flags

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3626: - Sprint: (was: 2022/09/05) > Refactor TableSchemaResolver to remove `includeMetadataFields` flags >

[jira] [Updated] (HUDI-4687) Avoid all illegal reflective access in the code

2022-09-07 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4687: -- Sprint: 2022/09/05 > Avoid all illegal reflective access in the code >

[jira] [Updated] (HUDI-4687) Avoid all illegal reflective access in the code

2022-09-07 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4687: -- Story Points: 4 > Avoid all illegal reflective access in the code >

[jira] [Updated] (HUDI-2071) Support Reading Bootstrap MOR RT Table In Spark DataSource Table

2022-09-07 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-2071: Sprint: 2022/09/19 (was: 2022/09/05) > Support Reading Bootstrap MOR RT Table In Spark DataSource Table >

[jira] [Updated] (HUDI-4662) Test MOR: Spark datasource and SQL with bootstrap

2022-09-07 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-4662: Sprint: 2022/09/19 (was: 2022/09/05) > Test MOR: Spark datasource and SQL with bootstrap >

[jira] [Updated] (HUDI-4125) Add IT (Azure CI) around bootstrapped Hudi table

2022-09-07 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-4125: Sprint: 2022/09/19 (was: 2022/09/05) > Add IT (Azure CI) around bootstrapped Hudi table >

[jira] [Updated] (HUDI-1265) Efficient bootstrap and migration of existing non-Hudi dataset

2022-09-07 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-1265: Description: This is an EPIC to revisit the logic of bootstrap for efficient migration of existing

[jira] [Updated] (HUDI-4663) Test MOR: Hive QL with bootstrap

2022-09-07 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-4663: Sprint: 2022/09/19 (was: 2022/09/05) > Test MOR: Hive QL with bootstrap >

[GitHub] [hudi] hudi-bot commented on pull request #6489: [HUDI-4485] [cli] Bumped spring shell to 2.1.1. Updated the default …

2022-09-07 Thread GitBox
hudi-bot commented on PR #6489: URL: https://github.com/apache/hudi/pull/6489#issuecomment-1239549303 ## CI report: * 3ae4fb8b374e12b1097a86d56e5996b7dc0ac79f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6625: [HUDI-4799] improve analyzer exception tip when can not resolve expre…

2022-09-07 Thread GitBox
hudi-bot commented on PR #6625: URL: https://github.com/apache/hudi/pull/6625#issuecomment-1239541972 ## CI report: * 5f385a174df1fa344b87a3a4ada3f3f6d61f1d76 Azure:

[jira] [Updated] (HUDI-2754) Performance improvement for IncrementalRelation

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2754: - Sprint: Cont' improve - 2022/03/7, 2022/08/22, 2022/09/05 (was: Cont' improve - 2022/03/7, 2022/08/22)

[jira] [Updated] (HUDI-4729) File group in pending compaction can not be queried when query ro table with spark

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4729: - Sprint: 2022/08/22, 2022/09/05 (was: 2022/08/22) > File group in pending compaction can not be queried

[jira] [Updated] (HUDI-3249) Performance Improvements

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3249: - Sprint: 2022/08/22, 2022/09/05 (was: 2022/08/22) > Performance Improvements > >

[jira] [Updated] (HUDI-4661) Test COW: Hive QL with bootstrap

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4661: - Sprint: 2022/08/22, 2022/09/05 (was: 2022/08/22) > Test COW: Hive QL with bootstrap >

[jira] [Updated] (HUDI-4736) Fix inflight clean action preventing clean service to continue when multiple cleans are not allowed

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4736: - Sprint: 2022/08/22, 2022/09/05 (was: 2022/08/22) > Fix inflight clean action preventing clean service to

[jira] [Updated] (HUDI-4326) Hudi spark datasource error after migrate from 0.8 to 0.11

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4326: - Sprint: 2022/08/22, 2022/09/05 (was: 2022/08/22) > Hudi spark datasource error after migrate from 0.8 to

[jira] [Updated] (HUDI-4588) Ingestion failing if source column is dropped

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4588: - Sprint: 2022/08/08, 2022/08/22, 2022/09/05 (was: 2022/08/08, 2022/08/22) > Ingestion failing if source

[jira] [Updated] (HUDI-954) Test COW : Presto Read Optimized Query with metadata bootstrap

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-954: Sprint: 2022/08/22, 2022/09/05 (was: 2022/08/22) > Test COW : Presto Read Optimized Query with metadata

[jira] [Updated] (HUDI-4629) Create hive table from existing hoodie Table failed when the table schema is not defined

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4629: - Sprint: 2022/08/22, 2022/09/05 (was: 2022/08/22) > Create hive table from existing hoodie Table failed

[jira] [Updated] (HUDI-3636) Clustering fails due to marker creation failure

2022-09-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3636: - Sprint: 2022/08/22, 2022/09/05 (was: 2022/08/22) > Clustering fails due to marker creation failure >

<    1   2   3   4   >