[jira] [Updated] (HUDI-2955) Upgrade Hadoop to 3.3.x

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2955: - Reviewers: Ethan Guo (was: Alexey Kudinkin, Ethan Guo) > Upgrade Hadoop to 3.3.x > --

[jira] [Updated] (HUDI-4584) SQLConf is not propagated correctly into RDDs

2022-08-23 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-4584: -- Story Points: 6 (was: 8) > SQLConf is not propagated correctly into RDDs >

[jira] [Updated] (HUDI-4691) Deduplicate Spark 3.2 and Spark 3.3 integrations

2022-08-23 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-4691: -- Story Points: 12 (was: 3) > Deduplicate Spark 3.2 and Spark 3.3 integrations >

[jira] [Updated] (HUDI-4588) Ingestion failing if source column is dropped

2022-08-23 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-4588: -- Story Points: 12 (was: 5) > Ingestion failing if source column is dropped > ---

[jira] [Updated] (HUDI-4503) Support table identifier with explicit catalog

2022-08-23 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-4503: -- Story Points: 4 (was: 2) > Support table identifier with explicit catalog > ---

[jira] [Updated] (HUDI-4626) Partitioning table by `_hoodie_partition_path` fails

2022-08-23 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-4626: -- Story Points: 4 (was: 2) > Partitioning table by `_hoodie_partition_path` fails > -

[jira] [Updated] (HUDI-4584) SQLConf is not propagated correctly into RDDs

2022-08-23 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-4584: -- Story Points: 8 (was: 4) > SQLConf is not propagated correctly into RDDs >

[jira] [Updated] (HUDI-4364) integrate column stats index with presto engine

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4364: - Sprint: (was: 2022/08/22) > integrate column stats index with presto engine > --

[jira] [Updated] (HUDI-3397) Make sure Spark RDDs triggering actual FS activity are only dereferenced once

2022-08-23 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-3397: -- Sprint: 2022/09/05 > Make sure Spark RDDs triggering actual FS activity are only dereferenced on

[jira] [Updated] (HUDI-4465) Optimizing file-listing path in MT

2022-08-23 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-4465: -- Sprint: 2022/08/22 > Optimizing file-listing path in MT > -- > >

[jira] [Updated] (HUDI-4467) Port borrowed code from Spark 3.3

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4467: - Story Points: 5 > Port borrowed code from Spark 3.3 > - > >

[jira] [Updated] (HUDI-4468) Simplify TimeTravel logic for Spark 3.3

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4468: - Sprint: 2022/08/22 (was: 2022/09/19) > Simplify TimeTravel logic for Spark 3.3 >

[jira] [Assigned] (HUDI-4626) Partitioning table by `_hoodie_partition_path` fails

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-4626: Assignee: Alexey Kudinkin > Partitioning table by `_hoodie_partition_path` fails >

[jira] [Updated] (HUDI-4468) Simplify TimeTravel logic for Spark 3.3

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4468: - Story Points: 5 > Simplify TimeTravel logic for Spark 3.3 > --- > >

[jira] [Assigned] (HUDI-4468) Simplify TimeTravel logic for Spark 3.3

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-4468: Assignee: Alexey Kudinkin > Simplify TimeTravel logic for Spark 3.3 > -

[jira] [Assigned] (HUDI-4467) Port borrowed code from Spark 3.3

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-4467: Assignee: Alexey Kudinkin > Port borrowed code from Spark 3.3 > - >

[jira] [Updated] (HUDI-4467) Port borrowed code from Spark 3.3

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4467: - Sprint: 2022/08/22 (was: 2022/09/19) > Port borrowed code from Spark 3.3 > --

[jira] [Updated] (HUDI-2754) Performance improvement for IncrementalRelation

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2754: - Story Points: 0.5 (was: 1) > Performance improvement for IncrementalRelation > --

[jira] [Updated] (HUDI-2754) Performance improvement for IncrementalRelation

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2754: - Reviewers: Alexey Kudinkin, Ethan Guo (was: Alexey Kudinkin) > Performance improvement for IncrementalRel

[jira] [Updated] (HUDI-2754) Performance improvement for IncrementalRelation

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2754: - Sprint: Cont' improve - 2022/03/7, 2022/08/22 (was: Cont' improve - 2022/03/7, 2022/09/19) > Performance

[jira] [Updated] (HUDI-3287) Remove unnecessary deps in hudi-kafka-connect

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3287: - Sprint: Hudi-Sprint-Mar-01, 2022/09/19 (was: Hudi-Sprint-Mar-01) > Remove unnecessary deps in hudi-kafka-

[jira] [Updated] (HUDI-3287) Remove unnecessary deps in hudi-kafka-connect

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3287: - Priority: Critical (was: Major) > Remove unnecessary deps in hudi-kafka-connect > ---

[jira] [Updated] (HUDI-4597) [GCP] 0 byte files appearing on GCS

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4597: - Priority: Major (was: Critical) > [GCP] 0 byte files appearing on GCS > -

[jira] [Updated] (HUDI-4597) [GCP] 0 byte files appearing on GCS

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4597: - Sprint: (was: 2022/09/05) > [GCP] 0 byte files appearing on GCS > --- >

[jira] [Assigned] (HUDI-4650) Commits Command: Include both active and archive timeline for a given range of intants

2022-08-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit reassigned HUDI-4650: - Assignee: Sagar Sumit > Commits Command: Include both active and archive timeline for a given ran

[jira] [Updated] (HUDI-2695) [DOCS] Trino Hudi connector on Hudi website

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2695: - Sprint: Hudi-Sprint-Jan-3, Hudi-Sprint-Jan-10, Hudi-Sprint-Jan-18, Hudi-Sprint-Jan-24, Hudi-Sprint-Jan-31,

[jira] [Assigned] (HUDI-4648) Add command to rename partition

2022-08-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit reassigned HUDI-4648: - Assignee: Sagar Sumit > Add command to rename partition > --- > >

[jira] [Assigned] (HUDI-4649) Add command to trace file group through a range of commits

2022-08-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit reassigned HUDI-4649: - Assignee: Sagar Sumit > Add command to trace file group through a range of commits >

[jira] [Updated] (HUDI-3204) spark on TimestampBasedKeyGenerator has no result when query by partition column

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3204: - Sprint: 2022/09/19 (was: 2022/08/22) > spark on TimestampBasedKeyGenerator has no result when query by pa

[jira] [Updated] (HUDI-4549) hive sync bundle causes class loader issue

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4549: - Sprint: 2022/08/08, 2022/08/22 (was: 2022/08/08) > hive sync bundle causes class loader issue > -

[jira] [Updated] (HUDI-4568) Cloudwatch reporter not being created with hudi-spark-bundle and hudi-aws-bundle in classpath

2022-08-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4568: -- Story Points: 2 > Cloudwatch reporter not being created with hudi-spark-bundle and > hudi-aws-bundle in

[jira] [Updated] (HUDI-4583) [DOCS] Optimal write configs for different workload patterns

2022-08-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4583: -- Story Points: 1 > [DOCS] Optimal write configs for different workload patterns > ---

[jira] [Updated] (HUDI-4588) Ingestion failing if source column is dropped

2022-08-23 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-4588: -- Story Points: 5 (was: 2) > Ingestion failing if source column is dropped >

[jira] [Updated] (HUDI-3207) Hudi Trino connector PR review

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3207: - Sprint: Hudi-Sprint-Jan-10, Hudi-Sprint-Jan-18, Hudi-Sprint-Jan-24, Hudi-Sprint-Jan-31, Hudi-Sprint-Feb-7,

[jira] [Updated] (HUDI-4665) Flip default for "ignore.failed.batch" for streaming sink

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4665: - Sprint: 2022/08/08, 2022/08/22 (was: 2022/08/08) > Flip default for "ignore.failed.batch" for streaming s

[jira] [Updated] (HUDI-4025) Add support to validate presto, trino and hive queries in integ test framework

2022-08-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4025: -- Story Points: 5 > Add support to validate presto, trino and hive queries in integ test framework > -

[jira] [Updated] (HUDI-4503) Support table identifier with explicit catalog

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4503: - Sprint: 2022/08/08, 2022/08/22 (was: 2022/08/08) > Support table identifier with explicit catalog > -

[jira] [Updated] (HUDI-4588) Ingestion failing if source column is dropped

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4588: - Sprint: 2022/08/08, 2022/08/22 (was: 2022/08/08) > Ingestion failing if source column is dropped > --

[jira] [Updated] (HUDI-4586) Address S3 timeouts in Bloom Index with metadata table

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4586: - Sprint: 2022/08/08, 2022/08/22 (was: 2022/08/08) > Address S3 timeouts in Bloom Index with metadata table

[jira] [Updated] (HUDI-4441) Disbale INFO level logs from tests

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4441: - Sprint: 2022/08/08, 2022/08/22 (was: 2022/08/08) > Disbale INFO level logs from tests > -

[jira] [Updated] (HUDI-3204) spark on TimestampBasedKeyGenerator has no result when query by partition column

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3204: - Sprint: (was: 2022/08/08) > spark on TimestampBasedKeyGenerator has no result when query by partition >

[jira] [Updated] (HUDI-3204) spark on TimestampBasedKeyGenerator has no result when query by partition column

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3204: - Sprint: 2022/08/22 > spark on TimestampBasedKeyGenerator has no result when query by partition > column >

[jira] [Updated] (HUDI-4503) Support table identifier with explicit catalog

2022-08-23 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-4503: -- Story Points: 2 (was: 1) Issue Type: Bug (was: Improvement) > Support table identifier w

[jira] [Updated] (HUDI-4585) Optimize query performance on Presto Hudi connector

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4585: - Sprint: 2022/08/08, 2022/08/22 (was: 2022/08/08) > Optimize query performance on Presto Hudi connector >

[jira] [Updated] (HUDI-4212) kafka-connect module: Unresolved dependency: 'jdk.tools:jdk.tools:jar:1.7'

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4212: - Sprint: 2022/08/08, 2022/08/22 (was: 2022/08/08) > kafka-connect module: Unresolved dependency: 'jdk.tool

[jira] [Assigned] (HUDI-4503) Support table identifier with explicit catalog

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-4503: Assignee: Alexey Kudinkin (was: Yann Byron) > Support table identifier with explicit catalog > ---

[jira] [Updated] (HUDI-4588) Ingestion failing if source column is dropped

2022-08-23 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4588: - Status: In Progress (was: Open) > Ingestion failing if source column is dropped > ---

[GitHub] [hudi] YannByron commented on issue #6424: [SUPPORT] After schema evaluation, when time travel queries the historical data, the results show the latest schema instead of the historical schema

2022-08-23 Thread GitBox
YannByron commented on issue #6424: URL: https://github.com/apache/hudi/issues/6424#issuecomment-1224221638 @xxWSHxx you're right. thank you for this very meaningful issue. A whole snapshot at a specified time point contains not only data but also metadata. a ticket to track this: http

[jira] [Created] (HUDI-4703) use the corresponding schema (not the latest schema) to response the time travel query

2022-08-23 Thread Yann Byron (Jira)
Yann Byron created HUDI-4703: Summary: use the corresponding schema (not the latest schema) to response the time travel query Key: HUDI-4703 URL: https://issues.apache.org/jira/browse/HUDI-4703 Project:

[jira] [Updated] (HUDI-4549) hive sync bundle causes class loader issue

2022-08-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4549: -- Story Points: 2 (was: 3) > hive sync bundle causes class loader issue > ---

[jira] [Updated] (HUDI-4549) hive sync bundle causes class loader issue

2022-08-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4549: -- Sprint: 2022/08/08 (was: 2022/08/22) > hive sync bundle causes class loader issue > ---

[jira] [Closed] (HUDI-4594) Add integration test with MinIO

2022-08-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-4594. - Resolution: Done > Add integration test with MinIO > --- > > K

[jira] [Closed] (HUDI-4595) Add connector documentation

2022-08-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-4595. - Resolution: Done > Add connector documentation > --- > > Key: HUDI

[hudi] branch asf-site updated: [DOCS] Change community sync schedule image (#6477)

2022-08-23 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 25f55d3a1a [DOCS] Change community sync sche

[GitHub] [hudi] xushiyan merged pull request #6477: [DOCS] Change community sync schedule image

2022-08-23 Thread GitBox
xushiyan merged PR #6477: URL: https://github.com/apache/hudi/pull/6477 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.

[GitHub] [hudi] YannByron commented on issue #6465: [SUPPORT] HoodieSparkSqlWriter use sparkContext.hadoopConfiguration when initTable

2022-08-23 Thread GitBox
YannByron commented on issue #6465: URL: https://github.com/apache/hudi/issues/6465#issuecomment-1224193860 It will be better if you can provide an example that fails or doesn't work because of this. -- This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [hudi] hudi-bot commented on pull request #6480: [HUDI-4687] add show_invalid_parquet procedure

2022-08-23 Thread GitBox
hudi-bot commented on PR #6480: URL: https://github.com/apache/hudi/pull/6480#issuecomment-1224188076 ## CI report: * 6b9041ed37d9eeb30cad27dfb1f51b9a608d36a9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1090

[jira] [Created] (HUDI-4702) Support updates and merges without primary key and precombine field

2022-08-23 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-4702: - Summary: Support updates and merges without primary key and precombine field Key: HUDI-4702 URL: https://issues.apache.org/jira/browse/HUDI-4702 Project: Apache Hudi

[jira] [Updated] (HUDI-4701) Support bulk insert without primary key and precombine field

2022-08-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4701: -- Fix Version/s: 0.13.0 > Support bulk insert without primary key and precombine field > -

[jira] [Created] (HUDI-4701) Support bulk insert without primary key and precombine field

2022-08-23 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-4701: - Summary: Support bulk insert without primary key and precombine field Key: HUDI-4701 URL: https://issues.apache.org/jira/browse/HUDI-4701 Project: Apache Hudi Iss

[GitHub] [hudi] YannByron commented on pull request #6476: [HUDI-3478] Support CDC for Spark in Hudi

2022-08-23 Thread GitBox
YannByron commented on PR #6476: URL: https://github.com/apache/hudi/pull/6476#issuecomment-1224147975 @prasannarajaperumal @xushiyan This pr has been updated according to the updated RFC: https://github.com/apache/hudi/pull/6256 please help to review this. -- This is an automated

[jira] [Created] (HUDI-4700) RFC for primary key-less data model

2022-08-23 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-4700: - Summary: RFC for primary key-less data model Key: HUDI-4700 URL: https://issues.apache.org/jira/browse/HUDI-4700 Project: Apache Hudi Issue Type: Task

[jira] [Updated] (HUDI-4700) RFC for primary key-less data model

2022-08-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4700: -- Fix Version/s: 0.13.0 > RFC for primary key-less data model > --- > >

[jira] [Updated] (HUDI-4699) Support for Primary key-less data model

2022-08-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4699: -- Description: Hudi requires users to specify a primary key field. Can we do away with this requirement? T

[jira] [Created] (HUDI-4699) Support for Primary key-less data model

2022-08-23 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-4699: - Summary: Support for Primary key-less data model Key: HUDI-4699 URL: https://issues.apache.org/jira/browse/HUDI-4699 Project: Apache Hudi Issue Type: Epic

[jira] [Updated] (HUDI-4699) Support for Primary key-less data model

2022-08-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4699: -- Component/s: writer-core > Support for Primary key-less data model > ---

[GitHub] [hudi] hudi-bot commented on pull request #6139: [HUDI-4396] Add a boolean parameter to decide whether the partition is cascade or not when hive table columns changes

2022-08-23 Thread GitBox
hudi-bot commented on PR #6139: URL: https://github.com/apache/hudi/pull/6139#issuecomment-1224092856 ## CI report: * a9e86bb7588ad10579e2ef02a0e41f1ce661aeaa Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1090

[GitHub] [hudi] pratyakshsharma commented on issue #6348: [SUPPORT] Hudi error while running HoodieMultiTableDeltaStreamer: Commit 20220809112130103 failed and rolled-back !

2022-08-23 Thread GitBox
pratyakshsharma commented on issue #6348: URL: https://github.com/apache/hudi/issues/6348#issuecomment-1224053630 did you supply the continuous flag as well? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [hudi] hudi-bot commented on pull request #6476: [HUDI-3478] Support CDC for Spark in Hudi

2022-08-23 Thread GitBox
hudi-bot commented on PR #6476: URL: https://github.com/apache/hudi/pull/6476#issuecomment-1224019319 ## CI report: * 0bc3211940a582e7186017a49fbb83813cc4ec11 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1089

[GitHub] [hudi] hudi-bot commented on pull request #6480: [HUDI-4687] add show_invalid_parquet procedure

2022-08-23 Thread GitBox
hudi-bot commented on PR #6480: URL: https://github.com/apache/hudi/pull/6480#issuecomment-1224013234 ## CI report: * 6b9041ed37d9eeb30cad27dfb1f51b9a608d36a9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1090

[GitHub] [hudi] hudi-bot commented on pull request #6480: [HUDI-4687] add show_invalid_parquet procedure

2022-08-23 Thread GitBox
hudi-bot commented on PR #6480: URL: https://github.com/apache/hudi/pull/6480#issuecomment-1224007037 ## CI report: * 6b9041ed37d9eeb30cad27dfb1f51b9a608d36a9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1090

[GitHub] [hudi] ROOBALJINDAL commented on issue #6348: [SUPPORT] Hudi error while running HoodieMultiTableDeltaStreamer: Commit 20220809112130103 failed and rolled-back !

2022-08-23 Thread GitBox
ROOBALJINDAL commented on issue #6348: URL: https://github.com/apache/hudi/issues/6348#issuecomment-1223963932 @rmahindra123 thanks, it worked. But I added 2 tables for ingestion but it is always picking first table mentioned in comma separated list. Can you tell what is missing? **K

[GitHub] [hudi] hudi-bot commented on pull request #6481: [HUDI-4698] Rename the package 'org.apache.flink.table.data' to avoid…

2022-08-23 Thread GitBox
hudi-bot commented on PR #6481: URL: https://github.com/apache/hudi/pull/6481#issuecomment-1223937258 ## CI report: * 3eb012affd4283f9970445bf3dbf4cb48afc25bf Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1090

[GitHub] [hudi] hudi-bot commented on pull request #6450: [HUDI-4665] Flipping default for "ignore failed batch" config in streaming sink to false

2022-08-23 Thread GitBox
hudi-bot commented on PR #6450: URL: https://github.com/apache/hudi/pull/6450#issuecomment-1223937054 ## CI report: * 50a075377f3723d1f8d4c222f653f0ae7446b28c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1089

[GitHub] [hudi] hudi-bot commented on pull request #6481: [HUDI-4698] Rename the package 'org.apache.flink.table.data' to avoid…

2022-08-23 Thread GitBox
hudi-bot commented on PR #6481: URL: https://github.com/apache/hudi/pull/6481#issuecomment-1223931891 ## CI report: * 3eb012affd4283f9970445bf3dbf4cb48afc25bf UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #6393: [HUDI-4619] Fix The retry mechanism of remotehoodietablefilesystemvie…

2022-08-23 Thread GitBox
hudi-bot commented on PR #6393: URL: https://github.com/apache/hudi/pull/6393#issuecomment-1223926360 ## CI report: * 09f49abeeca229df307426ba79bd77ed0392b79f UNKNOWN * fc88fa16b2fd11583d30ee3aa11e028c2cbf5709 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[jira] [Updated] (HUDI-4698) Rename the package 'org.apache.flink.table.data' to avoid conflicts with flink table core

2022-08-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4698: - Labels: pull-request-available (was: ) > Rename the package 'org.apache.flink.table.data' to avoi

[GitHub] [hudi] danny0405 opened a new pull request, #6481: [HUDI-4698] Rename the package 'org.apache.flink.table.data' to avoid…

2022-08-23 Thread GitBox
danny0405 opened a new pull request, #6481: URL: https://github.com/apache/hudi/pull/6481 … conflicts with flink table core ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or use

[jira] [Resolved] (HUDI-4686) Flip option 'write.ignore.failed' to default false

2022-08-23 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-4686. -- > Flip option 'write.ignore.failed' to default false > -- >

[jira] [Updated] (HUDI-4686) Flip option 'write.ignore.failed' to default false

2022-08-23 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-4686: - Fix Version/s: 0.12.1 > Flip option 'write.ignore.failed' to default false > -

[jira] [Commented] (HUDI-4686) Flip option 'write.ignore.failed' to default false

2022-08-23 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17583516#comment-17583516 ] Danny Chen commented on HUDI-4686: -- Fixed via master branch: 1879efa45d556c87d2cda56fa16d

[hudi] branch master updated: [HUDI-4686] Flip option 'write.ignore.failed' to default false (#6467)

2022-08-23 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 1879efa45d [HUDI-4686] Flip option 'write.ignor

[GitHub] [hudi] danny0405 merged pull request #6467: [HUDI-4686] Flip option 'write.ignore.failed' to default false

2022-08-23 Thread GitBox
danny0405 merged PR #6467: URL: https://github.com/apache/hudi/pull/6467 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache

[jira] [Created] (HUDI-4698) Rename the package 'org.apache.flink.table.data' to avoid conflicts with flink table core

2022-08-23 Thread Danny Chen (Jira)
Danny Chen created HUDI-4698: Summary: Rename the package 'org.apache.flink.table.data' to avoid conflicts with flink table core Key: HUDI-4698 URL: https://issues.apache.org/jira/browse/HUDI-4698 Project

[GitHub] [hudi] XuQianJin-Stars commented on pull request #6476: [HUDI-3478] Support CDC for Spark in Hudi

2022-08-23 Thread GitBox
XuQianJin-Stars commented on PR #6476: URL: https://github.com/apache/hudi/pull/6476#issuecomment-1223888947 nice work! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

[GitHub] [hudi] hudi-bot commented on pull request #6480: HUDI-4687 add show_invalid_parquet procedure

2022-08-23 Thread GitBox
hudi-bot commented on PR #6480: URL: https://github.com/apache/hudi/pull/6480#issuecomment-1223864667 ## CI report: * 6b9041ed37d9eeb30cad27dfb1f51b9a608d36a9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1090

[GitHub] [hudi] hudi-bot commented on pull request #6139: [HUDI-4396] Add a boolean parameter to decide whether the partition is cascade or not when hive table columns changes

2022-08-23 Thread GitBox
hudi-bot commented on PR #6139: URL: https://github.com/apache/hudi/pull/6139#issuecomment-1223863942 ## CI report: * 3687a0a01a0d31a6792d95aaf0536706d12fec76 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=101

[GitHub] [hudi] hudi-bot commented on pull request #6480: HUDI-4687 add show_invalid_parquet procedure

2022-08-23 Thread GitBox
hudi-bot commented on PR #6480: URL: https://github.com/apache/hudi/pull/6480#issuecomment-1223859393 ## CI report: * 6b9041ed37d9eeb30cad27dfb1f51b9a608d36a9 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #6139: [HUDI-4396] Add a boolean parameter to decide whether the partition is cascade or not when hive table columns changes

2022-08-23 Thread GitBox
hudi-bot commented on PR #6139: URL: https://github.com/apache/hudi/pull/6139#issuecomment-1223858704 ## CI report: * 3687a0a01a0d31a6792d95aaf0536706d12fec76 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=101

[GitHub] [hudi] hudi-bot commented on pull request #6352: [HUDI-4584] Fixing `SQLConf` not being propagated to executor

2022-08-23 Thread GitBox
hudi-bot commented on PR #6352: URL: https://github.com/apache/hudi/pull/6352#issuecomment-1223853439 ## CI report: * 285ed43d8ca7b525c3bd5334a0acd19ec2c7757c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1089

[GitHub] [hudi] ROOBALJINDAL commented on issue #6348: [SUPPORT] Hudi error while running HoodieMultiTableDeltaStreamer: Commit 20220809112130103 failed and rolled-back !

2022-08-23 Thread GitBox
ROOBALJINDAL commented on issue #6348: URL: https://github.com/apache/hudi/issues/6348#issuecomment-1223851452 @rmahindra123 then I get this error: ``` 22/08/23 10:05:17 INFO DebeziumSource: About to read 1 from Kafka for topic :ROOBJIN-LW13206.dbo.rrmprvcachemaonppes 22/08/23 1

[jira] [Updated] (HUDI-4687) Avoid all illegal reflective access in the code

2022-08-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4687: - Labels: jdk pull-request-available reflection writer (was: jdk reflection writer) > Avoid all il

[GitHub] [hudi] microbearz opened a new pull request, #6480: HUDI-4687 add show_invalid_parquet procedure

2022-08-23 Thread GitBox
microbearz opened a new pull request, #6480: URL: https://github.com/apache/hudi/pull/6480 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any performan

[jira] [Updated] (HUDI-4549) hive sync bundle causes class loader issue

2022-08-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4549: -- Status: In Progress (was: Open) > hive sync bundle causes class loader issue >

[jira] [Updated] (HUDI-4549) hive sync bundle causes class loader issue

2022-08-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4549: -- Status: Patch Available (was: In Progress) > hive sync bundle causes class loader issue > -

[GitHub] [hudi] hudi-bot commented on pull request #6450: [HUDI-4665] Flipping default for "ignore failed batch" config in streaming sink to false

2022-08-23 Thread GitBox
hudi-bot commented on PR #6450: URL: https://github.com/apache/hudi/pull/6450#issuecomment-1223780515 ## CI report: * 3bd700dea82006f1d3081c3eee7ab1b430728911 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1088

[GitHub] [hudi] hudi-bot commented on pull request #6476: [HUDI-3478] Support CDC for Spark in Hudi

2022-08-23 Thread GitBox
hudi-bot commented on PR #6476: URL: https://github.com/apache/hudi/pull/6476#issuecomment-1223780671 ## CI report: * 1fd639d2941b41ea33f076cc249539d34514046d Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1089

[GitHub] [hudi] hudi-bot commented on pull request #6393: [HUDI-4619] Fix The retry mechanism of remotehoodietablefilesystemvie…

2022-08-23 Thread GitBox
hudi-bot commented on PR #6393: URL: https://github.com/apache/hudi/pull/6393#issuecomment-1223780311 ## CI report: * 09f49abeeca229df307426ba79bd77ed0392b79f UNKNOWN * 4c7dcf78cdee3e26dbf291dc49f8ac64a05c8c60 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[jira] [Assigned] (HUDI-4697) Support show_invalid_parquet command based on Call Produce Command

2022-08-23 Thread jimmyz (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jimmyz reassigned HUDI-4697: Assignee: jimmyz > Support show_invalid_parquet command based on Call Produce Command > ---

[jira] [Created] (HUDI-4697) Support show_invalid_parquet command based on Call Produce Command

2022-08-23 Thread jimmyz (Jira)
jimmyz created HUDI-4697: Summary: Support show_invalid_parquet command based on Call Produce Command Key: HUDI-4697 URL: https://issues.apache.org/jira/browse/HUDI-4697 Project: Apache Hudi Issue T

<    1   2   3   >