[GitHub] [hudi] hudi-bot commented on pull request #8787: [HUDI-6254] Allow using absolute path in ManifestFileWriter

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8787: URL: https://github.com/apache/hudi/pull/8787#issuecomment-1560367058 ## CI report: * 21bf7ce299813e6b964327707f3415dd529be2a1 Azure:

[GitHub] [hudi] danny0405 commented on a diff in pull request #8782: [HUDI-6201] use concurrent map when possible in filesystemview

2023-05-23 Thread via GitHub
danny0405 commented on code in PR #8782: URL: https://github.com/apache/hudi/pull/8782#discussion_r1203314463 ## hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemView.java: ## @@ -199,7 +201,7 @@ protected boolean

[GitHub] [hudi] zhangyue19921010 opened a new pull request, #8790: [DNM][Test CI][TEST] Hudi 3088 default spark32 3

2023-05-23 Thread via GitHub
zhangyue19921010 opened a new pull request, #8790: URL: https://github.com/apache/hudi/pull/8790 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any

[GitHub] [hudi] fujianhua168 commented on issue #8754: [SUPPORT] PrestoDB encountered data quality issues while reading the Hudi Mor table.

2023-05-23 Thread via GitHub
fujianhua168 commented on issue #8754: URL: https://github.com/apache/hudi/issues/8754#issuecomment-1560360587 > Here's a branch in my Trino fork which has MOR Snapshot query support - https://github.com/codope/trino/tree/mor-snapshot-async-split I will verify the Trino snapshot query

[jira] [Closed] (HUDI-6096) hoodie.properties does not update when write config uses a new table name

2023-05-23 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason closed HUDI-6096. Resolution: Abandoned > hoodie.properties does not update when write config uses a new table name >

[GitHub] [hudi] prashantwason commented on pull request #8492: [HUDI-6096] Update table name in hoodie.properties from the write config when it is changed.

2023-05-23 Thread via GitHub
prashantwason commented on PR #8492: URL: https://github.com/apache/hudi/pull/8492#issuecomment-1560296284 Abandoning as the consensus is to not update this automatically. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] prashantwason closed pull request #8492: [HUDI-6096] Update table name in hoodie.properties from the write config when it is changed.

2023-05-23 Thread via GitHub
prashantwason closed pull request #8492: [HUDI-6096] Update table name in hoodie.properties from the write config when it is changed. URL: https://github.com/apache/hudi/pull/8492 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] hudi-bot commented on pull request #8782: [HUDI-6201] use concurrent map when possible in filesystemview

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8782: URL: https://github.com/apache/hudi/pull/8782#issuecomment-1560292873 ## CI report: * a279e36e052f9c06dabbfa908685fcf8a7991cf2 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8789: [HUDI-4932] Add partition inference config

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8789: URL: https://github.com/apache/hudi/pull/8789#issuecomment-1560288419 ## CI report: * 77a0ab2ed5291354c484ce4f364c599d6927cd43 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8782: [HUDI-6201] use concurrent map when possible in filesystemview

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8782: URL: https://github.com/apache/hudi/pull/8782#issuecomment-1560288351 ## CI report: * 395e5a0d3310a8b35347c179e886e6831931f516 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8789: [HUDI-4932] Add partition inference config

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8789: URL: https://github.com/apache/hudi/pull/8789#issuecomment-1560281708 ## CI report: * 77a0ab2ed5291354c484ce4f364c599d6927cd43 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] vinothchandar commented on pull request #8679: [DOCS] [RFC-69] Hudi 1.X

2023-05-23 Thread via GitHub
vinothchandar commented on PR #8679: URL: https://github.com/apache/hudi/pull/8679#issuecomment-1560281194 Folks - I have cleaned up a lot of items and streamlined most of the format, concurrency, metadata level changes - anything that affects storage bits and APIs,

[GitHub] [hudi] jonvex closed pull request #8440: [DO NOT MERGE] run gh actions with java 17

2023-05-23 Thread via GitHub
jonvex closed pull request #8440: [DO NOT MERGE] run gh actions with java 17 URL: https://github.com/apache/hudi/pull/8440 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] jonvex closed pull request #8439: [DO NOT MERGE] run tests with java 11

2023-05-23 Thread via GitHub
jonvex closed pull request #8439: [DO NOT MERGE] run tests with java 11 URL: https://github.com/apache/hudi/pull/8439 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] jonvex closed pull request #7947: [MINOR] DO NOT MERGE try reporting action

2023-05-23 Thread via GitHub
jonvex closed pull request #7947: [MINOR] DO NOT MERGE try reporting action URL: https://github.com/apache/hudi/pull/7947 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[jira] [Updated] (HUDI-4932) Add a config to allow partition column type inference in bootstrap

2023-05-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4932: - Labels: pull-request-available (was: ) > Add a config to allow partition column type inference

[GitHub] [hudi] jonvex opened a new pull request, #8789: [HUDI-4932] Add partition inference config

2023-05-23 Thread via GitHub
jonvex opened a new pull request, #8789: URL: https://github.com/apache/hudi/pull/8789 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any performance

[jira] [Updated] (HUDI-4932) Add a config to allow partition column type inference in bootstrap

2023-05-23 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-4932: -- Status: In Progress (was: Open) > Add a config to allow partition column type inference in

[GitHub] [hudi] hudi-bot commented on pull request #8303: [HUDI-5998] Speed up reads from bootstrapped tables in spark

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8303: URL: https://github.com/apache/hudi/pull/8303#issuecomment-1560248990 ## CI report: * f361b40cba23c728338a5163b0c00c50ac6c60b8 UNKNOWN * 27375abd2d676eb530d0ee2d2803efddce0bb92c Azure:

[GitHub] [hudi] the-other-tim-brown commented on a diff in pull request #8574: [HUDI-6139] Add support for Transformer schema validation in DeltaStreamer

2023-05-23 Thread via GitHub
the-other-tim-brown commented on code in PR #8574: URL: https://github.com/apache/hudi/pull/8574#discussion_r1203094395 ## hudi-utilities/src/main/java/org/apache/hudi/utilities/UtilHelpers.java: ## @@ -191,11 +192,14 @@ public static SchemaPostProcessor

[GitHub] [hudi] hudi-bot commented on pull request #8788: [DNM][MINOR] Add Github Actions to automatically add issues and PRs to support projects

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8788: URL: https://github.com/apache/hudi/pull/8788#issuecomment-1560210393 ## CI report: * e08ca8c8a6bbc2b52aee19bc9a1bd3f4192b5b9f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8788: [DNM][MINOR] Add Github Actions to automatically add issues and PRs to support projects

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8788: URL: https://github.com/apache/hudi/pull/8788#issuecomment-1560204217 ## CI report: * e08ca8c8a6bbc2b52aee19bc9a1bd3f4192b5b9f UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] yihua closed pull request #8788: [MINOR] Add Github Actions to automatically add issues and PRs to support projects

2023-05-23 Thread via GitHub
yihua closed pull request #8788: [MINOR] Add Github Actions to automatically add issues and PRs to support projects URL: https://github.com/apache/hudi/pull/8788 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] yihua opened a new pull request, #8788: [MINOR] Add Github Actions to automatically add issues and PRs to support projects

2023-05-23 Thread via GitHub
yihua opened a new pull request, #8788: URL: https://github.com/apache/hudi/pull/8788 ### Change Logs As above. ### Impact Makes the project management easier. ### Risk level none ### Documentation Update N/A ### Contributor's checklist

[GitHub] [hudi] hudi-bot commented on pull request #8574: [HUDI-6139] Add support for Transformer schema validation in DeltaStreamer

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8574: URL: https://github.com/apache/hudi/pull/8574#issuecomment-1560146505 ## CI report: * 4550fea4dfa7a73ae3face52bfc66d4b46adac37 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8604: [HUDI-6151] Rollback previously applied commits to MDT when operations are retried.

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8604: URL: https://github.com/apache/hudi/pull/8604#issuecomment-1560138691 ## CI report: * f1653d9899f1c925e1d662ea8ee6ae26edae573b Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8574: [HUDI-6139] Add support for Transformer schema validation in DeltaStreamer

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8574: URL: https://github.com/apache/hudi/pull/8574#issuecomment-1560138599 ## CI report: * 4550fea4dfa7a73ae3face52bfc66d4b46adac37 Azure:

[GitHub] [hudi] lokeshj1703 commented on pull request #8574: [HUDI-6139] Add support for Transformer schema validation in DeltaStreamer

2023-05-23 Thread via GitHub
lokeshj1703 commented on PR #8574: URL: https://github.com/apache/hudi/pull/8574#issuecomment-1560132333 I haven't made the schema validation changes in `ErrorTableAwareChainedTransformer` which was added recently. Can address that in a separate PR. -- This is an automated message from

[GitHub] [hudi] hudi-bot commented on pull request #8604: [HUDI-6151] Rollback previously applied commits to MDT when operations are retried.

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8604: URL: https://github.com/apache/hudi/pull/8604#issuecomment-1560130778 ## CI report: * f1653d9899f1c925e1d662ea8ee6ae26edae573b Azure:

[hudi] branch master updated: [HUDI-6190] Adjust description in the HoodieTableFactory.checkRecordKey exception (#8688)

2023-05-23 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new e2adc502a2a [HUDI-6190] Adjust description in the

[GitHub] [hudi] yihua merged pull request #8688: [HUDI-6190] Append description in the HoodieTableFactory.checkRecordKey exception.

2023-05-23 Thread via GitHub
yihua merged PR #8688: URL: https://github.com/apache/hudi/pull/8688 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] yihua commented on pull request #8688: [HUDI-6190] Append description in the HoodieTableFactory.checkRecordKey exception.

2023-05-23 Thread via GitHub
yihua commented on PR #8688: URL: https://github.com/apache/hudi/pull/8688#issuecomment-1560125331 CI is green. https://github.com/apache/hudi/assets/2497195/c2c83bbc-6635-421d-9d5c-c7e46ae5da31;> -- This is an automated message from the Apache Git Service. To respond to the

[hudi] branch master updated: [MINOR] Fix some typos and delete unused parameter (#8642)

2023-05-23 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 29bca0085f3 [MINOR] Fix some typos and delete

[GitHub] [hudi] yihua merged pull request #8642: [MINOR] Fix some typos and delete unused parameter

2023-05-23 Thread via GitHub
yihua merged PR #8642: URL: https://github.com/apache/hudi/pull/8642 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] yihua commented on pull request #8642: [MINOR] Fix some typos and delete unused parameter

2023-05-23 Thread via GitHub
yihua commented on PR #8642: URL: https://github.com/apache/hudi/pull/8642#issuecomment-1560123742 CI is green. https://github.com/apache/hudi/assets/2497195/6e9d1ef3-b704-4a63-bebf-b5ef9cd70ce0;> -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hudi] yihua commented on pull request #8604: [HUDI-6151] Rollback previously applied commits to MDT when operations are retried.

2023-05-23 Thread via GitHub
yihua commented on PR #8604: URL: https://github.com/apache/hudi/pull/8604#issuecomment-1560087006 @danny0405 @nsivabalan @prashantwason I rebased the PR on the latest PR. Once CI passes, we can land this. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hudi] yihua commented on pull request #8669: [HUDI-5362] Rebase IncrementalRelation over HoodieBaseRelation

2023-05-23 Thread via GitHub
yihua commented on PR #8669: URL: https://github.com/apache/hudi/pull/8669#issuecomment-1560083521 Is this a duplicate of #6045? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8487: URL: https://github.com/apache/hudi/pull/8487#issuecomment-1560083261 ## CI report: * 0db49c70e22e0f50b93390786f6a877295f680ec Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8487: URL: https://github.com/apache/hudi/pull/8487#issuecomment-1560072002 ## CI report: * 0db49c70e22e0f50b93390786f6a877295f680ec Azure:

[jira] [Closed] (HUDI-6098) Initial commit in MDT should use bulk insert for performance

2023-05-23 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason closed HUDI-6098. Resolution: Abandoned https://github.com/apache/hudi/pull/8684 > Initial commit in MDT should use

[GitHub] [hudi] prashantwason commented on pull request #8493: [HUDI-6098] Use bulk insert prepped for the initial write into MDT.

2023-05-23 Thread via GitHub
prashantwason commented on PR #8493: URL: https://github.com/apache/hudi/pull/8493#issuecomment-1560052432 Closing this as I have added the changes in another PR: https://github.com/apache/hudi/pull/8684 -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hudi] prashantwason closed pull request #8493: [HUDI-6098] Use bulk insert prepped for the initial write into MDT.

2023-05-23 Thread via GitHub
prashantwason closed pull request #8493: [HUDI-6098] Use bulk insert prepped for the initial write into MDT. URL: https://github.com/apache/hudi/pull/8493 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hudi] prashantwason commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.

2023-05-23 Thread via GitHub
prashantwason commented on PR #8487: URL: https://github.com/apache/hudi/pull/8487#issuecomment-1560048780 @nsivabalan Addressed all your feedback. PTAL. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hudi] prashantwason commented on a diff in pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.

2023-05-23 Thread via GitHub
prashantwason commented on code in PR #8487: URL: https://github.com/apache/hudi/pull/8487#discussion_r1202930477 ## hudi-utilities/src/test/java/org/apache/hudi/utilities/deltastreamer/TestHoodieDeltaStreamer.java: ## @@ -2506,13 +2507,21 @@ void

[jira] [Assigned] (HUDI-4932) Add a config to allow partition column type inference in bootstrap

2023-05-23 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler reassigned HUDI-4932: - Assignee: Jonathan Vexler (was: Ethan Guo) > Add a config to allow partition column

[GitHub] [hudi] hudi-bot commented on pull request #8787: [HUDI-6254] Allow using absolute path in ManifestFileWriter

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8787: URL: https://github.com/apache/hudi/pull/8787#issuecomment-1559946881 ## CI report: * 21bf7ce299813e6b964327707f3415dd529be2a1 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8638: added new exception types

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8638: URL: https://github.com/apache/hudi/pull/8638#issuecomment-1559946073 ## CI report: * c8cf2d86b1be30d3215b3b6e89b8bda33a1fe5dc UNKNOWN * 333d9faa53e71ba535a7cb8c60ce8b350a33452c UNKNOWN * 2b0d0627582948948f230f4eeaa07e9289827d7e Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8787: [HUDI-6254] Allow using absolute path in ManifestFileWriter

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8787: URL: https://github.com/apache/hudi/pull/8787#issuecomment-1559937739 ## CI report: * 21bf7ce299813e6b964327707f3415dd529be2a1 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #8688: [HUDI-6190] Append description in the HoodieTableFactory.checkRecordKey exception.

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8688: URL: https://github.com/apache/hudi/pull/8688#issuecomment-1559936733 ## CI report: * 3b233a8683b3daa7f7168c29ec6eb901f0581b56 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8638: added new exception types

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8638: URL: https://github.com/apache/hudi/pull/8638#issuecomment-1559936091 ## CI report: * c8cf2d86b1be30d3215b3b6e89b8bda33a1fe5dc UNKNOWN * 333d9faa53e71ba535a7cb8c60ce8b350a33452c UNKNOWN * 6cbafd0d08e12fc4e77a9f0058fe24b23e352a69 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8688: [HUDI-6190] Append description in the HoodieTableFactory.checkRecordKey exception.

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8688: URL: https://github.com/apache/hudi/pull/8688#issuecomment-1559923296 ## CI report: * 3b233a8683b3daa7f7168c29ec6eb901f0581b56 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8642: [MINOR] Fix some typos and delete unused parameter

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8642: URL: https://github.com/apache/hudi/pull/8642#issuecomment-1559923009 ## CI report: * 1f0ee57e9a9388a3b347b6ad4a73e764532fa7cb Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8445: [HUDI-3088] Use Spark 3.2 as default Spark version

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8445: URL: https://github.com/apache/hudi/pull/8445#issuecomment-1559922325 ## CI report: * 0bc1665bcdb60973799ec9de9356a77caab57e57 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8076: [HUDI-5884] Support bulk_insert for insert_overwrite and insert_overwrite_table

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8076: URL: https://github.com/apache/hudi/pull/8076#issuecomment-1559921592 ## CI report: * 6a239ada8998fd440f19c0082b26d206ed589870 UNKNOWN * 03ad0c018d1a929b55d30934d74c9ba84509e88b Azure:

[hudi] branch master updated (bce768bd241 -> ed1fc6e7a93)

2023-05-23 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from bce768bd241 [HUDI-6213] Parallelize deletion of files during rollback. (#8717) add ed1fc6e7a93 [HUDI-6197] Fix use

[GitHub] [hudi] yihua merged pull request #8689: [HUDI-6197] Fix use CONTAINER_ID to judge hudi is running on yarn

2023-05-23 Thread via GitHub
yihua merged PR #8689: URL: https://github.com/apache/hudi/pull/8689 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] xushiyan commented on a diff in pull request #8679: [DOCS] [RFC-69] Hudi 1.X

2023-05-23 Thread via GitHub
xushiyan commented on code in PR #8679: URL: https://github.com/apache/hudi/pull/8679#discussion_r1202790592 ## rfc/rfc-69/rfc-69.md: ## @@ -0,0 +1,159 @@ + +# RFC-69: Hudi 1.X + +## Proposers + +* Vinoth Chandar + +## Approvers + +* Hudi PMC + +## Status + +Under Review +

[jira] [Created] (HUDI-6255) Web UI for platformization

2023-05-23 Thread Raymond Xu (Jira)
Raymond Xu created HUDI-6255: Summary: Web UI for platformization Key: HUDI-6255 URL: https://issues.apache.org/jira/browse/HUDI-6255 Project: Apache Hudi Issue Type: Epic Components:

[GitHub] [hudi] prashantwason commented on pull request #8430: [HUDI-6060] Added a config to backup instants before deletion during rollbacks and restores.

2023-05-23 Thread via GitHub
prashantwason commented on PR #8430: URL: https://github.com/apache/hudi/pull/8430#issuecomment-1559891391 @nsivabalan I have address all your feedback. Ready to merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] amrishlal commented on pull request #8759: Add metrics counters for compaction start/stop events.

2023-05-23 Thread via GitHub
amrishlal commented on PR #8759: URL: https://github.com/apache/hudi/pull/8759#issuecomment-1559881452 @SteNicholas Just pinging to see if you are ok with this PR based on @nsivabalan comments above? -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hudi] jp0317 opened a new pull request, #8787: [HUDI-6254] Allow using absolute path in ManifestFileWriter

2023-05-23 Thread via GitHub
jp0317 opened a new pull request, #8787: URL: https://github.com/apache/hudi/pull/8787 ### Change Logs Allow writing the manifest file with absolute path in ManifestFileWriter. Currently the writer only uses the file name (excluding the full path). ### Impact This

[GitHub] [hudi] hudi-bot commented on pull request #8782: [HUDI-6201] use concurrent map when possible in filesystemview

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8782: URL: https://github.com/apache/hudi/pull/8782#issuecomment-1559869426 ## CI report: * 395e5a0d3310a8b35347c179e886e6831931f516 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8642: [MINOR] Fix some typos and delete unused parameter

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8642: URL: https://github.com/apache/hudi/pull/8642#issuecomment-1559868711 ## CI report: * 1f0ee57e9a9388a3b347b6ad4a73e764532fa7cb Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8638: added new exception types

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8638: URL: https://github.com/apache/hudi/pull/8638#issuecomment-1559868609 ## CI report: * c8cf2d86b1be30d3215b3b6e89b8bda33a1fe5dc UNKNOWN * 333d9faa53e71ba535a7cb8c60ce8b350a33452c UNKNOWN * 6cbafd0d08e12fc4e77a9f0058fe24b23e352a69 Azure:

[GitHub] [hudi] yihua commented on pull request #8689: [HUDI-6197] Fix use CONTAINER_ID to judge hudi is running on yarn

2023-05-23 Thread via GitHub
yihua commented on PR #8689: URL: https://github.com/apache/hudi/pull/8689#issuecomment-1559865323 Hi @Akihito-Liang Thanks for your first contribution. Please remember to fill in all information in the PR description to pass the Github action `validate pr / validate-pr (pull_request)`.

[GitHub] [hudi] amrishlal commented on a diff in pull request #8759: Add metrics counters for compaction start/stop events.

2023-05-23 Thread via GitHub
amrishlal commented on code in PR #8759: URL: https://github.com/apache/hudi/pull/8759#discussion_r1202747051 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/compact/RunCompactionActionExecutor.java: ## @@ -65,10 +73,14 @@ public

[GitHub] [hudi] hudi-bot commented on pull request #8638: added new exception types

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8638: URL: https://github.com/apache/hudi/pull/8638#issuecomment-1559857438 ## CI report: * c8cf2d86b1be30d3215b3b6e89b8bda33a1fe5dc UNKNOWN * 333d9faa53e71ba535a7cb8c60ce8b350a33452c UNKNOWN * 6cbafd0d08e12fc4e77a9f0058fe24b23e352a69 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8782: [HUDI-6201] use concurrent map when possible in filesystemview

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8782: URL: https://github.com/apache/hudi/pull/8782#issuecomment-1559859064 ## CI report: * 395e5a0d3310a8b35347c179e886e6831931f516 Azure:

[hudi] branch master updated (f04f9597840 -> bce768bd241)

2023-05-23 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from f04f9597840 [MINOR] Avoid synchronized block in HoodieLockMetrics if key is present in cache (#8778) add

[GitHub] [hudi] yihua merged pull request #8717: [HUDI-6213] Parallelize deletion of files during rollback.

2023-05-23 Thread via GitHub
yihua merged PR #8717: URL: https://github.com/apache/hudi/pull/8717 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[hudi] branch master updated (72fffddb695 -> f04f9597840)

2023-05-23 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 72fffddb695 [HUDI-6232] Add option to skip table archival in glue sync client (#8744) add f04f9597840 [MINOR]

[GitHub] [hudi] yihua merged pull request #8778: [MINOR] Avoid synchronized block in HoodieLockMetrics if key is present in cache

2023-05-23 Thread via GitHub
yihua merged PR #8778: URL: https://github.com/apache/hudi/pull/8778 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[hudi] branch master updated (0735dea6d8e -> 72fffddb695)

2023-05-23 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 0735dea6d8e [HUDI-4944] Do not decode URI twice in HoodieBootstrapRDD (#8618) add 72fffddb695 [HUDI-6232] Add

[GitHub] [hudi] yihua merged pull request #8744: [HUDI-6232] Add option to skip table archival in glue sync client

2023-05-23 Thread via GitHub
yihua merged PR #8744: URL: https://github.com/apache/hudi/pull/8744 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] yihua closed pull request #8693: [DNM][HUDI-6204] Test bundle validation on Spark 3.3.2 with older commits

2023-05-23 Thread via GitHub
yihua closed pull request #8693: [DNM][HUDI-6204] Test bundle validation on Spark 3.3.2 with older commits URL: https://github.com/apache/hudi/pull/8693 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hudi] yihua commented on pull request #8693: [DNM][HUDI-6204] Test bundle validation on Spark 3.3.2 with older commits

2023-05-23 Thread via GitHub
yihua commented on PR #8693: URL: https://github.com/apache/hudi/pull/8693#issuecomment-1559850885 Closing this which is for testing only. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] hudi-bot commented on pull request #8303: [HUDI-5998] Speed up reads from bootstrapped tables in spark

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8303: URL: https://github.com/apache/hudi/pull/8303#issuecomment-1559844835 ## CI report: * b8772a74388873c35b1a13ba6ef99ecda9246646 Azure:

[GitHub] [hudi] the-other-tim-brown commented on a diff in pull request #8638: added new exception types

2023-05-23 Thread via GitHub
the-other-tim-brown commented on code in PR #8638: URL: https://github.com/apache/hudi/pull/8638#discussion_r1202720999 ## hudi-common/src/main/java/org/apache/hudi/exception/HoodieMetaSyncException.java: ## @@ -0,0 +1,29 @@ +/* + * Licensed to the Apache Software Foundation

[jira] [Created] (HUDI-6254) Allow using absolute path in ManifestFileWriter

2023-05-23 Thread Jinpeng Zhou (Jira)
Jinpeng Zhou created HUDI-6254: -- Summary: Allow using absolute path in ManifestFileWriter Key: HUDI-6254 URL: https://issues.apache.org/jira/browse/HUDI-6254 Project: Apache Hudi Issue Type:

[GitHub] [hudi] jonvex commented on a diff in pull request #8638: added new exception types

2023-05-23 Thread via GitHub
jonvex commented on code in PR #8638: URL: https://github.com/apache/hudi/pull/8638#discussion_r1202703409 ## hudi-common/src/main/java/org/apache/hudi/exception/HoodieMetaSyncException.java: ## @@ -0,0 +1,29 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [hudi] jonvex commented on a diff in pull request #8638: added new exception types

2023-05-23 Thread via GitHub
jonvex commented on code in PR #8638: URL: https://github.com/apache/hudi/pull/8638#discussion_r1202703409 ## hudi-common/src/main/java/org/apache/hudi/exception/HoodieMetaSyncException.java: ## @@ -0,0 +1,29 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [hudi] jonvex commented on a diff in pull request #8638: added new exception types

2023-05-23 Thread via GitHub
jonvex commented on code in PR #8638: URL: https://github.com/apache/hudi/pull/8638#discussion_r1202697891 ## hudi-utilities/src/main/java/org/apache/hudi/utilities/schema/RowBasedSchemaProvider.java: ## @@ -44,7 +45,12 @@ public RowBasedSchemaProvider(StructType rowStruct) {

[GitHub] [hudi] jonvex commented on a diff in pull request #8638: added new exception types

2023-05-23 Thread via GitHub
jonvex commented on code in PR #8638: URL: https://github.com/apache/hudi/pull/8638#discussion_r1202691156 ## hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionUtils.scala: ## @@ -138,18 +139,26 @@ object AvroConversionUtils { def

[jira] [Closed] (HUDI-4944) The encoded slash (%2F) in partition path is not properly decoded during Spark read

2023-05-23 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler closed HUDI-4944. - Resolution: Fixed > The encoded slash (%2F) in partition path is not properly decoded during >

[jira] [Assigned] (HUDI-4944) The encoded slash (%2F) in partition path is not properly decoded during Spark read

2023-05-23 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler reassigned HUDI-4944: - Assignee: Jonathan Vexler > The encoded slash (%2F) in partition path is not properly

[GitHub] [hudi] hudi-bot commented on pull request #8303: [HUDI-5998] Speed up reads from bootstrapped tables in spark

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8303: URL: https://github.com/apache/hudi/pull/8303#issuecomment-1559788925 ## CI report: * b8772a74388873c35b1a13ba6ef99ecda9246646 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8303: [HUDI-5998] Speed up reads from bootstrapped tables in spark

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8303: URL: https://github.com/apache/hudi/pull/8303#issuecomment-1559774438 ## CI report: * b8772a74388873c35b1a13ba6ef99ecda9246646 Azure:

[GitHub] [hudi] jonvex commented on a diff in pull request #8303: [HUDI-5998] Speed up reads from bootstrapped tables in spark

2023-05-23 Thread via GitHub
jonvex commented on code in PR #8303: URL: https://github.com/apache/hudi/pull/8303#discussion_r1202649672 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieBootstrapRelation.scala: ## @@ -188,11 +188,23 @@ case class

[jira] [Created] (HUDI-6253) Treat full bootstrap table as regular table

2023-05-23 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-6253: - Summary: Treat full bootstrap table as regular table Key: HUDI-6253 URL: https://issues.apache.org/jira/browse/HUDI-6253 Project: Apache Hudi Issue Type:

[GitHub] [hudi] hudi-bot commented on pull request #8669: [HUDI-5362] Rebase IncrementalRelation over HoodieBaseRelation

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8669: URL: https://github.com/apache/hudi/pull/8669#issuecomment-1559753980 ## CI report: * 0eacefd8bc063e0c574068f09670014804f10dc2 UNKNOWN * 743315d3a828ebbc9623ac3b17778cf2ff4e7d63 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8452: [HUDI-6077] Add more partition push down filters

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8452: URL: https://github.com/apache/hudi/pull/8452#issuecomment-1559753266 ## CI report: * 8082df232089396b2a9f9be2b915e51b3645f172 UNKNOWN * 197d58ce002e65cbe5969b2193fb0e8dffe7eac2 Azure:

[GitHub] [hudi] ad1happy2go commented on issue #7191: [SUPPORT] Missing Data with Amazon Athena in Glue Table with Hudi 0.10.1

2023-05-23 Thread via GitHub
ad1happy2go commented on issue #7191: URL: https://github.com/apache/hudi/issues/7191#issuecomment-1559749477 @aniketnanna After the above fix, its creating the partition with `__HIVE_DEFAULT_PARTITION__` and confirmed that Athena is not missing any data. Glue Code here -

[GitHub] [hudi] hbgstc123 closed pull request #8748: [HUDI-6234] make sure clean is run after flink table service

2023-05-23 Thread via GitHub
hbgstc123 closed pull request #8748: [HUDI-6234] make sure clean is run after flink table service URL: https://github.com/apache/hudi/pull/8748 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[jira] [Updated] (HUDI-6234) make sure clean is run after flink offline service

2023-05-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6234: - Labels: pull-request-available (was: ) > make sure clean is run after flink offline service >

[GitHub] [hudi] codope commented on a diff in pull request #8445: [HUDI-3088] Use Spark 3.2 as default Spark version

2023-05-23 Thread via GitHub
codope commented on code in PR #8445: URL: https://github.com/apache/hudi/pull/8445#discussion_r1202564227 ## hudi-client/hudi-client-common/src/test/java/org/apache/hudi/io/storage/TestHoodieHFileReaderWriter.java: ## @@ -198,10 +200,10 @@ public void

[GitHub] [hudi] yihua commented on a diff in pull request #8782: [HUDI-6201] use concurrent map when possible in filesystemview

2023-05-23 Thread via GitHub
yihua commented on code in PR #8782: URL: https://github.com/apache/hudi/pull/8782#discussion_r1202556828 ## hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemView.java: ## @@ -199,7 +201,7 @@ protected boolean

[GitHub] [hudi] hudi-bot commented on pull request #8786: [DNM][Test CI] Hudi 3088 default spark32 3

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8786: URL: https://github.com/apache/hudi/pull/8786#issuecomment-1559674975 ## CI report: * 2f5b3ed456190be74e4397255365da7b934adc66 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #8725: [HUDI-6219] Ensure consistency between Spark catalog schema and Hudi schema

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8725: URL: https://github.com/apache/hudi/pull/8725#issuecomment-1559659488 ## CI report: * 61db5e5a854227042d97e50706d5213f60041f0a Azure:

[GitHub] [hudi] xushiyan opened a new pull request, #8786: [DNM][Test CI] Hudi 3088 default spark32 3

2023-05-23 Thread via GitHub
xushiyan opened a new pull request, #8786: URL: https://github.com/apache/hudi/pull/8786 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any

[GitHub] [hudi] hudi-bot commented on pull request #8725: [HUDI-6219] Ensure consistency between Spark catalog schema and Hudi schema

2023-05-23 Thread via GitHub
hudi-bot commented on PR #8725: URL: https://github.com/apache/hudi/pull/8725#issuecomment-1559643867 ## CI report: * 61db5e5a854227042d97e50706d5213f60041f0a Azure:

<    1   2   3   >