[jira] [Assigned] (HUDI-5083) A bug occurs when the schema changes multiple times to a once existed column

2022-10-24 Thread shenshengli (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli reassigned HUDI-5083: - Assignee: shenshengli > A bug occurs when the schema changes multiple times to a once existed col

[jira] [Updated] (HUDI-5083) A bug occurs when the schema changes multiple times to a once existed column

2022-10-24 Thread shenshengli (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HUDI-5083: -- Description: (was: https://github.com/shenshengli/hudi/pull/1) > A bug occurs when the schema change

[jira] [Updated] (HUDI-5083) A bug occurs when the schema changes multiple times to a once existed column

2022-10-24 Thread shenshengli (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HUDI-5083: -- Description: https://github.com/shenshengli/hudi/pull/1 > A bug occurs when the schema changes multiple

[GitHub] [hudi] Zhangshunyu commented on issue #7032: [SUPPORT] When metatable enabled, some query using index column as filter will get empty result

2022-10-24 Thread GitBox
Zhangshunyu commented on issue #7032: URL: https://github.com/apache/hudi/issues/7032#issuecomment-1288515267 @alexeykudinkin @yihua @nsivabalan Could you pls have a look at this problem? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [hudi] shenshengli opened a new pull request, #7043: HUDI-5083 Fixed a bug when schema evolution

2022-10-24 Thread GitBox
shenshengli opened a new pull request, #7043: URL: https://github.com/apache/hudi/pull/7043 A bug occurs when the schema changes multiple times to a once existed column ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ###

[GitHub] [hudi] shenshengli closed pull request #7043: HUDI-5083 Fixed a bug when schema evolution

2022-10-24 Thread GitBox
shenshengli closed pull request #7043: HUDI-5083 Fixed a bug when schema evolution URL: https://github.com/apache/hudi/pull/7043 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[jira] [Updated] (HUDI-5083) A bug occurs when the schema changes multiple times to a once existed column

2022-10-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5083: - Labels: pull-request-available (was: ) > A bug occurs when the schema changes multiple times to a

[jira] [Created] (HUDI-5084) Upgrade Commons Text to 1.10.0

2022-10-24 Thread Jason-Morries Adam (Jira)
Jason-Morries Adam created HUDI-5084: Summary: Upgrade Commons Text to 1.10.0 Key: HUDI-5084 URL: https://issues.apache.org/jira/browse/HUDI-5084 Project: Apache Hudi Issue Type: Bug

[GitHub] [hudi] hudi-bot commented on pull request #6448: [HUDI-4647] Keep the hive sync settings in spark sql consistent

2022-10-24 Thread GitBox
hudi-bot commented on PR #6448: URL: https://github.com/apache/hudi/pull/6448#issuecomment-1288523075 ## CI report: * 2e3520f444fab7df0d9c91300f82df9321b7105a Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=124

[GitHub] [hudi] hudi-bot commented on pull request #6946: [HUDI-5027] Improve getHBaseConnection Use Constants Replace HardCode.

2022-10-24 Thread GitBox
hudi-bot commented on PR #6946: URL: https://github.com/apache/hudi/pull/6946#issuecomment-1288523973 ## CI report: * 86099181bd76a59cdd1b537eb724f6f51ed0c711 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1249

[GitHub] [hudi] hudi-bot commented on pull request #7001: [HUDI-5061] bulk insert operation don't throw other exception except IOE Exception

2022-10-24 Thread GitBox
hudi-bot commented on PR #7001: URL: https://github.com/apache/hudi/pull/7001#issuecomment-1288524232 ## CI report: * 67282ced98d0531a1096bcc418c0126836d0fb51 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1250

[GitHub] [hudi] hudi-bot commented on pull request #7020: [HUDI-5069] TestInlineCompaction.testSuccessfulCompactionBasedOnNumAndTime is flaky

2022-10-24 Thread GitBox
hudi-bot commented on PR #7020: URL: https://github.com/apache/hudi/pull/7020#issuecomment-1288524386 ## CI report: * 23754c9d84c66721016d846ca6a20614626baa35 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1250

[GitHub] [hudi] tooptoop4 commented on pull request #6707: [WIP][HUDI-4871] Upgrade spark3.3 to Spark 3.3.1

2022-10-24 Thread GitBox
tooptoop4 commented on PR #6707: URL: https://github.com/apache/hudi/pull/6707#issuecomment-1288525134 ready? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

[GitHub] [hudi] hbgstc123 commented on a diff in pull request #6976: [HUDI-5042]fix clustering schedule problem in flink

2022-10-24 Thread GitBox
hbgstc123 commented on code in PR #6976: URL: https://github.com/apache/hudi/pull/6976#discussion_r1002948270 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/util/StreamerUtil.java: ## @@ -171,7 +171,7 @@ public static HoodieWriteConfig getHoodieClientConfig(

[GitHub] [hudi] shenshengli opened a new pull request, #7044: HUDI-5083 Fixed a bug when schema evolution

2022-10-24 Thread GitBox
shenshengli opened a new pull request, #7044: URL: https://github.com/apache/hudi/pull/7044 A bug occurs when the schema changes multiple times to a once existed column -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [hudi] hudi-bot commented on pull request #6448: [HUDI-4647] Keep the hive sync settings in spark sql consistent

2022-10-24 Thread GitBox
hudi-bot commented on PR #6448: URL: https://github.com/apache/hudi/pull/6448#issuecomment-1288530255 ## CI report: * 2e3520f444fab7df0d9c91300f82df9321b7105a Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=124

[GitHub] [hudi] hudi-bot commented on pull request #6976: [HUDI-5042]fix clustering schedule problem in flink

2022-10-24 Thread GitBox
hudi-bot commented on PR #6976: URL: https://github.com/apache/hudi/pull/6976#issuecomment-1288531199 ## CI report: * 89e792414d09daaa8a367ecc4011450cc21e7069 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1238

[GitHub] [hudi] hudi-bot commented on pull request #6999: [HUDI-5057] Fix msck repair hudi table

2022-10-24 Thread GitBox
hudi-bot commented on PR #6999: URL: https://github.com/apache/hudi/pull/6999#issuecomment-1288531311 ## CI report: * 19f2ee38cba5f3ea77fa6c0a7f14a4a4f1344ad3 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=124

[GitHub] [hudi] shenshengli closed pull request #7044: HUDI-5083 Fixed a bug when schema evolution

2022-10-24 Thread GitBox
shenshengli closed pull request #7044: HUDI-5083 Fixed a bug when schema evolution URL: https://github.com/apache/hudi/pull/7044 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] hudi-bot commented on pull request #6989: [HUDI-5000] Support schema evolution for Hive/presto

2022-10-24 Thread GitBox
hudi-bot commented on PR #6989: URL: https://github.com/apache/hudi/pull/6989#issuecomment-1288538206 ## CI report: * 11d8108e89bc1de462978acbaee3905f9cb9edba Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=124

[GitHub] [hudi] hudi-bot commented on pull request #6999: [HUDI-5057] Fix msck repair hudi table

2022-10-24 Thread GitBox
hudi-bot commented on PR #6999: URL: https://github.com/apache/hudi/pull/6999#issuecomment-1288538266 ## CI report: * 19f2ee38cba5f3ea77fa6c0a7f14a4a4f1344ad3 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=124

[GitHub] [hudi] hudi-bot commented on pull request #6976: [HUDI-5042]fix clustering schedule problem in flink

2022-10-24 Thread GitBox
hudi-bot commented on PR #6976: URL: https://github.com/apache/hudi/pull/6976#issuecomment-1288538087 ## CI report: * c6954076cdb8818f7df54e297a9184c48c7217d0 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=125

[GitHub] [hudi] xushiyan commented on a diff in pull request #6824: [HUDI-4946] fix merge into with no preCombineField has dup row by onl…

2022-10-24 Thread GitBox
xushiyan commented on code in PR #6824: URL: https://github.com/apache/hudi/pull/6824#discussion_r1002955844 ## hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/MergeIntoHoodieTableCommand.scala: ## @@ -160,7 +167,7 @@ case class MergeIntoHoodieT

[GitHub] [hudi] shenshengli opened a new pull request, #7045: HUDI-5083 Fixed a bug when schema evolution

2022-10-24 Thread GitBox
shenshengli opened a new pull request, #7045: URL: https://github.com/apache/hudi/pull/7045 A bug occurs when the schema changes multiple times to a once existed column. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] xiaokxluoshu opened a new issue, #7046: [SUPPORT] hadoop use 3.2.2 ,There are so many java.lang.NoSuchMethodError

2022-10-24 Thread GitBox
xiaokxluoshu opened a new issue, #7046: URL: https://github.com/apache/hudi/issues/7046 **Environment Description** * Hudi version : 0.12.0 * Spark version : 2.4 * Flink version: 1.13.6 * Hive version : * Hadoop version : 3.2.2 * Storage (HDFS/S3

[GitHub] [hudi] xiaokxluoshu commented on issue #7046: [SUPPORT] hadoop use 3.2.2 ,There are so many java.lang.NoSuchMethodError

2022-10-24 Thread GitBox
xiaokxluoshu commented on issue #7046: URL: https://github.com/apache/hudi/issues/7046#issuecomment-1288544166 There is one more condition for this exception: response status 500 error occurs when a request is made to hadoop. Everything else will work. FSDataInputStreamWrapper updateInputSt

[GitHub] [hudi] hudi-bot commented on pull request #6989: [HUDI-5000] Support schema evolution for Hive/presto

2022-10-24 Thread GitBox
hudi-bot commented on PR #6989: URL: https://github.com/apache/hudi/pull/6989#issuecomment-1288545204 ## CI report: * 11d8108e89bc1de462978acbaee3905f9cb9edba Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=124

[jira] [Updated] (HUDI-5027) Replace hardcoded hbase config keys with HbaseConstants

2022-10-24 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-5027: - Summary: Replace hardcoded hbase config keys with HbaseConstants (was: Improve SparkHoodieHBaseIndex#get

[jira] [Updated] (HUDI-5027) Replace hardcoded hbase config keys with HbaseConstants

2022-10-24 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-5027: - Fix Version/s: 0.12.2 > Replace hardcoded hbase config keys with HbaseConstants > ---

[jira] [Updated] (HUDI-5027) Replace hardcoded hbase config keys with HbaseConstants

2022-10-24 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-5027: - Component/s: code-quality (was: cli) > Replace hardcoded hbase config keys with Hbase

[jira] [Updated] (HUDI-5027) Replace hardcoded hbase config keys with HbaseConstants

2022-10-24 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-5027: - Priority: Minor (was: Major) > Replace hardcoded hbase config keys with HbaseConstants > ---

[GitHub] [hudi] SteNicholas commented on pull request #6991: [HUDI-5049] HoodieCatalog supports the implementation of dropPartition

2022-10-24 Thread GitBox
SteNicholas commented on PR #6991: URL: https://github.com/apache/hudi/pull/6991#issuecomment-1288549663 @danny0405, I have addressed above comments. PTAL. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hudi] xushiyan merged pull request #6946: [HUDI-5027] Replace hardcoded hbase config keys with constant variables

2022-10-24 Thread GitBox
xushiyan merged PR #6946: URL: https://github.com/apache/hudi/pull/6946 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.

[hudi] branch master updated: [HUDI-5027] Replace hardcoded hbase config keys with constant variables (#6946)

2022-10-24 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 5cf8699297 [HUDI-5027] Replace hardcoded hbase c

[GitHub] [hudi] SteNicholas commented on pull request #6991: [HUDI-5049] HoodieCatalog supports the implementation of dropPartition

2022-10-24 Thread GitBox
SteNicholas commented on PR #6991: URL: https://github.com/apache/hudi/pull/6991#issuecomment-1288566933 @danny0405, I have addressed above comments. PTAL. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hudi] wangyum commented on pull request #6707: [HUDI-4871] Upgrade spark3.3 to Spark 3.3.1

2022-10-24 Thread GitBox
wangyum commented on PR #6707: URL: https://github.com/apache/hudi/pull/6707#issuecomment-1288601503 The Spark 3.3.1 vote passes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[jira] [Updated] (HUDI-5083) A bug occurs when the schema changes multiple times to a once existed column

2022-10-24 Thread shenshengli (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HUDI-5083: -- Description: https://github.com/apache/hudi/issues/7040 > A bug occurs when the schema changes multiple

[GitHub] [hudi] KnightChess commented on issue #6679: [SUPPORT] Expect job status failed in spark batch model

2022-10-24 Thread GitBox
KnightChess commented on issue #6679: URL: https://github.com/apache/hudi/issues/6679#issuecomment-1288632725 @nsivabalan thanks reply, I use 0.11.0 version. But we are batch job, not streaming job. Follow the config which you advice in code, I found the execption processing logic in stream

[GitHub] [hudi] hudi-bot commented on pull request #6999: [HUDI-5057] Fix msck repair hudi table

2022-10-24 Thread GitBox
hudi-bot commented on PR #6999: URL: https://github.com/apache/hudi/pull/6999#issuecomment-1288634673 ## CI report: * 19f2ee38cba5f3ea77fa6c0a7f14a4a4f1344ad3 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=124

[GitHub] [hudi] hudi-bot commented on pull request #7045: HUDI-5083 Fixed a bug when schema evolution

2022-10-24 Thread GitBox
hudi-bot commented on PR #7045: URL: https://github.com/apache/hudi/pull/7045#issuecomment-1288635283 ## CI report: * c326073206cf20ca528c674678608d8713db7f1e UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #6227: [HUDI-4496] Fixing Orc support broken for Spark 3.x and more

2022-10-24 Thread GitBox
hudi-bot commented on PR #6227: URL: https://github.com/apache/hudi/pull/6227#issuecomment-1288641984 ## CI report: * b236a680cc0abd5b1583c09570002bad77d9ad6f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1243

[GitHub] [hudi] hudi-bot commented on pull request #6707: [HUDI-4871] Upgrade spark3.3 to Spark 3.3.1

2022-10-24 Thread GitBox
hudi-bot commented on PR #6707: URL: https://github.com/apache/hudi/pull/6707#issuecomment-1288644145 ## CI report: * d0381797611ba0c237f21eaccd33815231833897 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #6991: [HUDI-5049] HoodieCatalog supports the implementation of dropPartition

2022-10-24 Thread GitBox
hudi-bot commented on PR #6991: URL: https://github.com/apache/hudi/pull/6991#issuecomment-1288645848 ## CI report: * fbb4c300133ad1c677826b836652152032b4c616 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1239

[GitHub] [hudi] hudi-bot commented on pull request #6999: [HUDI-5057] Fix msck repair hudi table

2022-10-24 Thread GitBox
hudi-bot commented on PR #6999: URL: https://github.com/apache/hudi/pull/6999#issuecomment-1288646008 ## CI report: * 5b091c27f98965150dbe479e39c948806ca02786 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=125

[GitHub] [hudi] hudi-bot commented on pull request #7045: HUDI-5083 Fixed a bug when schema evolution

2022-10-24 Thread GitBox
hudi-bot commented on PR #7045: URL: https://github.com/apache/hudi/pull/7045#issuecomment-1288647382 ## CI report: * c326073206cf20ca528c674678608d8713db7f1e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1251

[GitHub] [hudi] hudi-bot commented on pull request #6227: [HUDI-4496] Fixing Orc support broken for Spark 3.x and more

2022-10-24 Thread GitBox
hudi-bot commented on PR #6227: URL: https://github.com/apache/hudi/pull/6227#issuecomment-1288659835 ## CI report: * b236a680cc0abd5b1583c09570002bad77d9ad6f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1243

[GitHub] [hudi] hudi-bot commented on pull request #6707: [HUDI-4871] Upgrade spark3.3 to Spark 3.3.1

2022-10-24 Thread GitBox
hudi-bot commented on PR #6707: URL: https://github.com/apache/hudi/pull/6707#issuecomment-1288660626 ## CI report: * d0381797611ba0c237f21eaccd33815231833897 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1252

[GitHub] [hudi] hudi-bot commented on pull request #6991: [HUDI-5049] HoodieCatalog supports the implementation of dropPartition

2022-10-24 Thread GitBox
hudi-bot commented on PR #6991: URL: https://github.com/apache/hudi/pull/6991#issuecomment-1288661437 ## CI report: * fbb4c300133ad1c677826b836652152032b4c616 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1239

[GitHub] [hudi] danny0405 commented on pull request #6991: [HUDI-5049] HoodieCatalog supports the implementation of dropPartition

2022-10-24 Thread GitBox
danny0405 commented on PR #6991: URL: https://github.com/apache/hudi/pull/6991#issuecomment-1288690781 [5049.patch.zip](https://github.com/apache/hudi/files/9850345/5049.patch.zip) Thanks for the contribution, i have reviewed and applied a patch. -- This is an automated message from the

[jira] [Created] (HUDI-5085) When a job has multiple sink tables, the index loading status is abnormal.

2022-10-24 Thread yangxiao (Jira)
yangxiao created HUDI-5085: -- Summary: When a job has multiple sink tables, the index loading status is abnormal. Key: HUDI-5085 URL: https://issues.apache.org/jira/browse/HUDI-5085 Project: Apache Hudi

[jira] [Updated] (HUDI-5085) When a flink job has multiple sink tables, the index loading status is abnormal.

2022-10-24 Thread yangxiao (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yangxiao updated HUDI-5085: --- Summary: When a flink job has multiple sink tables, the index loading status is abnormal. (was: When a job ha

[GitHub] [hudi] ChenShuai1981 opened a new issue, #7047: [SUPPORT] HoodieFlinkCompactor with NoSuchMethodError: org.apache.hudi.org.apache.avro.specific.SpecificRecordBuilderBase

2022-10-24 Thread GitBox
ChenShuai1981 opened a new issue, #7047: URL: https://github.com/apache/hudi/issues/7047 **_Tips before filing an issue_** - Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)? - Join the mailing list to engage in conversations and get faster support at dev-su

[GitHub] [hudi] xiarixiaoyao commented on pull request #7045: [HUDI-5083]Fixed a bug when schema evolution

2022-10-24 Thread GitBox
xiarixiaoyao commented on PR #7045: URL: https://github.com/apache/hudi/pull/7045#issuecomment-1288722652 @shenshengli Thank you for your contribution. pls add some UT maybe you can add fllow codes , after line 293 in TestSpark3DDL ``` // drop + rename + insert

[GitHub] [hudi] SteNicholas commented on pull request #6991: [HUDI-5049] HoodieCatalog supports the implementation of dropPartition

2022-10-24 Thread GitBox
SteNicholas commented on PR #6991: URL: https://github.com/apache/hudi/pull/6991#issuecomment-1288724249 @danny0405, I have applied above patch and thanks for giving the patch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [hudi] Zhangshunyu commented on issue #7032: [SUPPORT] When metatable enabled, some query using index column as filter will get empty result

2022-10-24 Thread GitBox
Zhangshunyu commented on issue #7032: URL: https://github.com/apache/hudi/issues/7032#issuecomment-1288748469 > We are observing the same behavior with Hudi 0.11.1 and Spark 3.3.0. In our case we are filtering by a string column containing a timestamp like "202001110858". We obtain differen

[GitHub] [hudi] hudi-bot commented on pull request #5416: [HUDI-3963] Use Lock-Free Message Queue Disruptor Improving Hoodie Writing Efficiency

2022-10-24 Thread GitBox
hudi-bot commented on PR #5416: URL: https://github.com/apache/hudi/pull/5416#issuecomment-1288748792 ## CI report: * b838e1f406902c9bdfb5e84d53ef5a5effd0765b UNKNOWN * 6114ee2aa59f087e5ef0b1b53979eec143b33f5e UNKNOWN * 92760dbf5a047fe1f9941fa4b36c944eb3bec5c7 UNKNOWN * 0f

[GitHub] [hudi] SteNicholas commented on pull request #6991: [HUDI-5049] HoodieCatalog supports the implementation of dropPartition

2022-10-24 Thread GitBox
SteNicholas commented on PR #6991: URL: https://github.com/apache/hudi/pull/6991#issuecomment-1288750198 @danny0405, I have applied the above patch and removed the unused import in the patch. PTAL. -- This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [hudi] hudi-bot commented on pull request #6632: [HUDI-4753] more accurate record size estimation for log writing and spillable map

2022-10-24 Thread GitBox
hudi-bot commented on PR #6632: URL: https://github.com/apache/hudi/pull/6632#issuecomment-1288750609 ## CI report: * d9e12ddf962b670b8ec1e2260d5389c688e16001 UNKNOWN * ba3513d5b65e39f7cbb71e851ddd34cfe9d846a0 UNKNOWN * 8b7f94e6743c5f2decfeddaf164585a2a471a6c6 Azure: [FAILUR

[GitHub] [hudi] honeyaya commented on pull request #7007: [HUDI-4809] Glue support drop partitions

2022-10-24 Thread GitBox
honeyaya commented on PR #7007: URL: https://github.com/apache/hudi/pull/7007#issuecomment-1288755789 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [hudi] hudi-bot commented on pull request #6991: [HUDI-5049] HoodieCatalog supports the implementation of dropPartition

2022-10-24 Thread GitBox
hudi-bot commented on PR #6991: URL: https://github.com/apache/hudi/pull/6991#issuecomment-1288758812 ## CI report: * fbb4c300133ad1c677826b836652152032b4c616 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1239

[GitHub] [hudi] hudi-bot commented on pull request #7007: [HUDI-4809] Glue support drop partitions

2022-10-24 Thread GitBox
hudi-bot commented on PR #7007: URL: https://github.com/apache/hudi/pull/7007#issuecomment-1288759012 ## CI report: * a13855ef1e551ffe30aca9ce99a32243962bb8a2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1236

[GitHub] [hudi] hudi-bot commented on pull request #6991: [HUDI-5049] HoodieCatalog supports the implementation of dropPartition

2022-10-24 Thread GitBox
hudi-bot commented on PR #6991: URL: https://github.com/apache/hudi/pull/6991#issuecomment-1288765795 ## CI report: * 4d799a4b93f7489169b3ab0e7f37f0013e4693f1 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=125

[jira] [Assigned] (HUDI-5086) The doc of org.apache.hudi.sink.meta.CkpMetadata#bootstrap is not correct

2022-10-24 Thread JinxinTang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang reassigned HUDI-5086: Assignee: JinxinTang > The doc of org.apache.hudi.sink.meta.CkpMetadata#bootstrap is not correct >

[jira] [Created] (HUDI-5086) The doc of org.apache.hudi.sink.meta.CkpMetadata#bootstrap is not correct

2022-10-24 Thread JinxinTang (Jira)
JinxinTang created HUDI-5086: Summary: The doc of org.apache.hudi.sink.meta.CkpMetadata#bootstrap is not correct Key: HUDI-5086 URL: https://issues.apache.org/jira/browse/HUDI-5086 Project: Apache Hudi

[GitHub] [hudi] zhangyue19921010 commented on pull request #5416: [HUDI-3963] Use Lock-Free Message Queue Disruptor Improving Hoodie Writing Efficiency

2022-10-24 Thread GitBox
zhangyue19921010 commented on PR #5416: URL: https://github.com/apache/hudi/pull/5416#issuecomment-1288858105 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[GitHub] [hudi] TJX2014 opened a new pull request, #7048: [HUDI-5086] Fix doc of org.apache.hudi.sink.meta.CkpMetadata#bootstrap

2022-10-24 Thread GitBox
TJX2014 opened a new pull request, #7048: URL: https://github.com/apache/hudi/pull/7048 ### Change Logs Fix doc of org.apache.hudi.sink.meta.CkpMetadata#bootstrap ### Impact none ### Risk level (write none, low medium or high below) none ### Documentation Update

[jira] [Updated] (HUDI-5086) The doc of org.apache.hudi.sink.meta.CkpMetadata#bootstrap is not correct

2022-10-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5086: - Labels: pull-request-available (was: ) > The doc of org.apache.hudi.sink.meta.CkpMetadata#bootstr

[GitHub] [hudi] hudi-bot commented on pull request #7042: [HUDI-5082] Improve the cdc log file name format

2022-10-24 Thread GitBox
hudi-bot commented on PR #7042: URL: https://github.com/apache/hudi/pull/7042#issuecomment-1288867160 ## CI report: * 6ad211f90e9d94467ca6888e11bc28903b79ad15 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1250

[GitHub] [hudi] hudi-bot commented on pull request #7003: [minor] add more test for rfc46

2022-10-24 Thread GitBox
hudi-bot commented on PR #7003: URL: https://github.com/apache/hudi/pull/7003#issuecomment-1288866802 ## CI report: * 01c496501a59412c66df656a6d8801f1d2c45d6b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1250

[jira] [Updated] (HUDI-5082) Improve the cdc log file name format

2022-10-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5082: - Labels: pull-request-available (was: ) > Improve the cdc log file name format > -

[GitHub] [hudi] hudi-bot commented on pull request #5416: [HUDI-3963] Use Lock-Free Message Queue Disruptor Improving Hoodie Writing Efficiency

2022-10-24 Thread GitBox
hudi-bot commented on PR #5416: URL: https://github.com/apache/hudi/pull/5416#issuecomment-1288872377 ## CI report: * b838e1f406902c9bdfb5e84d53ef5a5effd0765b UNKNOWN * 6114ee2aa59f087e5ef0b1b53979eec143b33f5e UNKNOWN * 92760dbf5a047fe1f9941fa4b36c944eb3bec5c7 UNKNOWN * 0f

[GitHub] [hudi] hudi-bot commented on pull request #7000: [HUDI-5060] Make all clean policies support incremental mode to find partition paths

2022-10-24 Thread GitBox
hudi-bot commented on PR #7000: URL: https://github.com/apache/hudi/pull/7000#issuecomment-1288874983 ## CI report: * f0c09d506905d6e80f109b900e6e04bacffec4e6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1250

[GitHub] [hudi] hudi-bot commented on pull request #7048: [HUDI-5086] Fix doc of org.apache.hudi.sink.meta.CkpMetadata#bootstrap

2022-10-24 Thread GitBox
hudi-bot commented on PR #7048: URL: https://github.com/apache/hudi/pull/7048#issuecomment-1288875335 ## CI report: * 05c916be042f486e8a2dd089fe111cb3d95f594e UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7000: [HUDI-5060] Make all clean policies support incremental mode to find partition paths

2022-10-24 Thread GitBox
hudi-bot commented on PR #7000: URL: https://github.com/apache/hudi/pull/7000#issuecomment-122411 ## CI report: * f77806bcd4a38c2f4c1d44e970199d19bfc72737 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=125

[GitHub] [hudi] hudi-bot commented on pull request #7048: [HUDI-5086] Fix doc of org.apache.hudi.sink.meta.CkpMetadata#bootstrap

2022-10-24 Thread GitBox
hudi-bot commented on PR #7048: URL: https://github.com/apache/hudi/pull/7048#issuecomment-122858 ## CI report: * 05c916be042f486e8a2dd089fe111cb3d95f594e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1252

[GitHub] [hudi] Zhangshunyu commented on issue #7032: [SUPPORT] When metatable enabled, some query using index column as filter will get empty result

2022-10-24 Thread GitBox
Zhangshunyu commented on issue #7032: URL: https://github.com/apache/hudi/issues/7032#issuecomment-1288901483 we checked the parquet file minmax, its ok but the minmax in transposedColStatsDF of hoodiefile index is wrong: it use min as max... -- This is an automated message from th

[GitHub] [hudi] danielfordfc opened a new issue, #7049: [SUPPORT] SQLQueryBasedTransformer Not writing transformed parquet data

2022-10-24 Thread GitBox
danielfordfc opened a new issue, #7049: URL: https://github.com/apache/hudi/issues/7049 We are using Hudi 0.11.0 Hudi Deltastreamer on emr-6.7.0 to read data in from our Confluent Kafka cluster w/ Schema registry , and write it to a Glue Catalog table to be queried through Athena. Ou

[GitHub] [hudi] sknukala commented on issue #6907: [SUPPORT] hoodie commit time format change

2022-10-24 Thread GitBox
sknukala commented on issue #6907: URL: https://github.com/apache/hudi/issues/6907#issuecomment-1288944321 @nsivabalan : Adding a config to current hudi version 0.12 and future versions would help. Please let me know -- This is an automated message from the Apache Git Service. To respond

[GitHub] [hudi] hudi-bot commented on pull request #6989: [HUDI-5000] Support schema evolution for Hive/presto

2022-10-24 Thread GitBox
hudi-bot commented on PR #6989: URL: https://github.com/apache/hudi/pull/6989#issuecomment-1288952951 ## CI report: * e47fda7bf01414a4258b0dc54818a2e30b62ba3c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1251

[GitHub] [hudi] hudi-bot commented on pull request #6448: [HUDI-4647] Keep the hive sync settings in spark sql consistent

2022-10-24 Thread GitBox
hudi-bot commented on PR #6448: URL: https://github.com/apache/hudi/pull/6448#issuecomment-1288959595 ## CI report: * 4644aaa389dc545ef33180c70ee439c7d9f7f708 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1251

[GitHub] [hudi] wzx140 opened a new pull request, #7050: [Minor] update rfc46 doc

2022-10-24 Thread GitBox
wzx140 opened a new pull request, #7050: URL: https://github.com/apache/hudi/pull/7050 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any performance

[GitHub] [hudi] hudi-bot commented on pull request #7050: [MINOR] update rfc46 doc

2022-10-24 Thread GitBox
hudi-bot commented on PR #7050: URL: https://github.com/apache/hudi/pull/7050#issuecomment-1288968103 ## CI report: * 788df1c2f85af37930fb568cdce494debaf85ab9 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] Zhangshunyu commented on issue #7032: [SUPPORT] When metatable enabled, some query using index column as filter will get empty result

2022-10-24 Thread GitBox
Zhangshunyu commented on issue #7032: URL: https://github.com/apache/hudi/issues/7032#issuecomment-1288973266 @alexeykudinkin -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [hudi] hudi-bot commented on pull request #7050: [MINOR] update rfc46 doc

2022-10-24 Thread GitBox
hudi-bot commented on PR #7050: URL: https://github.com/apache/hudi/pull/7050#issuecomment-1288975488 ## CI report: * 788df1c2f85af37930fb568cdce494debaf85ab9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1252

[GitHub] [hudi] Zhangshunyu commented on issue #7032: [SUPPORT] [The max values are incorrect in hudi metatable dataframe] When metatable enabled, some query using index column as filter will get empt

2022-10-24 Thread GitBox
Zhangshunyu commented on issue #7032: URL: https://github.com/apache/hudi/issues/7032#issuecomment-1288977072 ![image](https://user-images.githubusercontent.com/13940237/197527861-8369f1e4-209e-4680-a998-24e560cb1003.png) we print it like this -- This is an automated message from the A

[GitHub] [hudi] mikeashley87 commented on issue #6819: [SUPPORT] NotSerializeableException glue2.0 - spark 2.4.3, hudi-spark-bundle_2.11-0.11.0.jar

2022-10-24 Thread GitBox
mikeashley87 commented on issue #6819: URL: https://github.com/apache/hudi/issues/6819#issuecomment-1289019820 @nsivabalan I tried using this version - https://mvnrepository.com/artifact/org.apache.hudi/hudi-spark2.4-bundle_2.11/0.12.0 it did not fix the issue. not clear what the l

[GitHub] [hudi] mikeashley87 commented on issue #6819: [SUPPORT] NotSerializeableException glue2.0 - spark 2.4.3, hudi-spark-bundle_2.11-0.11.0.jar

2022-10-24 Thread GitBox
mikeashley87 commented on issue #6819: URL: https://github.com/apache/hudi/issues/6819#issuecomment-1289041623 @nsivabalan the version referenced here says hudi 12.2 - I am not seeing it in maven. is it going to be published there or am I looking in the wrong spot ? https://mvnr

[GitHub] [hudi] YannByron commented on issue #7025: [SUPPORT] an ex occurs when updating a mor table in hudi

2022-10-24 Thread GitBox
YannByron commented on issue #7025: URL: https://github.com/apache/hudi/issues/7025#issuecomment-1289048001 @xiaoshao For `update` operation, the `preCombineField` is necessary. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] hudi-bot commented on pull request #6976: [HUDI-5042]fix clustering schedule problem in flink

2022-10-24 Thread GitBox
hudi-bot commented on PR #6976: URL: https://github.com/apache/hudi/pull/6976#issuecomment-1289052009 ## CI report: * c58a445df023b6e1a107dd484a7b7525f4d78b53 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1251

[GitHub] [hudi] hudi-bot commented on pull request #7050: [MINOR] update rfc46 doc

2022-10-24 Thread GitBox
hudi-bot commented on PR #7050: URL: https://github.com/apache/hudi/pull/7050#issuecomment-1289062369 ## CI report: * 788df1c2f85af37930fb568cdce494debaf85ab9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1252

[GitHub] [hudi] hudi-bot commented on pull request #7050: [MINOR] update rfc46 doc

2022-10-24 Thread GitBox
hudi-bot commented on PR #7050: URL: https://github.com/apache/hudi/pull/7050#issuecomment-1289070726 ## CI report: * 788df1c2f85af37930fb568cdce494debaf85ab9 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=125

[GitHub] [hudi] hkszn commented on issue #6718: [SUPPORT] Deltastreamer concurrent writes in continuous mode

2022-10-24 Thread GitBox
hkszn commented on issue #6718: URL: https://github.com/apache/hudi/issues/6718#issuecomment-1289149383 What is apache id? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [hudi] hudi-bot commented on pull request #6999: [HUDI-5057] Fix msck repair hudi table

2022-10-24 Thread GitBox
hudi-bot commented on PR #6999: URL: https://github.com/apache/hudi/pull/6999#issuecomment-1289166481 ## CI report: * 2e2e3c8bad4a90c1e84642ac19fc2aa2dc20c735 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1251

[GitHub] [hudi] YangXiao-allen opened a new pull request, #7051: When a flink job has multiple sink tables, the index loading status i…

2022-10-24 Thread GitBox
YangXiao-allen opened a new pull request, #7051: URL: https://github.com/apache/hudi/pull/7051 …s abnormal ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature

[GitHub] [hudi] hudi-bot commented on pull request #6989: [HUDI-5000] Support schema evolution for Hive/presto

2022-10-24 Thread GitBox
hudi-bot commented on PR #6989: URL: https://github.com/apache/hudi/pull/6989#issuecomment-1289176322 ## CI report: * e47fda7bf01414a4258b0dc54818a2e30b62ba3c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1251

[GitHub] [hudi] hudi-bot commented on pull request #6999: [HUDI-5057] Fix msck repair hudi table

2022-10-24 Thread GitBox
hudi-bot commented on PR #6999: URL: https://github.com/apache/hudi/pull/6999#issuecomment-1289176501 ## CI report: * 2e2e3c8bad4a90c1e84642ac19fc2aa2dc20c735 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1251

[GitHub] [hudi] hudi-bot commented on pull request #6989: [HUDI-5000] Support schema evolution for Hive/presto

2022-10-24 Thread GitBox
hudi-bot commented on PR #6989: URL: https://github.com/apache/hudi/pull/6989#issuecomment-1289185577 ## CI report: * e47fda7bf01414a4258b0dc54818a2e30b62ba3c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1251

[GitHub] [hudi] hudi-bot commented on pull request #6999: [HUDI-5057] Fix msck repair hudi table

2022-10-24 Thread GitBox
hudi-bot commented on PR #6999: URL: https://github.com/apache/hudi/pull/6999#issuecomment-1289185689 ## CI report: * 2e2e3c8bad4a90c1e84642ac19fc2aa2dc20c735 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1251

[GitHub] [hudi] hudi-bot commented on pull request #7051: When a flink job has multiple sink tables, the index loading status i…

2022-10-24 Thread GitBox
hudi-bot commented on PR #7051: URL: https://github.com/apache/hudi/pull/7051#issuecomment-1289186249 ## CI report: * 04aab81125034fd4329701b52708f846de53aa73 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7051: When a flink job has multiple sink tables, the index loading status i…

2022-10-24 Thread GitBox
hudi-bot commented on PR #7051: URL: https://github.com/apache/hudi/pull/7051#issuecomment-1289195245 ## CI report: * 04aab81125034fd4329701b52708f846de53aa73 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1253

  1   2   3   4   >