[GitHub] [hudi] aib628 commented on issue #8848: [SUPPORT] Hive Sync tool fails to sync Hoodi table written using Flink 1.16 to HMS

2023-07-25 Thread via GitHub
aib628 commented on issue #8848: URL: https://github.com/apache/hudi/issues/8848#issuecomment-1651074254 > Hi @danny0405 i had try it, and the seem problem reproduced. ``` CREATE TABLE t1( uuid VARCHAR(20) PRIMARY KEY NOT ENFORCED, name VARCHAR(10), age INT,

[GitHub] [hudi] Zouxxyy closed pull request #7615: [HUDI-5510] Reload active timeline when commit finish

2023-07-25 Thread via GitHub
Zouxxyy closed pull request #7615: [HUDI-5510] Reload active timeline when commit finish URL: https://github.com/apache/hudi/pull/7615 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [hudi] SteNicholas commented on a diff in pull request #9211: [HUDI-6540] Support failed writes clean policy for Flink

2023-07-25 Thread via GitHub
SteNicholas commented on code in PR #9211: URL: https://github.com/apache/hudi/pull/9211#discussion_r1274428725 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/TestWriteCopyOnWrite.java: ## @@ -114,10 +116,28 @@ public void testCheckpointFails() throws Exc

[jira] [Updated] (HUDI-6592) Flink insert overwrite should support dynamic partition instead of whole table

2023-07-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6592: - Labels: pull-request-available (was: ) > Flink insert overwrite should support dynamic partition

[GitHub] [hudi] SteNicholas opened a new pull request, #9287: [HUDI-6592] Flink insert overwrite should support dynamic partition instead of whole table

2023-07-25 Thread via GitHub
SteNicholas opened a new pull request, #9287: URL: https://github.com/apache/hudi/pull/9287 ### Change Logs Flink insert overwrite should support dynamic partition instead of the whole table, which behavior is consistent with the semantics of insert overwrite in Flink. ### Imp

[jira] [Created] (HUDI-6592) Flink insert overwrite should support dynamic partition instead of whole table

2023-07-25 Thread Nicholas Jiang (Jira)
Nicholas Jiang created HUDI-6592: Summary: Flink insert overwrite should support dynamic partition instead of whole table Key: HUDI-6592 URL: https://issues.apache.org/jira/browse/HUDI-6592 Project: A

[GitHub] [hudi] codope opened a new pull request, #9286: [MINOR] Add rowId field to HoodieRecordIndexInfo in metadata payload

2023-07-25 Thread via GitHub
codope opened a new pull request, #9286: URL: https://github.com/apache/hudi/pull/9286 ### Change Logs Add rowId field to HoodieRecordIndexInfo in metadata payload. The rowId is not being read/written right now. Default is 0L. In future, we would want to map a record to file and rowI

[GitHub] [hudi] hudi-bot commented on pull request #9255: [HUDI-6503] Make TableServiceClient's txnManager consistent with Writ…

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9255: URL: https://github.com/apache/hudi/pull/9255#issuecomment-1650991561 ## CI report: * 463953fa8ffd4e41dcc02c67cf931d894a12848d Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1883

[GitHub] [hudi] chandu-1101 commented on issue #9141: [BUG] Example from Hudi Quick start doesnt work!

2023-07-25 Thread via GitHub
chandu-1101 commented on issue #9141: URL: https://github.com/apache/hudi/issues/9141#issuecomment-1650981047 Wow! Wonderful. Thank you once again. I will put the flag check and get back. -- This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [hudi] big-doudou commented on pull request #9182: [HUDI-6588] Fix duplicate fileId on TM partial-failover and recovery

2023-07-25 Thread via GitHub
big-doudou commented on PR #9182: URL: https://github.com/apache/hudi/pull/9182#issuecomment-1650926469 > Each failed attempt of a subtask would trigger invocation of `StreamWriteOperatorCoordinator#subtaskFailed`, the original write metadata would got cleaned, The StreamWriteOperato

[GitHub] [hudi] JingFengWang opened a new issue, #9285: When compiling hudi-0.13.0, the package org.apache.http does not exist error is thrown

2023-07-25 Thread via GitHub
JingFengWang opened a new issue, #9285: URL: https://github.com/apache/hudi/issues/9285 **Steps to reproduce the behavior:** **run command:** mvn clean package -DskipTests -Dflink1.13 -Dscala-2.11 **exception log:** [ERROR] COMPILATION ERROR : [INFO] ---

[GitHub] [hudi] ad1happy2go commented on issue #9143: [SUPPORT] Failure to delete records with missing attributes from PostgresDebeziumSource

2023-07-25 Thread via GitHub
ad1happy2go commented on issue #9143: URL: https://github.com/apache/hudi/issues/9143#issuecomment-1650912511 @Sam-Serpoosh In this case we need to maintain global uniqueness , so Global Index should be the right option. On a large dataset it might have downside, but as partition value is n

[GitHub] [hudi] Sam-Serpoosh commented on issue #9143: [SUPPORT] Failure to delete records with missing attributes from PostgresDebeziumSource

2023-07-25 Thread via GitHub
Sam-Serpoosh commented on issue #9143: URL: https://github.com/apache/hudi/issues/9143#issuecomment-1650903089 @ad1happy2go IIUC, here are the options: - Leverage `REPLICA IDENTITY FULL` which has some downsides as mentioned in PG documentation and the article I shared in my earlier c

[GitHub] [hudi] danny0405 commented on pull request #9255: [HUDI-6503] Make TableServiceClient's txnManager consistent with Writ…

2023-07-25 Thread via GitHub
danny0405 commented on PR #9255: URL: https://github.com/apache/hudi/pull/9255#issuecomment-1650874179 Okay, a static lock mapping makes sense to me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hudi] Zouxxyy commented on pull request #9255: [HUDI-6503] Make TableServiceClient's txnManager consistent with Writ…

2023-07-25 Thread via GitHub
Zouxxyy commented on PR #9255: URL: https://github.com/apache/hudi/pull/9255#issuecomment-1650870862 > Can you elaborate a little more why the table service client can hold a separate lock ? Because `InProcessLockProvider` is valid as long as it is in the same JVM process (see `stati

[GitHub] [hudi] hudi-bot commented on pull request #9255: [HUDI-6503] Make TableServiceClient's txnManager consistent with Writ…

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9255: URL: https://github.com/apache/hudi/pull/9255#issuecomment-1650869802 ## CI report: * 4e64258913f8f19b139ab1407f0c08d812f65669 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1883

[GitHub] [hudi] hudi-bot commented on pull request #9255: [HUDI-6503] Make TableServiceClient's txnManager consistent with Writ…

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9255: URL: https://github.com/apache/hudi/pull/9255#issuecomment-1650864450 ## CI report: * 4e64258913f8f19b139ab1407f0c08d812f65669 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1883

[GitHub] [hudi] danny0405 commented on a diff in pull request #9229: [HUDI-6565] Spark offline compaction add failed retry mechanism

2023-07-25 Thread via GitHub
danny0405 commented on code in PR #9229: URL: https://github.com/apache/hudi/pull/9229#discussion_r1274291759 ## hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieCompactor.java: ## @@ -101,6 +104,12 @@ public static class Config implements Serializable { public

[GitHub] [hudi] danny0405 commented on pull request #9182: [HUDI-6588] Fix duplicate fileId on TM partial-failover and recovery

2023-07-25 Thread via GitHub
danny0405 commented on PR #9182: URL: https://github.com/apache/hudi/pull/9182#issuecomment-1650862935 Each failed attempt of a subtask would trigger invocation of `StreamWriteOperatorCoordinator#subtaskFailed`, the original write metadata would got cleaned, -- This is an automated messa

[GitHub] [hudi] danny0405 merged pull request #9274: [MINOR] fix millis append format error

2023-07-25 Thread via GitHub
danny0405 merged PR #9274: URL: https://github.com/apache/hudi/pull/9274 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache

[hudi] branch master updated: [MINOR] Fix millis append format error (#9274)

2023-07-25 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 2a022393388 [MINOR] Fix millis append format er

[GitHub] [hudi] codope closed issue #8761: [SUPPORT] "Illegal Lambda Deserialization" When Leveraging PostgresDebeziumSource

2023-07-25 Thread via GitHub
codope closed issue #8761: [SUPPORT] "Illegal Lambda Deserialization" When Leveraging PostgresDebeziumSource URL: https://github.com/apache/hudi/issues/8761 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hudi] danny0405 commented on pull request #9255: [HUDI-6503] Make TableServiceClient's txnManager consistent with Writ…

2023-07-25 Thread via GitHub
danny0405 commented on PR #9255: URL: https://github.com/apache/hudi/pull/9255#issuecomment-1650853733 > I think about it again, and think that it’s okay to not pass the txtmanger Can you elaborate a little more why the table service client can hold a separate lock ? -- This is an

[GitHub] [hudi] danny0405 commented on a diff in pull request #9211: [HUDI-6540] Support failed writes clean policy for Flink

2023-07-25 Thread via GitHub
danny0405 commented on code in PR #9211: URL: https://github.com/apache/hudi/pull/9211#discussion_r1274284407 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/TestWriteCopyOnWrite.java: ## @@ -114,10 +116,28 @@ public void testCheckpointFails() throws Excep

[GitHub] [hudi] hudi-bot commented on pull request #9277: [HUDI-6558] support SQL update for no-precombine field tables

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9277: URL: https://github.com/apache/hudi/pull/9277#issuecomment-1650807747 ## CI report: * cf692aeb6c7774b01a236cf058225debb8caff53 UNKNOWN * da9dd1fc203c01d0a000d49dcbd58a0a1d729354 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #9277: [HUDI-6558] support SQL update for no-precombine field tables

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9277: URL: https://github.com/apache/hudi/pull/9277#issuecomment-1650614871 ## CI report: * cf692aeb6c7774b01a236cf058225debb8caff53 UNKNOWN * da9dd1fc203c01d0a000d49dcbd58a0a1d729354 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] kazdy commented on pull request #9277: [HUDI-6558] support SQL update for no-precombine field tables

2023-07-25 Thread via GitHub
kazdy commented on PR #9277: URL: https://github.com/apache/hudi/pull/9277#issuecomment-1650583696 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [hudi] hudi-bot commented on pull request #9276: Mor perf spark33

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9276: URL: https://github.com/apache/hudi/pull/9276#issuecomment-1650565195 ## CI report: * 54a4e7e9aeabb42258e0d1f2b6cfa2960275c330 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1883

[GitHub] [hudi] hudi-bot commented on pull request #9276: Mor perf spark33

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9276: URL: https://github.com/apache/hudi/pull/9276#issuecomment-1650555772 ## CI report: * 37d3b9365a38e8f266c1c486e9d18c9ef34be2a0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1880

[GitHub] [hudi] bhasudha opened a new pull request, #9284: [DOCS] Change algolia search to leverage crawler instead of legacy do…

2023-07-25 Thread via GitHub
bhasudha opened a new pull request, #9284: URL: https://github.com/apache/hudi/pull/9284 …csearch ### Change Logs Migrating to new crawler based search ### Impact can affect website search functionality ### Risk level (write none, low medium or high below)

[GitHub] [hudi] kkalanda-score closed pull request #9283: CI (Take 2)

2023-07-25 Thread via GitHub
kkalanda-score closed pull request #9283: CI (Take 2) URL: https://github.com/apache/hudi/pull/9283 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: comm

[GitHub] [hudi] kkalanda-score opened a new pull request, #9283: CI (Take 2)

2023-07-25 Thread via GitHub
kkalanda-score opened a new pull request, #9283: URL: https://github.com/apache/hudi/pull/9283 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any perfo

[hudi] branch master updated (42799c0956f -> 03bc5549c7a)

2023-07-25 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 42799c0956f [HUDI-6438] Config parameter 'MAKE_NEW_COLUMNS_NULLABLE' to allow for marking a newly created column as null

[GitHub] [hudi] yihua merged pull request #9136: [HUDI-6509] Add GitHub CI for Java 17

2023-07-25 Thread via GitHub
yihua merged PR #9136: URL: https://github.com/apache/hudi/pull/9136 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

[GitHub] [hudi] yihua commented on pull request #9136: [HUDI-6509] Add GitHub CI for Java 17

2023-07-25 Thread via GitHub
yihua commented on PR #9136: URL: https://github.com/apache/hudi/pull/9136#issuecomment-1650381811 CI has passed for [6b33d37](https://github.com/apache/hudi/pull/9136/commits/6b33d37bc57d2b5be3649590fee6767f34cccea3). No need to rerun CI again. https://github.com/apache/hudi/assets/249

[jira] [Created] (HUDI-6591) getAllPartitionPaths perf fix need to account for parquet/orc partition path meta files

2023-07-25 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-6591: - Summary: getAllPartitionPaths perf fix need to account for parquet/orc partition path meta files Key: HUDI-6591 URL: https://issues.apache.org/jira/browse/HUDI-6591

[GitHub] [hudi] yihua commented on a diff in pull request #9136: [HUDI-6509] Add GitHub CI for Java 17

2023-07-25 Thread via GitHub
yihua commented on code in PR #9136: URL: https://github.com/apache/hudi/pull/9136#discussion_r1273945242 ## .github/workflows/bot.yml: ## @@ -112,6 +112,61 @@ jobs: run: mvn test -Pfunctional-tests -D"$SCALA_PROFILE" -D"$SPARK_PROFILE" -pl "$SPARK_COMMON_MO

[GitHub] [hudi] hudi-bot commented on pull request #9255: [HUDI-6503] Make TableServiceClient's txnManager consistent with Writ…

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9255: URL: https://github.com/apache/hudi/pull/9255#issuecomment-1650373193 ## CI report: * 4e64258913f8f19b139ab1407f0c08d812f65669 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1883

[GitHub] [hudi] amrishlal commented on issue #7244: [SUPPORT] DBT Merge creates duplicates

2023-07-25 Thread via GitHub
amrishlal commented on issue #7244: URL: https://github.com/apache/hudi/issues/7244#issuecomment-1650363772 Verified using the latest master using the same model as @ad1happy2go above and successfully ran the model **DBT run** ```amrish@Amrishs-MBP github-issue-7244 % dbt run 18

[GitHub] [hudi] hudi-bot commented on pull request #9277: [HUDI-6558] support SQL update for no-precombine field tables

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9277: URL: https://github.com/apache/hudi/pull/9277#issuecomment-1650303969 ## CI report: * cf692aeb6c7774b01a236cf058225debb8caff53 UNKNOWN * da9dd1fc203c01d0a000d49dcbd58a0a1d729354 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] chattarajoy commented on pull request #8795: [HUDI-6258] support olap engine query mor table in table name without ro/rt suffix

2023-07-25 Thread via GitHub
chattarajoy commented on PR #8795: URL: https://github.com/apache/hudi/pull/8795#issuecomment-1650303459 Is there a place where I can find the timeline on when this can possibly be released? -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [hudi] xushiyan commented on a diff in pull request #9221: [HUDI-6550] Add Hadoop conf to HiveConf for HiveSyncConfig

2023-07-25 Thread via GitHub
xushiyan commented on code in PR #9221: URL: https://github.com/apache/hudi/pull/9221#discussion_r1273870474 ## hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncConfig.java: ## @@ -98,8 +98,9 @@ public HiveSyncConfig(Properties props) { public HiveSyncCon

[GitHub] [hudi] rmnlchh commented on issue #9282: [ISSUE] Hudi 0.13.0. Spark 3.3.2 Deltastreamed table read failure

2023-07-25 Thread via GitHub
rmnlchh commented on issue #9282: URL: https://github.com/apache/hudi/issues/9282#issuecomment-1650228180 > @rmnlchh Just curious, Did you set these configs > > ``` > sc.set("spark.sql.legacy.parquet.nanosAsLong", "false"); > sc.set("spark.sql.parquet.binaryAsString", "false");

[GitHub] [hudi] ad1happy2go commented on issue #9282: [ISSUE] Hudi 0.13.0. Spark 3.3.2 Deltastreamed table read failure

2023-07-25 Thread via GitHub
ad1happy2go commented on issue #9282: URL: https://github.com/apache/hudi/issues/9282#issuecomment-1650223209 @rmnlchh Just curious, Did you set these configs ``` sc.set("spark.sql.legacy.parquet.nanosAsLong", "false"); sc.set("spark.sql.parquet.binaryAsString", "false"); sc.set(

[jira] [Updated] (HUDI-6589) Upsert failing for array type if value given [null]

2023-07-25 Thread Aditya Goenka (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Goenka updated HUDI-6589: Fix Version/s: (was: 0.12.1) > Upsert failing for array type if value given [null] > ---

[jira] [Closed] (HUDI-6589) Upsert failing for array type if value given [null]

2023-07-25 Thread Aditya Goenka (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Goenka closed HUDI-6589. --- Fix Version/s: 0.12.1 (was: 0.15.0) Resolution: Resolved > Upsert failin

[jira] [Commented] (HUDI-6589) Upsert failing for array type if value given [null]

2023-07-25 Thread Aditya Goenka (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17747076#comment-17747076 ] Aditya Goenka commented on HUDI-6589: - Upon further investigation and debugging, it ha

[jira] [Resolved] (HUDI-6589) Upsert failing for array type if value given [null]

2023-07-25 Thread Aditya Goenka (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Goenka resolved HUDI-6589. - > Upsert failing for array type if value given [null] > --

[GitHub] [hudi] bhasudha commented on issue #9282: [ISSUE] Hudi 0.13.0. Spark 3.3.2 Deltastreamed table read failure

2023-07-25 Thread via GitHub
bhasudha commented on issue #9282: URL: https://github.com/apache/hudi/issues/9282#issuecomment-1650203382 @yihua @ad1happy2go if you can help reproduce and trige this further. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] hudi-bot commented on pull request #9105: [HUDI-6459] Add Rollback and multi-writer tests for Record Level Index

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9105: URL: https://github.com/apache/hudi/pull/9105#issuecomment-1650200162 ## CI report: * 6bd80d5ce84b468293bc292f43dd0ca236c646d8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1883

[GitHub] [hudi] rmnlchh opened a new issue, #9282: [ISSUE] Hudi 0.13.0. Spark 3.3.2 Deltastreamed table read failure

2023-07-25 Thread via GitHub
rmnlchh opened a new issue, #9282: URL: https://github.com/apache/hudi/issues/9282 As part of our pipelines, we use tables that are being deltastreamed. Trying to upgrade to EMR 6.11 (which bring hudi 0.13.0/spark 3.3.2) we started facing issue which is discussed in https://github.com/a

[GitHub] [hudi] hudi-bot commented on pull request #9246: [HUDI-6548] Two log compaction instants can be scheduled at the same time

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9246: URL: https://github.com/apache/hudi/pull/9246#issuecomment-1650135501 ## CI report: * c2effa1ea1fdd82828efbf88afbf6cd6be019eb3 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1883

[GitHub] [hudi] hudi-bot commented on pull request #9255: [HUDI-6503] Make TableServiceClient's txnManager consistent with Writ…

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9255: URL: https://github.com/apache/hudi/pull/9255#issuecomment-1650135633 ## CI report: * 257b18bc9faffdf7d063fb153e5ee1b53d57 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1879

[GitHub] [hudi] hudi-bot commented on pull request #9277: [HUDI-6558] support SQL update for no-precombine field tables

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9277: URL: https://github.com/apache/hudi/pull/9277#issuecomment-1650121068 ## CI report: * d062a4c9cecf2f35a2f07a046a4139c7d0aea301 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1882

[GitHub] [hudi] hudi-bot commented on pull request #9255: [HUDI-6503] Make TableServiceClient's txnManager consistent with Writ…

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9255: URL: https://github.com/apache/hudi/pull/9255#issuecomment-1650120837 ## CI report: * 257b18bc9faffdf7d063fb153e5ee1b53d57 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1879

[GitHub] [hudi] hudi-bot commented on pull request #9277: [HUDI-6558] support SQL update for no-precombine field tables

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9277: URL: https://github.com/apache/hudi/pull/9277#issuecomment-1650105696 ## CI report: * d062a4c9cecf2f35a2f07a046a4139c7d0aea301 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1882

[jira] [Updated] (HUDI-6590) Improve BigQuery Sync Schema and Partition Handling

2023-07-25 Thread Timothy Brown (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Brown updated HUDI-6590: Summary: Improve BigQuery Sync Schema and Partition Handling (was: Improve BigQuery Sync Support)

[jira] [Assigned] (HUDI-6590) Improve BigQuery Sync Support

2023-07-25 Thread Timothy Brown (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Brown reassigned HUDI-6590: --- Assignee: Timothy Brown > Improve BigQuery Sync Support > - > >

[jira] [Created] (HUDI-6590) Improve BigQuery Sync Support

2023-07-25 Thread Timothy Brown (Jira)
Timothy Brown created HUDI-6590: --- Summary: Improve BigQuery Sync Support Key: HUDI-6590 URL: https://issues.apache.org/jira/browse/HUDI-6590 Project: Apache Hudi Issue Type: Improvement

[GitHub] [hudi] hudi-bot commented on pull request #9277: [HUDI-6558] support SQL update for no-precombine field tables

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9277: URL: https://github.com/apache/hudi/pull/9277#issuecomment-1650029611 ## CI report: * d062a4c9cecf2f35a2f07a046a4139c7d0aea301 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1882

[GitHub] [hudi] hudi-bot commented on pull request #9212: [HUDI-6541] Multiple writers should create new and different instant time to avoid marker conflict of same instant

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9212: URL: https://github.com/apache/hudi/pull/9212#issuecomment-1650029030 ## CI report: * 32766783236e3f0b5adcc973a77ff9cf782726e5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1882

[GitHub] [hudi] hudi-bot commented on pull request #9277: [HUDI-6558] support SQL update for no-precombine field tables

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9277: URL: https://github.com/apache/hudi/pull/9277#issuecomment-1649902580 ## CI report: * d062a4c9cecf2f35a2f07a046a4139c7d0aea301 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1882

[GitHub] [hudi] ad1happy2go commented on issue #7244: [SUPPORT] DBT Merge creates duplicates

2023-07-25 Thread via GitHub
ad1happy2go commented on issue #7244: URL: https://github.com/apache/hudi/issues/7244#issuecomment-1649877906 @faizhasan @rshanmugam1 Apologies for the delay here. I tried to reproduce and found out that it is working fine. I tried with 0.12.1 version. Model I used, exactly like we have i

[GitHub] [hudi] hudi-bot commented on pull request #9246: [HUDI-6548] Two log compaction instants can be scheduled at the same time

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9246: URL: https://github.com/apache/hudi/pull/9246#issuecomment-1649827320 ## CI report: * 136d780d0a9ca38f88c613433f05f868be01d0d5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1873

[GitHub] [hudi] hudi-bot commented on pull request #9105: [HUDI-6459] Add Rollback and multi-writer tests for Record Level Index

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9105: URL: https://github.com/apache/hudi/pull/9105#issuecomment-1649826685 ## CI report: * 5611851113d971d2f76fe2072ca87c1df0eae6ea Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1840

[GitHub] [hudi] hudi-bot commented on pull request #9246: [HUDI-6548] Two log compaction instants can be scheduled at the same time

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9246: URL: https://github.com/apache/hudi/pull/9246#issuecomment-1649811839 ## CI report: * 136d780d0a9ca38f88c613433f05f868be01d0d5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1873

[GitHub] [hudi] hudi-bot commented on pull request #9105: [HUDI-6459] Add Rollback and multi-writer tests for Record Level Index

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9105: URL: https://github.com/apache/hudi/pull/9105#issuecomment-1649811247 ## CI report: * 5611851113d971d2f76fe2072ca87c1df0eae6ea Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1840

[GitHub] [hudi] hudi-bot commented on pull request #9211: [HUDI-6540] Support failed writes clean policy for Flink

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9211: URL: https://github.com/apache/hudi/pull/9211#issuecomment-1649795214 ## CI report: * b6afe889ca6b47f4d1d934bb552cc1c489f9d0af UNKNOWN * f8607c6bd9ecf09e8da2d6b372a80eff2221108d Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] lokeshj1703 commented on a diff in pull request #9246: [HUDI-6548] Two log compaction instants can be scheduled at the same time

2023-07-25 Thread via GitHub
lokeshj1703 commented on code in PR #9246: URL: https://github.com/apache/hudi/pull/9246#discussion_r1273472332 ## hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieDefaultTimeline.java: ## @@ -277,7 +278,7 @@ public HoodieTimeline getCommitsTimeline() {

[GitHub] [hudi] zaza commented on issue #6900: [SUPPORT]Hudi Failed to read MARKERS file

2023-07-25 Thread via GitHub
zaza commented on issue #6900: URL: https://github.com/apache/hudi/issues/6900#issuecomment-1649736303 This is definitely still an issue, we were hit by an error that looks identical to what @umehrot2 reported a while ago: ``` ERROR UpsertPartitioner: Error trying to compute averag

[GitHub] [hudi] hudi-bot commented on pull request #9280: [HUDI-6587] Handle hollow commit for time travel query

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9280: URL: https://github.com/apache/hudi/pull/9280#issuecomment-1649724112 ## CI report: * db146be5542714a978e1d6fcdbd146e2aa834931 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1882

[GitHub] [hudi] hudi-bot commented on pull request #9212: [HUDI-6541] Multiple writers should create new and different instant time to avoid marker conflict of same instant

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9212: URL: https://github.com/apache/hudi/pull/9212#issuecomment-1649723729 ## CI report: * f494be8b2b8d4e9d5a6d595eea8bc907602efd35 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1882

[GitHub] [hudi] hudi-bot commented on pull request #9212: [HUDI-6541] Multiple writers should create new and different instant time to avoid marker conflict of same instant

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9212: URL: https://github.com/apache/hudi/pull/9212#issuecomment-1649706487 ## CI report: * f494be8b2b8d4e9d5a6d595eea8bc907602efd35 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1882

[jira] [Updated] (HUDI-6589) Upsert failing for array type if value given [null]

2023-07-25 Thread Aditya Goenka (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Goenka updated HUDI-6589: Priority: Critical (was: Major) > Upsert failing for array type if value given [null] > ---

[jira] [Created] (HUDI-6589) Upsert failing for array type if value given [null]

2023-07-25 Thread Aditya Goenka (Jira)
Aditya Goenka created HUDI-6589: --- Summary: Upsert failing for array type if value given [null] Key: HUDI-6589 URL: https://issues.apache.org/jira/browse/HUDI-6589 Project: Apache Hudi Issue Typ

[GitHub] [hudi] hudi-bot commented on pull request #9277: [HUDI-6558] support SQL update for no-precombine field tables

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9277: URL: https://github.com/apache/hudi/pull/9277#issuecomment-1649630856 ## CI report: * 4d363f192f951fb54799602270fb0ca16ce19d39 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1881

[GitHub] [hudi] hudi-bot commented on pull request #9277: [HUDI-6558] support SQL update for no-precombine field tables

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9277: URL: https://github.com/apache/hudi/pull/9277#issuecomment-1649620494 ## CI report: * 4d363f192f951fb54799602270fb0ca16ce19d39 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1881

[GitHub] [hudi] hudi-bot commented on pull request #9209: [HUDI-6539] New LSM tree style archived timeline

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9209: URL: https://github.com/apache/hudi/pull/9209#issuecomment-1649620034 ## CI report: * 8f2dc4ec3e26f1908ae5d15f194bf70ca7dab27e UNKNOWN * a7f8558aaffdaab4850780224e1385c3e682372a Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #9277: [HUDI-6558] support SQL update for no-precombine field tables

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9277: URL: https://github.com/apache/hudi/pull/9277#issuecomment-1649609922 ## CI report: * 4d363f192f951fb54799602270fb0ca16ce19d39 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1881

[GitHub] [hudi] hudi-bot commented on pull request #9212: [HUDI-6541] Multiple writers should create new and different instant time to avoid marker conflict of same instant

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9212: URL: https://github.com/apache/hudi/pull/9212#issuecomment-1649609463 ## CI report: * f494be8b2b8d4e9d5a6d595eea8bc907602efd35 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1882

[GitHub] [hudi] kazdy commented on pull request #9277: [HUDI-6558] support SQL update for no-precombine field tables

2023-07-25 Thread via GitHub
kazdy commented on PR #9277: URL: https://github.com/apache/hudi/pull/9277#issuecomment-1649530790 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [hudi] SteNicholas commented on pull request #9211: [HUDI-6540] Support failed writes clean policy for Flink

2023-07-25 Thread via GitHub
SteNicholas commented on PR #9211: URL: https://github.com/apache/hudi/pull/9211#issuecomment-1649522478 @danny0405, could you take a look at this pull request? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [hudi] hudi-bot commented on pull request #9211: [HUDI-6540] Support failed writes clean policy for Flink

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9211: URL: https://github.com/apache/hudi/pull/9211#issuecomment-1649517541 ## CI report: * b6afe889ca6b47f4d1d934bb552cc1c489f9d0af UNKNOWN * 278399029bc5dc3ab81d0366b65aaed3cf019b7c Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-

[GitHub] [hudi] hudi-bot commented on pull request #9211: [HUDI-6540] Support failed writes clean policy for Flink

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9211: URL: https://github.com/apache/hudi/pull/9211#issuecomment-1649500610 ## CI report: * b6afe889ca6b47f4d1d934bb552cc1c489f9d0af UNKNOWN * d8e39cb69480b8eb9014f09f6b84e741b9092a9f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] Zouxxyy opened a new pull request, #9281: [WIP] Add HooideTable in BaseHoodieClient

2023-07-25 Thread via GitHub
Zouxxyy opened a new pull request, #9281: URL: https://github.com/apache/hudi/pull/9281 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any performance

[GitHub] [hudi] stream2000 commented on pull request #9211: [HUDI-6540] Support failed writes clean policy for Flink

2023-07-25 Thread via GitHub
stream2000 commented on PR #9211: URL: https://github.com/apache/hudi/pull/9211#issuecomment-1649457125 > even if the failed writes clean policy could be inferred from optimistic concurrent control is enabled, this support has no conflict with the inference. Do you mean by even if we

[GitHub] [hudi] adityaverma1997 commented on issue #9257: [SUPPORT] Parquet files got cleaned up even when cleaning operation failed hence leading to subsequent failed clustering and cleaning

2023-07-25 Thread via GitHub
adityaverma1997 commented on issue #9257: URL: https://github.com/apache/hudi/issues/9257#issuecomment-1649440527 Correct me if I am wrong here, though I am running async cleaning but cleaning frequency is controlled by the following hudi configuration, which is: ``` hoodie.clean.max.c

[GitHub] [hudi] hudi-bot commented on pull request #9280: [HUDI-6587] Handle hollow commit for time travel query

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9280: URL: https://github.com/apache/hudi/pull/9280#issuecomment-1649431922 ## CI report: * db146be5542714a978e1d6fcdbd146e2aa834931 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1882

[GitHub] [hudi] hudi-bot commented on pull request #9274: [MINOR] fix millis append format error

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9274: URL: https://github.com/apache/hudi/pull/9274#issuecomment-1649431799 ## CI report: * 94d9dbcb05d1505d4a1d5e82dca8a8ba946f47da Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1880

[GitHub] [hudi] SteNicholas commented on a diff in pull request #9211: [HUDI-6540] Support failed writes clean policy for Flink

2023-07-25 Thread via GitHub
SteNicholas commented on code in PR #9211: URL: https://github.com/apache/hudi/pull/9211#discussion_r1273233193 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableSink.java: ## @@ -95,6 +95,9 @@ public SinkRuntimeProvider getSinkRuntimeProvider(Co

[GitHub] [hudi] hudi-bot commented on pull request #9280: [HUDI-6587] Handle hollow commit for time travel query

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9280: URL: https://github.com/apache/hudi/pull/9280#issuecomment-1649418587 ## CI report: * db146be5542714a978e1d6fcdbd146e2aa834931 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] xushiyan commented on a diff in pull request #9221: [HUDI-6550] Add Hadoop conf to HiveConf for HiveSyncConfig

2023-07-25 Thread via GitHub
xushiyan commented on code in PR #9221: URL: https://github.com/apache/hudi/pull/9221#discussion_r1273223195 ## hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncConfig.java: ## @@ -98,8 +98,9 @@ public HiveSyncConfig(Properties props) { public HiveSyncCon

[GitHub] [hudi] xushiyan commented on pull request #9221: [HUDI-6550] Add Hadoop conf to HiveConf for HiveSyncConfig

2023-07-25 Thread via GitHub
xushiyan commented on PR #9221: URL: https://github.com/apache/hudi/pull/9221#issuecomment-1649394492 > Hi @xushiyan, I noticed the casting from hadoopConf to hiveConf was introduced by this PR from you(#6202) but I couldn't find any context. Could you help me learn why we made that change?

[jira] [Updated] (HUDI-6587) Handle hollow commit for time travel query

2023-07-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6587: - Labels: pull-request-available (was: ) > Handle hollow commit for time travel query > ---

[GitHub] [hudi] xushiyan opened a new pull request, #9280: [HUDI-6587] Handle hollow commit for time travel query

2023-07-25 Thread via GitHub
xushiyan opened a new pull request, #9280: URL: https://github.com/apache/hudi/pull/9280 ### Change Logs Fail time-travel query when the given timestamp covers any hollow commit. ### Impact Time travel query behavior. Time travel usually won't cover hollow commit, which

[GitHub] [hudi] stream2000 commented on a diff in pull request #9211: [HUDI-6540] Support failed writes clean policy for Flink

2023-07-25 Thread via GitHub
stream2000 commented on code in PR #9211: URL: https://github.com/apache/hudi/pull/9211#discussion_r1273176064 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableSink.java: ## @@ -95,6 +95,9 @@ public SinkRuntimeProvider getSinkRuntimeProvider(Con

[GitHub] [hudi] hudi-bot commented on pull request #9209: [HUDI-6539] New LSM tree style archived timeline

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9209: URL: https://github.com/apache/hudi/pull/9209#issuecomment-1649344787 ## CI report: * 8f2dc4ec3e26f1908ae5d15f194bf70ca7dab27e UNKNOWN * c281ded6d554350dfe362cce496d6d72cfe0bbbe Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-

[GitHub] [hudi] ksmou commented on a diff in pull request #9229: [HUDI-6565] Spark offline compaction add failed retry mechanism

2023-07-25 Thread via GitHub
ksmou commented on code in PR #9229: URL: https://github.com/apache/hudi/pull/9229#discussion_r1273169390 ## hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieCompactor.java: ## @@ -101,6 +104,12 @@ public static class Config implements Serializable { public Stri

[GitHub] [hudi] big-doudou commented on issue #8892: [SUPPORT] [BUG] Duplicate fileID ??? from bucket ?? of partition found during the BucketStreamWriteFunction index bootstrap.

2023-07-25 Thread via GitHub
big-doudou commented on issue #8892: URL: https://github.com/apache/hudi/issues/8892#issuecomment-1649335027 > What if this is an new task that has not yet had a successful checkpoint -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [hudi] hudi-bot commented on pull request #9209: [HUDI-6539] New LSM tree style archived timeline

2023-07-25 Thread via GitHub
hudi-bot commented on PR #9209: URL: https://github.com/apache/hudi/pull/9209#issuecomment-1649331921 ## CI report: * 8f2dc4ec3e26f1908ae5d15f194bf70ca7dab27e UNKNOWN * c281ded6d554350dfe362cce496d6d72cfe0bbbe Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-

  1   2   >