Re: [PR] [HUDI-7138] Fix error table writer and schema registry provider [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10173: URL: https://github.com/apache/hudi/pull/10173#issuecomment-1831391203 ## CI report: * 966784294d477e7d0f05b63a65a011f1b3280e2f Azure:

Re: [PR] [HUDI-7158] Expose event time config to record event time in commit file [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10207: URL: https://github.com/apache/hudi/pull/10207#issuecomment-1831391403 ## CI report: * c9bb337aa403e056f7af7b3c1ce1e169561b2454 Azure:

Re: [PR] [HUDI-7135] Spark reads hudi table error when flink creates the table without pre… [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10157: URL: https://github.com/apache/hudi/pull/10157#issuecomment-1831391110 ## CI report: * b4a8099bfd9548b65a81dab1edc91711a46245d9 Azure:

Re: [PR] [HUDI-6822] Fix deletes handling in hbase index when partition path is updated [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #9630: URL: https://github.com/apache/hudi/pull/9630#issuecomment-1831390344 ## CI report: * b6faf77fe94fabb9c9325bcc59a38ebbfecc3595 Azure:

Re: [PR] [HUDI-7135] Spark reads hudi table error when flink creates the table without pre… [hudi]

2023-11-28 Thread via GitHub
empcl commented on PR #10157: URL: https://github.com/apache/hudi/pull/10157#issuecomment-1831384978 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [HUDI-7158] Expose event time config to record event time in commit file [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10207: URL: https://github.com/apache/hudi/pull/10207#issuecomment-1831380724 ## CI report: * c9bb337aa403e056f7af7b3c1ce1e169561b2454 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

(hudi) branch master updated: [HUDI-7103] Support time travel queies for COW tables (#10109)

2023-11-28 Thread codope
This is an automated email from the ASF dual-hosted git repository. codope pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 91eababd36c [HUDI-7103] Support time travel

Re: [PR] [HUDI-7103] Support time travel queies for COW tables [hudi]

2023-11-28 Thread via GitHub
codope merged PR #10109: URL: https://github.com/apache/hudi/pull/10109 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [HUDI-7138] Fix error table writer and schema registry provider [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10173: URL: https://github.com/apache/hudi/pull/10173#issuecomment-1831380536 ## CI report: * 966784294d477e7d0f05b63a65a011f1b3280e2f Azure:

Re: [PR] [HUDI-6822] Fix deletes handling in hbase index when partition path is updated [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #9630: URL: https://github.com/apache/hudi/pull/9630#issuecomment-1831379723 ## CI report: * b6faf77fe94fabb9c9325bcc59a38ebbfecc3595 Azure:

Re: [PR] [HUDI-7135] Spark reads hudi table error when flink creates the table without pre… [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10157: URL: https://github.com/apache/hudi/pull/10157#issuecomment-1831370851 ## CI report: * b4a8099bfd9548b65a81dab1edc91711a46245d9 Azure:

Re: [PR] [HUDI-6822] Fix deletes handling in hbase index when partition path is updated [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #9630: URL: https://github.com/apache/hudi/pull/9630#issuecomment-1831370222 ## CI report: * b6faf77fe94fabb9c9325bcc59a38ebbfecc3595 Azure:

[jira] [Updated] (HUDI-7158) Expose event time field to record event time in commit file

2023-11-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-7158: - Labels: pull-request-available (was: ) > Expose event time field to record event time in commit

[PR] [HUDI-7158] Expose event time config to record event time in commit file [hudi]

2023-11-28 Thread via GitHub
zhangyue19921010 opened a new pull request, #10207: URL: https://github.com/apache/hudi/pull/10207 ### Change Logs Currently, we use `precombine.field` as `eventtime.field`. Generally speaking, **precombine**, **ordering** and **event time** are all time fields. But sometimes

Re: [PR] [HUDI-7140] [DNM] Trial Patch to test CI run [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10176: URL: https://github.com/apache/hudi/pull/10176#issuecomment-1831323543 ## CI report: * 461e14bb4b2d7582e7ccf349437aa06a2e6346c2 Azure:

Re: [PR] [HUDI-7138] Fix error table writer and schema registry provider [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10173: URL: https://github.com/apache/hudi/pull/10173#issuecomment-1831323510 ## CI report: * 966784294d477e7d0f05b63a65a011f1b3280e2f Azure:

Re: [PR] [HUDI-7135] Spark reads hudi table error when flink creates the table without pre… [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10157: URL: https://github.com/apache/hudi/pull/10157#issuecomment-1831323425 ## CI report: * b4a8099bfd9548b65a81dab1edc91711a46245d9 Azure:

Re: [PR] [HUDI-7103] Support time travel queies for COW tables [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10109: URL: https://github.com/apache/hudi/pull/10109#issuecomment-1831323277 ## CI report: * 74f246c1e3eae40f081e9861a84d254d8fc01002 UNKNOWN * 2a6794db7db68119aebe7d674304641df1ca8f1e Azure:

Re: [PR] [HUDI-7138] Fix error table writer and schema registry provider [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10173: URL: https://github.com/apache/hudi/pull/10173#issuecomment-1831316824 ## CI report: * 966784294d477e7d0f05b63a65a011f1b3280e2f Azure:

Re: [PR] [HUDI-7140] [DNM] Trial Patch to test CI run [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10176: URL: https://github.com/apache/hudi/pull/10176#issuecomment-1831316881 ## CI report: * 461e14bb4b2d7582e7ccf349437aa06a2e6346c2 Azure:

[jira] [Created] (HUDI-7158) Expose event time field to record event time in commit file

2023-11-28 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-7158: --- Summary: Expose event time field to record event time in commit file Key: HUDI-7158 URL: https://issues.apache.org/jira/browse/HUDI-7158 Project: Apache Hudi Issue

Re: [PR] 0.13.1 custom [hudi]

2023-11-28 Thread via GitHub
sabbasani closed pull request #10206: 0.13.1 custom URL: https://github.com/apache/hudi/pull/10206 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[PR] 0.13.1 custom [hudi]

2023-11-28 Thread via GitHub
sabbasani opened a new pull request, #10206: URL: https://github.com/apache/hudi/pull/10206 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any

Re: [PR] [HUDI-7135] Spark reads hudi table error when flink creates the table without pre… [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10157: URL: https://github.com/apache/hudi/pull/10157#issuecomment-1831309833 ## CI report: * b4a8099bfd9548b65a81dab1edc91711a46245d9 Azure:

Re: [PR] [HUDI-7102] Fix a bug for time travel queries on MOR tables [hudi]

2023-11-28 Thread via GitHub
nsivabalan commented on code in PR #10102: URL: https://github.com/apache/hudi/pull/10102#discussion_r1408813732 ## hudi-common/src/main/java/org/apache/hudi/common/table/log/BaseHoodieLogRecordReader.java: ## @@ -260,7 +260,7 @@ private void scanInternalV1(Option keySpecOpt) {

Re: [I] [SUPPORT] Deep integration flink cdc? [hudi]

2023-11-28 Thread via GitHub
melin commented on issue #9873: URL: https://github.com/apache/hudi/issues/9873#issuecomment-1831304888 Both paimon and hudi can support loss data writing very well. There is a strong demand for real-time synchronization of data from relational databases to data lakes. paimon CDC Ingestion

Re: [PR] [HUDI-7135] Spark reads hudi table error when flink creates the table without pre… [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10157: URL: https://github.com/apache/hudi/pull/10157#issuecomment-1831303384 ## CI report: * b4a8099bfd9548b65a81dab1edc91711a46245d9 Azure:

Re: [PR] [HUDI-6822] Fix deletes handling in hbase index when partition path is updated [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #9630: URL: https://github.com/apache/hudi/pull/9630#issuecomment-1831302922 ## CI report: * a37708f3db08326bda8b446afca3eebfa9f1b1fd Azure:

(hudi) branch master updated: [HUDI-7086] Fix the default for gcp pub sub max sync time to 1min (#10171)

2023-11-28 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new b300728913a [HUDI-7086] Fix the default for

Re: [PR] [HUDI-7086] Fix the default for gcp pub sub max sync time to 1min for PR #10073 [hudi]

2023-11-28 Thread via GitHub
nsivabalan merged PR #10171: URL: https://github.com/apache/hudi/pull/10171 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [HUDI-7086] Fix the default for gcp pub sub max sync time to 1min for PR #10073 [hudi]

2023-11-28 Thread via GitHub
nsivabalan commented on PR #10171: URL: https://github.com/apache/hudi/pull/10171#issuecomment-1831297815 https://github.com/apache/hudi/assets/513218/48c83551-6345-4c10-8b84-479a22ce521f;> -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] [HUDI-7135] Spark reads hudi table error when flink creates the table without pre… [hudi]

2023-11-28 Thread via GitHub
empcl commented on PR #10157: URL: https://github.com/apache/hudi/pull/10157#issuecomment-1831294777 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [I] [SUPPORT] Compaction & Clustering are not working [hudi]

2023-11-28 Thread via GitHub
Cpandey43 commented on issue #10183: URL: https://github.com/apache/hudi/issues/10183#issuecomment-1831293157 > @Cpandey43 I had the same questions. The name `async` is misleading, currently this suggests the current writer to not perform the specific table service (clustering, compaction)

Re: [PR] [HUDI-6822] Fix deletes handling in hbase index when partition path is updated [hudi]

2023-11-28 Thread via GitHub
nsivabalan commented on code in PR #9630: URL: https://github.com/apache/hudi/pull/9630#discussion_r1408800688 ## hudi-common/src/main/java/org/apache/hudi/common/model/HoodieRecord.java: ## @@ -133,6 +133,11 @@ public String getFieldName() { */ protected

Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]

2023-11-28 Thread via GitHub
zhuanshenbsj1 commented on code in PR #10197: URL: https://github.com/apache/hudi/pull/10197#discussion_r1408795152 ## hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemViewLock.java: ## @@ -0,0 +1,66 @@ +/* + * Licensed to the Apache Software

Re: [PR] [HUDI-6207] spark support bucket index query for table with bucket index [hudi]

2023-11-28 Thread via GitHub
KnightChess commented on code in PR #10191: URL: https://github.com/apache/hudi/pull/10191#discussion_r1408788336 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/BucketIndexSupport.scala: ## @@ -0,0 +1,164 @@ +/* + * Licensed to the Apache Software

Re: [PR] [HUDI-6207] spark support bucket index query for table with bucket index [hudi]

2023-11-28 Thread via GitHub
KnightChess commented on code in PR #10191: URL: https://github.com/apache/hudi/pull/10191#discussion_r1408788336 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/BucketIndexSupport.scala: ## @@ -0,0 +1,164 @@ +/* + * Licensed to the Apache Software

Re: [PR] [HUDI-6207] spark support bucket index query for table with bucket index [hudi]

2023-11-28 Thread via GitHub
KnightChess commented on code in PR #10191: URL: https://github.com/apache/hudi/pull/10191#discussion_r1408780543 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/BucketIndexSupport.scala: ## @@ -0,0 +1,164 @@ +/* + * Licensed to the Apache Software

Re: [PR] [HUDI-6822] Fix deletes handling in hbase index when partition path is updated [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #9630: URL: https://github.com/apache/hudi/pull/9630#issuecomment-1831266076 ## CI report: * a37708f3db08326bda8b446afca3eebfa9f1b1fd Azure:

Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]

2023-11-28 Thread via GitHub
stream2000 commented on code in PR #10197: URL: https://github.com/apache/hudi/pull/10197#discussion_r1408778292 ## hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemViewLock.java: ## @@ -0,0 +1,66 @@ +/* + * Licensed to the Apache Software

Re: [PR] [HUDI-6207] spark support bucket index query for table with bucket index [hudi]

2023-11-28 Thread via GitHub
KnightChess commented on code in PR #10191: URL: https://github.com/apache/hudi/pull/10191#discussion_r1408777462 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieFileIndex.scala: ## @@ -340,9 +347,18 @@ case class HoodieFileIndex(spark:

Re: [I] [Streamer 0.14.0] Upgrading Deltastreamer script to 0.14.0, empty table failing to initialize metadata table [hudi]

2023-11-28 Thread via GitHub
rohitmittapalli commented on issue #10203: URL: https://github.com/apache/hudi/issues/10203#issuecomment-1831257974 User error. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] [Streamer 0.14.0] Upgrading Deltastreamer script to 0.14.0, empty table failing to initialize metadata table [hudi]

2023-11-28 Thread via GitHub
rohitmittapalli closed issue #10203: [Streamer 0.14.0] Upgrading Deltastreamer script to 0.14.0, empty table failing to initialize metadata table URL: https://github.com/apache/hudi/issues/10203 -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] [HUDI-6207] spark support bucket index query for table with bucket index [hudi]

2023-11-28 Thread via GitHub
KnightChess commented on code in PR #10191: URL: https://github.com/apache/hudi/pull/10191#discussion_r1408769304 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieFileIndex.scala: ## @@ -340,9 +347,18 @@ case class HoodieFileIndex(spark:

Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10197: URL: https://github.com/apache/hudi/pull/10197#issuecomment-1831255563 ## CI report: * 0fbcde41acc3320680a6ecbd4e351f59eeaf5c37 Azure:

Re: [PR] [HUDI-7103] Support time travel queies for COW tables [hudi]

2023-11-28 Thread via GitHub
linliu-code commented on code in PR #10109: URL: https://github.com/apache/hudi/pull/10109#discussion_r1408768596 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieHadoopFsRelationFactory.scala: ## @@ -231,28 +231,23 @@ class

Re: [PR] [HUDI-6207] spark support bucket index query for table with bucket index [hudi]

2023-11-28 Thread via GitHub
KnightChess commented on code in PR #10191: URL: https://github.com/apache/hudi/pull/10191#discussion_r1408766751 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieIndexConfig.java: ## @@ -334,6 +334,11 @@ public class HoodieIndexConfig extends

Re: [PR] [HUDI-7148] Add an additional fix to the potential thread insecurity problem of heartbeat client. [hudi]

2023-11-28 Thread via GitHub
eric9204 commented on code in PR #10188: URL: https://github.com/apache/hudi/pull/10188#discussion_r1408745384 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/heartbeat/HoodieHeartbeatClient.java: ## @@ -266,6 +266,11 @@ private void

Re: [PR] [HUDI-7153] Fixing range overflow with kakfa source and spark partition management [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10205: URL: https://github.com/apache/hudi/pull/10205#issuecomment-1831224683 ## CI report: * 678b3d7e20eec614e1fec2eef99dfb0ec74bdae9 Azure:

Re: [PR] [HUDI-6822] Fix deletes handling in hbase index when partition path is updated [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #9630: URL: https://github.com/apache/hudi/pull/9630#issuecomment-1831224173 ## CI report: * a37708f3db08326bda8b446afca3eebfa9f1b1fd Azure:

Re: [PR] [HUDI-6207] spark support bucket index query for table with bucket index [hudi]

2023-11-28 Thread via GitHub
danny0405 commented on code in PR #10191: URL: https://github.com/apache/hudi/pull/10191#discussion_r1408738271 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/BucketIndexSupport.scala: ## @@ -0,0 +1,164 @@ +/* + * Licensed to the Apache Software

Re: [PR] [HUDI-6207] spark support bucket index query for table with bucket index [hudi]

2023-11-28 Thread via GitHub
danny0405 commented on code in PR #10191: URL: https://github.com/apache/hudi/pull/10191#discussion_r1408737359 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/BucketIndexSupport.scala: ## @@ -0,0 +1,164 @@ +/* + * Licensed to the Apache Software

Re: [PR] [HUDI-7153] Fixing range overflow with kakfa source and spark partition management [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10205: URL: https://github.com/apache/hudi/pull/10205#issuecomment-1831213928 ## CI report: * 678b3d7e20eec614e1fec2eef99dfb0ec74bdae9 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

Re: [PR] [HUDI-6822] Fix deletes handling in hbase index when partition path is updated [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #9630: URL: https://github.com/apache/hudi/pull/9630#issuecomment-1831213498 ## CI report: * a37708f3db08326bda8b446afca3eebfa9f1b1fd Azure:

Re: [PR] [HUDI-6207] spark support bucket index query for table with bucket index [hudi]

2023-11-28 Thread via GitHub
danny0405 commented on code in PR #10191: URL: https://github.com/apache/hudi/pull/10191#discussion_r1408732461 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieFileIndex.scala: ## @@ -340,9 +347,18 @@ case class HoodieFileIndex(spark:

Re: [PR] [HUDI-7059] Hudi filter pushdown for positional merging [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10167: URL: https://github.com/apache/hudi/pull/10167#issuecomment-1831207734 ## CI report: * 25ee0036413d10722d804e0c935162f2175d7934 UNKNOWN * bd88c58c4ca36bd39363637420cc018deb8bb056 Azure:

Re: [PR] [HUDI-7137] Implement Bootstrap for new FG reader [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10137: URL: https://github.com/apache/hudi/pull/10137#issuecomment-1831207611 ## CI report: * 77205b47c45501a0d9de1ebc74d5bb8c960cd95a UNKNOWN * b0b711e0c355320da652fa7f2d8669539873d4d6 Azure:

Re: [PR] [HUDI-6207] spark support bucket index query for table with bucket index [hudi]

2023-11-28 Thread via GitHub
danny0405 commented on code in PR #10191: URL: https://github.com/apache/hudi/pull/10191#discussion_r1408727800 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieFileIndex.scala: ## @@ -340,9 +347,18 @@ case class HoodieFileIndex(spark:

Re: [PR] [HUDI-7153] Fixing range overflow with kakfa source and spark partition management [hudi]

2023-11-28 Thread via GitHub
nsivabalan commented on code in PR #10205: URL: https://github.com/apache/hudi/pull/10205#discussion_r1408728130 ## hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/KafkaOffsetGen.java: ## @@ -156,7 +163,13 @@ public static OffsetRange[]

[jira] [Updated] (HUDI-7133) Improve dbt example for better guidance

2023-11-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-7133: - Fix Version/s: 0.14.0 (was: 0.14.1) > Improve dbt example for better guidance >

[jira] [Updated] (HUDI-7133) Improve dbt example for better guidance

2023-11-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-7133: - Fix Version/s: 1.0.0 > Improve dbt example for better guidance > ---

[jira] [Closed] (HUDI-7149) Add a dbt example project with CDC capability

2023-11-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-7149. Resolution: Done > Add a dbt example project with CDC capability >

[jira] [Closed] (HUDI-7133) Improve dbt example for better guidance

2023-11-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-7133. Resolution: Done > Improve dbt example for better guidance > --- > >

(hudi) branch master updated: [HUDI-7149] Add a dbt example project with CDC capability (#10192)

2023-11-28 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 8370c62cf7c [HUDI-7149] Add a dbt example

Re: [PR] [HUDI-7149] Add a dbt example project with CDC capability [hudi]

2023-11-28 Thread via GitHub
xushiyan merged PR #10192: URL: https://github.com/apache/hudi/pull/10192 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[PR] [HUDI-7153] Fixing range overflow with kakfa source and spark partition management [hudi]

2023-11-28 Thread via GitHub
nsivabalan opened a new pull request, #10205: URL: https://github.com/apache/hudi/pull/10205 ### Change Logs Fixing range overflow with kakfa source and spark partition management ### Impact Pipeline will not fail with range overflow errors. ### Risk level (write

Re: [PR] Merge schema in ParuqetDFSSource [hudi]

2023-11-28 Thread via GitHub
nsivabalan commented on code in PR #10199: URL: https://github.com/apache/hudi/pull/10199#discussion_r1408722685 ## hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/ParquetDFSSource.java: ## @@ -52,6 +52,6 @@ public Pair>, String> fetchNextBatch(Option lastCkpt

[jira] [Updated] (HUDI-7153) Increasing Kafka minPartitions in Streamer causes corrupted offsets

2023-11-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-7153: - Labels: pull-request-available (was: ) > Increasing Kafka minPartitions in Streamer causes

Re: [PR] [HUDI-7142] Support Custom partitioner in append mode [hudi]

2023-11-28 Thread via GitHub
danny0405 commented on code in PR #10180: URL: https://github.com/apache/hudi/pull/10180#discussion_r1408718598 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/partitioner/DefaultInsertPartitioner.java: ## @@ -0,0 +1,89 @@ +/* + * Licensed to the Apache

Re: [PR] [HUDI-7142] Support Custom partitioner in append mode [hudi]

2023-11-28 Thread via GitHub
danny0405 commented on code in PR #10180: URL: https://github.com/apache/hudi/pull/10180#discussion_r1408718132 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/utils/Pipelines.java: ## @@ -213,14 +215,35 @@ public static DataStream append(

Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]

2023-11-28 Thread via GitHub
zhuanshenbsj1 commented on code in PR #10197: URL: https://github.com/apache/hudi/pull/10197#discussion_r1408708315 ## hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemViewLock.java: ## @@ -0,0 +1,66 @@ +/* + * Licensed to the Apache Software

Re: [PR] [HUDI-7103] Support time travel queies for COW tables [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10109: URL: https://github.com/apache/hudi/pull/10109#issuecomment-1831176653 ## CI report: * 7365ed3dfa8109d3f41f5acfb2c3d90dc16a1583 Azure:

Re: [PR] [HUDI-7150] ExternalSpillableMap support values method [hudi]

2023-11-28 Thread via GitHub
xuzifu666 commented on code in PR #10194: URL: https://github.com/apache/hudi/pull/10194#discussion_r1408698876 ## hudi-common/src/main/java/org/apache/hudi/common/util/collection/ExternalSpillableMap.java: ## @@ -272,7 +272,10 @@ public Collection values() { return

Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]

2023-11-28 Thread via GitHub
stream2000 commented on code in PR #10197: URL: https://github.com/apache/hudi/pull/10197#discussion_r1408704768 ## hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemViewLock.java: ## @@ -0,0 +1,66 @@ +/* + * Licensed to the Apache Software

Re: [PR] [HUDI-6497] WIP HoodieStorage abstraction [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10185: URL: https://github.com/apache/hudi/pull/10185#issuecomment-1831171560 ## CI report: * 72201eb9e3ee19dc3e2cd815bc035af8f435b98f UNKNOWN * dd88a687f7a95799cc4da6e71809c679cdf91673 Azure:

Re: [PR] [HUDI-7059] Hudi filter pushdown for positional merging [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10167: URL: https://github.com/apache/hudi/pull/10167#issuecomment-1831171479 ## CI report: * 3e5d15f1c8463da55402f8a6ce2974fc7c34ed2c Azure:

Re: [PR] [MINOR] Add log to print wrong number of instant metadata files [hudi]

2023-11-28 Thread via GitHub
nsivabalan merged PR #10196: URL: https://github.com/apache/hudi/pull/10196 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [HUDI-7103] Support time travel queies for COW tables [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10109: URL: https://github.com/apache/hudi/pull/10109#issuecomment-1831171353 ## CI report: * 7365ed3dfa8109d3f41f5acfb2c3d90dc16a1583 Azure:

(hudi) branch master updated (cadeadecb7b -> 817d81ad14f)

2023-11-28 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from cadeadecb7b [minor] when metric prefix length is 0 ignore the metric prefix (#10190) add 817d81ad14f [MINOR]

Re: [PR] [MINOR] Add log to print wrong number of instant metadata files [hudi]

2023-11-28 Thread via GitHub
nsivabalan commented on PR #10196: URL: https://github.com/apache/hudi/pull/10196#issuecomment-1831171006 https://github.com/apache/hudi/assets/513218/98077eea-34a4-4e9b-9711-85ab533dce1a;> -- This is an automated message from the Apache Git Service. To respond to the message, please

(hudi) branch master updated: [minor] when metric prefix length is 0 ignore the metric prefix (#10190)

2023-11-28 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new cadeadecb7b [minor] when metric prefix length

Re: [PR] [minor] when metric prefix length is 0 ignore the metric prefix [hudi]

2023-11-28 Thread via GitHub
nsivabalan merged PR #10190: URL: https://github.com/apache/hudi/pull/10190 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [minor] when metric prefix length is 0 ignore the metric prefix [hudi]

2023-11-28 Thread via GitHub
nsivabalan commented on PR #10190: URL: https://github.com/apache/hudi/pull/10190#issuecomment-1831170579 https://github.com/apache/hudi/assets/513218/f7056282-a574-493b-ac0d-3efec00f8a00;> -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] [HUDI-6497] WIP HoodieStorage abstraction [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10185: URL: https://github.com/apache/hudi/pull/10185#issuecomment-1831165846 ## CI report: * 72201eb9e3ee19dc3e2cd815bc035af8f435b98f UNKNOWN * 3342f0aa8fd7055ef77723939e831f092df4262b Azure:

Re: [PR] [HUDI-7059] Hudi filter pushdown for positional merging [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10167: URL: https://github.com/apache/hudi/pull/10167#issuecomment-1831165764 ## CI report: * db36e58f51ce908ff4d4e810b7997ab0ab7afa05 Azure:

Re: [PR] [HUDI-7103] Support time travel queies for COW tables [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10109: URL: https://github.com/apache/hudi/pull/10109#issuecomment-1831165616 ## CI report: * 7365ed3dfa8109d3f41f5acfb2c3d90dc16a1583 Azure:

Re: [PR] [HUDI-7150] ExternalSpillableMap support values method [hudi]

2023-11-28 Thread via GitHub
xuzifu666 commented on code in PR #10194: URL: https://github.com/apache/hudi/pull/10194#discussion_r1408698876 ## hudi-common/src/main/java/org/apache/hudi/common/util/collection/ExternalSpillableMap.java: ## @@ -272,7 +272,10 @@ public Collection values() { return

[I] [SUPPORT] [hudi]

2023-11-28 Thread via GitHub
XenosK opened a new issue, #10204: URL: https://github.com/apache/hudi/issues/10204 **_Tips before filing an issue_** - Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)? - Join the mailing list to engage in conversations and get faster support at

Re: [PR] [HUDI-7102] Fix a bug for time travel queries on MOR tables [hudi]

2023-11-28 Thread via GitHub
yihua merged PR #10102: URL: https://github.com/apache/hudi/pull/10102 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(hudi) branch master updated: [HUDI-7102] Fix bugs related to time travel queries (#10102)

2023-11-28 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 91daa7da589 [HUDI-7102] Fix bugs related to time

Re: [PR] [HUDI-6207] spark support bucket index query for table with bucket index [hudi]

2023-11-28 Thread via GitHub
danny0405 commented on code in PR #10191: URL: https://github.com/apache/hudi/pull/10191#discussion_r1408668639 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieIndexConfig.java: ## @@ -334,6 +334,11 @@ public class HoodieIndexConfig extends

Re: [PR] [HUDI-7150] ExternalSpillableMap support values method [hudi]

2023-11-28 Thread via GitHub
danny0405 commented on code in PR #10194: URL: https://github.com/apache/hudi/pull/10194#discussion_r1408667255 ## hudi-common/src/main/java/org/apache/hudi/common/util/collection/ExternalSpillableMap.java: ## @@ -272,7 +272,10 @@ public Collection values() { return

Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10197: URL: https://github.com/apache/hudi/pull/10197#issuecomment-1831132843 ## CI report: * cd1abe7cd0935fb52cb08062458c222a91756684 Azure:

Re: [PR] [HUDI-7059] Hudi filter pushdown for positional merging [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10167: URL: https://github.com/apache/hudi/pull/10167#issuecomment-1831132699 ## CI report: * db36e58f51ce908ff4d4e810b7997ab0ab7afa05 Azure:

Re: [PR] [HUDI-7137] Implement Bootstrap for new FG reader [hudi]

2023-11-28 Thread via GitHub
hudi-bot commented on PR #10137: URL: https://github.com/apache/hudi/pull/10137#issuecomment-1831132621 ## CI report: * 77205b47c45501a0d9de1ebc74d5bb8c960cd95a UNKNOWN * b144ee0b098dcb17972152e07cd57412ad498d47 Azure:

Re: [PR] [HUDI-7059] Hudi filter pushdown for positional merging [hudi]

2023-11-28 Thread via GitHub
linliu-code commented on PR #10167: URL: https://github.com/apache/hudi/pull/10167#issuecomment-1831130832 @yihua -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[jira] [Updated] (HUDI-7148) Add an additional fix to the potential thread insecurity problem of heartbeat client.

2023-11-28 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-7148: - Fix Version/s: 1.1.0 0.14.1 > Add an additional fix to the potential thread insecurity

[jira] [Closed] (HUDI-7148) Add an additional fix to the potential thread insecurity problem of heartbeat client.

2023-11-28 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-7148. Resolution: Fixed Fixed via master branch: d1dfa5b6be3adbe18920dc2819455dbca8b29898 > Add an additional

Re: [PR] [HUDI-7147] Fix CDC write flush bug [hudi]

2023-11-28 Thread via GitHub
zhangyue19921010 commented on code in PR #10186: URL: https://github.com/apache/hudi/pull/10186#discussion_r1408663704 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieCDCLogger.java: ## @@ -183,15 +183,16 @@ public void put(HoodieRecord hoodieRecord,

(hudi) branch master updated: [HUDI-7148] Add an additional fix to the potential thread insecurity problem of heartbeat client (#10188)

2023-11-28 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new d1dfa5b6be3 [HUDI-7148] Add an additional fix

  1   2   3   >