Re: [PR] [Feature] Kafka debezium json supports automatic discovery of primary keys [incubator-paimon]

2024-01-29 Thread via GitHub
sunxiaojian commented on PR #2815: URL: https://github.com/apache/incubator-paimon/pull/2815#issuecomment-1916262440 @yuzelin -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] [core] Introduce privilege system for catalog [incubator-paimon]

2024-01-29 Thread via GitHub
JingsongLi commented on code in PR #2789: URL: https://github.com/apache/incubator-paimon/pull/2789#discussion_r1470702570 ## paimon-core/src/main/java/org/apache/paimon/catalog/AbstractCatalog.java: ## @@ -54,32 +61,43 @@ import static org.apache.paimon.utils.Preconditions.che

[PR] [core] Introduce Context for LookupStoreFactory to avoid writing metadata [incubator-paimon]

2024-01-29 Thread via GitHub
JingsongLi opened a new pull request, #2817: URL: https://github.com/apache/incubator-paimon/pull/2817 … ### Purpose The previous approach made the file structure too complex, which is not conducive to adding file compression in the future. We can interact so

Re: [PR] [Core] Support branch batch/streaming read and write [incubator-paimon]

2024-01-29 Thread via GitHub
TaoZex commented on PR #2748: URL: https://github.com/apache/incubator-paimon/pull/2748#issuecomment-1916151074 > Hi @TaoZex, I browsed through your request and thx for ur work. In my humble opinion, this PR contains lots of changes and can be separated into several PRs in order to improve

Re: [PR] [Core] Support branch batch/streaming read and write [incubator-paimon]

2024-01-29 Thread via GitHub
TaoZex commented on code in PR #2748: URL: https://github.com/apache/incubator-paimon/pull/2748#discussion_r1470636068 ## paimon-core/src/main/java/org/apache/paimon/utils/BranchManager.java: ## @@ -72,6 +73,9 @@ public Path branchPath(String branchName) { } public v

Re: [PR] [Core] Support branch batch/streaming read and write [incubator-paimon]

2024-01-29 Thread via GitHub
TaoZex commented on code in PR #2748: URL: https://github.com/apache/incubator-paimon/pull/2748#discussion_r1470634782 ## paimon-core/src/main/java/org/apache/paimon/utils/BranchManager.java: ## @@ -72,6 +73,9 @@ public Path branchPath(String branchName) { } public v

Re: [PR] [Core] Support branch batch/streaming read and write [incubator-paimon]

2024-01-29 Thread via GitHub
TaoZex commented on code in PR #2748: URL: https://github.com/apache/incubator-paimon/pull/2748#discussion_r1470634482 ## paimon-core/src/test/java/org/apache/paimon/table/FileStoreTableTestBase.java: ## @@ -1240,6 +1248,50 @@ public void testSchemaPathOption() throws Exception

Re: [PR] [CDC]Support oracle sync table cdc [incubator-paimon]

2024-01-29 Thread via GitHub
TaoZex commented on PR #2809: URL: https://github.com/apache/incubator-paimon/pull/2809#issuecomment-1916050236 Close this pr and I will reopen it when flink oracle cdc is more perfect. -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [PR] [CDC]Support oracle sync table cdc [incubator-paimon]

2024-01-29 Thread via GitHub
TaoZex closed pull request #2809: [CDC]Support oracle sync table cdc URL: https://github.com/apache/incubator-paimon/pull/2809 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] [core] Support filter pushdown in FilesTable [incubator-paimon]

2024-01-29 Thread via GitHub
JingsongLi merged PR #2808: URL: https://github.com/apache/incubator-paimon/pull/2808 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...

Re: [PR] [core] Support filter pushdown in FilesTable [incubator-paimon]

2024-01-29 Thread via GitHub
XuQianJin-Stars commented on PR #2808: URL: https://github.com/apache/incubator-paimon/pull/2808#issuecomment-1916022713 Overall it looks like a good implementation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] [cdc] Align some types of debezium json format data with mysql/postgres cdc [incubator-paimon]

2024-01-29 Thread via GitHub
harveyyue commented on PR #2813: URL: https://github.com/apache/incubator-paimon/pull/2813#issuecomment-1915979548 @yuzelin Could you help to look this pr? It will align decimal/set data types to mysql/pg cdc. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] [Hive] HiveCatalog: Table StorageDescriptor location and control HMS FileSystem scan. [incubator-paimon]

2024-01-29 Thread via GitHub
JingsongLi commented on code in PR #2760: URL: https://github.com/apache/incubator-paimon/pull/2760#discussion_r1470517772 ## paimon-hive/paimon-hive-catalog/src/main/java/org/apache/paimon/hive/HiveCatalogOptions.java: ## @@ -52,5 +52,13 @@ public final class HiveCatalogOptions

Re: [PR] [Hive] HiveCatalog: Table StorageDescriptor location and control HMS FileSystem scan. [incubator-paimon]

2024-01-29 Thread via GitHub
JingsongLi commented on code in PR #2760: URL: https://github.com/apache/incubator-paimon/pull/2760#discussion_r1470517305 ## paimon-hive/paimon-hive-catalog/src/main/java/org/apache/paimon/hive/HiveCatalog.java: ## @@ -439,14 +441,41 @@ protected void alterTableImpl(Identifier

Re: [PR] [Core] Support branch batch/streaming read and write [incubator-paimon]

2024-01-29 Thread via GitHub
FangYongs commented on code in PR #2748: URL: https://github.com/apache/incubator-paimon/pull/2748#discussion_r1470514654 ## paimon-core/src/test/java/org/apache/paimon/table/FileStoreTableTestBase.java: ## @@ -1240,6 +1248,50 @@ public void testSchemaPathOption() throws Excepti

Re: [PR] [Flink]add sort compaction config [incubator-paimon]

2024-01-29 Thread via GitHub
wg1026688210 commented on PR #2749: URL: https://github.com/apache/incubator-paimon/pull/2749#issuecomment-1914871524 @leaves12138 @JingsongLi PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] [core] Introduce more compressions for spill [incubator-paimon]

2024-01-29 Thread via GitHub
JingsongLi merged PR #2812: URL: https://github.com/apache/incubator-paimon/pull/2812 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...

Re: [PR] [Chore]: license script generate & update gitignore [incubator-paimon-webui]

2024-01-29 Thread via GitHub
s7monk commented on PR #184: URL: https://github.com/apache/incubator-paimon-webui/pull/184#issuecomment-1914761943 LGTM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[PR] [Feature] Kafka debezium json supports automatic discovery of primary keys [incubator-paimon]

2024-01-29 Thread via GitHub
sunxiaojian opened a new pull request, #2815: URL: https://github.com/apache/incubator-paimon/pull/2815 ### Purpose Linked issue: close #xxx ### Tests ### API and Format ### Documentation -- This is an automated me

Re: [I] [cdc] support debezium-json format with schema for kafka sync action [incubator-paimon]

2024-01-29 Thread via GitHub
sunxiaojian commented on issue #2701: URL: https://github.com/apache/incubator-paimon/issues/2701#issuecomment-1914578744 > > @zhangjun0x01 I remember this feature is already supported. Is there any problem now? 我记得这个功能已经支持了。现在有什么问题吗? > > The json given by flink-cdc does not contain

Re: [PR] [cdc]Fix duplicate schema IDs when multiple parallel updates [incubator-paimon]

2024-01-29 Thread via GitHub
JingsongLi commented on PR #2797: URL: https://github.com/apache/incubator-paimon/pull/2797#issuecomment-1914501925 @chenxi0599 Please fix compile checkstyle. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] [hive] alter paimon table options cannot be synchronized to hive par [incubator-paimon]

2024-01-29 Thread via GitHub
JingsongLi merged PR #2810: URL: https://github.com/apache/incubator-paimon/pull/2810 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...

[PR] [cdc] Align some types of debezium json format data with mysql/postgres cdc [incubator-paimon]

2024-01-29 Thread via GitHub
harveyyue opened a new pull request, #2813: URL: https://github.com/apache/incubator-paimon/pull/2813 ### Purpose Linked issue: close #xxx ### Tests ### API and Format ### Documentation -- This is an automated mess

Re: [PR] [hive] alter paimon table options cannot be synchronized to hive par [incubator-paimon]

2024-01-29 Thread via GitHub
JingsongLi commented on PR #2810: URL: https://github.com/apache/incubator-paimon/pull/2810#issuecomment-1914295997 Looks reasonable to me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] [cdc]Fix duplicate schema IDs when multiple parallel updates [incubator-paimon]

2024-01-29 Thread via GitHub
JingsongLi commented on PR #2797: URL: https://github.com/apache/incubator-paimon/pull/2797#issuecomment-1914292063 +1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u