[GitHub] [hudi] mincwang commented on a change in pull request #3842: [MINOR] Show source table operator details on the flink web when reading hudi table

2021-10-22 Thread GitBox
mincwang commented on a change in pull request #3842: URL: https://github.com/apache/hudi/pull/3842#discussion_r734928697 ## File path: hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableSource.java ## @@ -268,6 +266,16 @@ private DataType getProducedDataType() {

[GitHub] [hudi] hudi-bot edited a comment on pull request #3849: [HUDI-2077] Fix TestHoodieDeltaStreamerWithMultiWriter

2021-10-22 Thread GitBox
hudi-bot edited a comment on pull request #3849: URL: https://github.com/apache/hudi/pull/3849#issuecomment-950068934 ## CI report: * caa150262ba9758d22ff16f9b00f6e5d4d0b6de1 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3849: [HUDI-2077] Fix TestHoodieDeltaStreamerWithMultiWriter

2021-10-22 Thread GitBox
hudi-bot commented on pull request #3849: URL: https://github.com/apache/hudi/pull/3849#issuecomment-950068934 ## CI report: * caa150262ba9758d22ff16f9b00f6e5d4d0b6de1 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[GitHub] [hudi] xushiyan opened a new pull request #3849: [HUDI-2077] Fix TestHoodieDeltaStreamerWithMultiWriter

2021-10-22 Thread GitBox
xushiyan opened a new pull request #3849: URL: https://github.com/apache/hudi/pull/3849 Remove the logic of using deltastreamer to prep test table. Use fixture (compressed test table) instead. ## Committer checklist - [ ] Has a corresponding JIRA in PR title & commit

[GitHub] [hudi] hudi-bot edited a comment on pull request #3746: [HUDI-2515] Add close when producing records failed

2021-10-22 Thread GitBox
hudi-bot edited a comment on pull request #3746: URL: https://github.com/apache/hudi/pull/3746#issuecomment-933086527 ## CI report: * 5531fdbbb2adf2484ac3ff19f3a404df01ac27c9 Azure:

[GitHub] [hudi] dongkelun commented on a change in pull request #3746: [HUDI-2515] Add close when producing records failed

2021-10-22 Thread GitBox
dongkelun commented on a change in pull request #3746: URL: https://github.com/apache/hudi/pull/3746#discussion_r734925672 ## File path: hudi-common/src/test/java/org/apache/hudi/common/util/TestParquetReaderIterator.java ## @@ -59,6 +59,6 @@ public void testParquetIterator()

[GitHub] [hudi] hudi-bot edited a comment on pull request #3746: [HUDI-2515] Add close when producing records failed

2021-10-22 Thread GitBox
hudi-bot edited a comment on pull request #3746: URL: https://github.com/apache/hudi/pull/3746#issuecomment-933086527 ## CI report: * bc2c849af2a376bae8c46f10a9386963faf0ccbf Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3746: [HUDI-2515] Add close when producing records failed

2021-10-22 Thread GitBox
hudi-bot edited a comment on pull request #3746: URL: https://github.com/apache/hudi/pull/3746#issuecomment-933086527 ## CI report: * bc2c849af2a376bae8c46f10a9386963faf0ccbf Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3843: [HUDI-2468] Metadata table support for rolling back the first commit

2021-10-22 Thread GitBox
hudi-bot edited a comment on pull request #3843: URL: https://github.com/apache/hudi/pull/3843#issuecomment-949237299 ## CI report: * 6cdf6076aea4fed2add859df4faf040abe15a716 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3843: [HUDI-2468] Metadata table support for rolling back the first commit

2021-10-22 Thread GitBox
hudi-bot edited a comment on pull request #3843: URL: https://github.com/apache/hudi/pull/3843#issuecomment-949237299 ## CI report: * 6cdf6076aea4fed2add859df4faf040abe15a716 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path

2021-10-22 Thread GitBox
hudi-bot edited a comment on pull request #3817: URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601 ## CI report: * c005b08e4be6127f18364cb75d7f7f23d4e98ec9 Azure:

[GitHub] [hudi] manojpec commented on pull request #3843: [HUDI-2468] Metadata table support for rolling back the first commit

2021-10-22 Thread GitBox
manojpec commented on pull request #3843: URL: https://github.com/apache/hudi/pull/3843#issuecomment-950043840 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path

2021-10-22 Thread GitBox
hudi-bot edited a comment on pull request #3817: URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601 ## CI report: * a11cd8809c5d83f7911c3074ab69b65fd6b0621b Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path

2021-10-22 Thread GitBox
hudi-bot edited a comment on pull request #3817: URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601 ## CI report: * a11cd8809c5d83f7911c3074ab69b65fd6b0621b Azure:

[hudi] branch asf-site updated: [HUDI-2539] Update the config keys of 0.8.0 version in the docs to 0.9.0 (#3775)

2021-10-22 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 7de5edb [HUDI-2539] Update the config keys

[GitHub] [hudi] xushiyan merged pull request #3775: [HUDI-2539] Update the config keys of 0.8.0 version in the docs to 0.9.0

2021-10-22 Thread GitBox
xushiyan merged pull request #3775: URL: https://github.com/apache/hudi/pull/3775 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[hudi] branch master updated (1e285dc -> 5ed35bf)

2021-10-22 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 1e285dc [HUDI-2489]Tuning HoodieROTablePathFilter by caching hoodieTableFileSystemView, aiming to reduce

[GitHub] [hudi] nsivabalan commented on pull request #3741: [HUDI-2501] Add HoodieData abstraction and refactor compaction actions in hudi-client module

2021-10-22 Thread GitBox
nsivabalan commented on pull request #3741: URL: https://github.com/apache/hudi/pull/3741#issuecomment-949923546 @yihua : Good job on the patch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] nsivabalan merged pull request #3741: [HUDI-2501] Add HoodieData abstraction and refactor compaction actions in hudi-client module

2021-10-22 Thread GitBox
nsivabalan merged pull request #3741: URL: https://github.com/apache/hudi/pull/3741 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] yihua commented on a change in pull request #3778: [HUDI-2502] Refactor index in hudi-client module

2021-10-22 Thread GitBox
yihua commented on a change in pull request #3778: URL: https://github.com/apache/hudi/pull/3778#discussion_r734808026 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/index/inmemory/HoodieInMemoryHashIndex.java ## @@ -7,72 +7,77 @@ * "License");

[GitHub] [hudi] yihua commented on a change in pull request #3778: [HUDI-2502] Refactor index in hudi-client module

2021-10-22 Thread GitBox
yihua commented on a change in pull request #3778: URL: https://github.com/apache/hudi/pull/3778#discussion_r734805982 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/index/bloom/HoodieBloomIndex.java ## @@ -0,0 +1,237 @@ +/* + * Licensed to the

[GitHub] [hudi] manojpec commented on a change in pull request #3843: [HUDI-2468] Metadata table support for rolling back the first commit

2021-10-22 Thread GitBox
manojpec commented on a change in pull request #3843: URL: https://github.com/apache/hudi/pull/3843#discussion_r734772787 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTable.java ## @@ -706,11 +707,23 @@ public HoodieEngineContext

[jira] [Created] (HUDI-2604) CommitMetadata should implement a common interface like all other action metadata

2021-10-22 Thread Manoj Govindassamy (Jira)
Manoj Govindassamy created HUDI-2604: Summary: CommitMetadata should implement a common interface like all other action metadata Key: HUDI-2604 URL: https://issues.apache.org/jira/browse/HUDI-2604

[jira] [Updated] (HUDI-2603) Metadata table bootstrapping is missed out when the feature is disabled intermittently

2021-10-22 Thread Manoj Govindassamy (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HUDI-2603: - Description: Metadata table is boostrapped whenever it finds its commits not synced up

[jira] [Created] (HUDI-2603) Metadata table bootstrapping is missed out when the feature is disabled intermittently

2021-10-22 Thread Manoj Govindassamy (Jira)
Manoj Govindassamy created HUDI-2603: Summary: Metadata table bootstrapping is missed out when the feature is disabled intermittently Key: HUDI-2603 URL: https://issues.apache.org/jira/browse/HUDI-2603

[GitHub] [hudi] nsivabalan commented on a change in pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path

2021-10-22 Thread GitBox
nsivabalan commented on a change in pull request #3817: URL: https://github.com/apache/hudi/pull/3817#discussion_r734753400 ## File path: hudi-spark-datasource/hudi-spark/src/main/java/org/apache/hudi/HoodieDatasetBulkInsertHelper.java ## @@ -79,18 +78,19 @@

[GitHub] [hudi] stym06 commented on issue #2265: Arrays with nulls in them result in broken parquet files

2021-10-22 Thread GitBox
stym06 commented on issue #2265: URL: https://github.com/apache/hudi/issues/2265#issuecomment-949826677 We are facing the same issue. Seems like this is happening for nulls in array datatype. We are fetching the schema from confluent schema registry where the datatype is array ``` {

[jira] [Updated] (HUDI-2573) Deadlock w/ multi writer due to double locking

2021-10-22 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2573: -- Status: Patch Available (was: In Progress) > Deadlock w/ multi writer due to double

[jira] [Updated] (HUDI-1937) When clustering fail, generating unfinished replacecommit timeline.

2021-10-22 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-1937: -- Status: In Progress (was: Open) > When clustering fail, generating unfinished

[jira] [Updated] (HUDI-2442) Simplify out-of-box clustering configs

2021-10-22 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2442: -- Status: In Progress (was: Open) > Simplify out-of-box clustering configs >

[jira] [Updated] (HUDI-1877) clustering support for external index

2021-10-22 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-1877: -- Status: Patch Available (was: In Progress) > clustering support for external index >

[jira] [Updated] (HUDI-2469) Implement protobuf based protocol for control plane instead of json

2021-10-22 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2469: -- Status: In Progress (was: Open) > Implement protobuf based protocol for control plane

[jira] [Resolved] (HUDI-2469) Implement protobuf based protocol for control plane instead of json

2021-10-22 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan resolved HUDI-2469. --- Resolution: Fixed > Implement protobuf based protocol for control plane instead of

[GitHub] [hudi] manojpec commented on a change in pull request #3843: [HUDI-2468] Metadata table support for rolling back the first commit

2021-10-22 Thread GitBox
manojpec commented on a change in pull request #3843: URL: https://github.com/apache/hudi/pull/3843#discussion_r734702790 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTable.java ## @@ -706,11 +707,23 @@ public HoodieEngineContext

[GitHub] [hudi] novakov-alexey commented on pull request #3580: [HUDI-1869] Upgrading Spark3 To 3.1

2021-10-22 Thread GitBox
novakov-alexey commented on pull request #3580: URL: https://github.com/apache/hudi/pull/3580#issuecomment-949802449 @pengzhiwei2018 do you want to upgrade us to Spark 3.2 with Scala 2.13 ? -- This is an automated message from the Apache Git Service. To respond to the message, please log

[jira] [Updated] (HUDI-2489) Tuning HoodieROTablePathFilter by caching, aiming to reduce unnecessary list/get requests

2021-10-22 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2489: -- Status: In Progress (was: Open) > Tuning HoodieROTablePathFilter by caching, aiming to

[jira] [Updated] (HUDI-2443) KVComparator in HFile for metadata table is tied to HBase version and shading

2021-10-22 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2443: -- Status: In Progress (was: Open) > KVComparator in HFile for metadata table is tied to

[jira] [Updated] (HUDI-1295) Design and Impl bloom filters as a part of metadata table

2021-10-22 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-1295: -- Status: In Progress (was: Open) > Design and Impl bloom filters as a part of metadata

[jira] [Updated] (HUDI-2332) Implement scheduling of compaction/ clustering for Kafka Connect

2021-10-22 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-2332: Status: Closed (was: Patch Available) > Implement scheduling of compaction/ clustering for Kafka Connect >

[jira] [Reopened] (HUDI-2332) Implement scheduling of compaction/ clustering for Kafka Connect

2021-10-22 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo reopened HUDI-2332: - > Implement scheduling of compaction/ clustering for Kafka Connect >

[jira] [Updated] (HUDI-2502) Refactor index in hudi-client module

2021-10-22 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-2502: Status: Patch Available (was: In Progress) > Refactor index in hudi-client module >

[jira] [Updated] (HUDI-2332) Implement scheduling of compaction/ clustering for Kafka Connect

2021-10-22 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-2332: Status: Patch Available (was: In Progress) > Implement scheduling of compaction/ clustering for Kafka

[jira] [Updated] (HUDI-2501) Refactor compaction actions in hudi-client module

2021-10-22 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-2501: Status: Patch Available (was: In Progress) > Refactor compaction actions in hudi-client module >

[jira] [Updated] (HUDI-2602) Publish design doc/RFC for metadata based range index

2021-10-22 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2602: -- Parent: HUDI-1822 Issue Type: Sub-task (was: Improvement) > Publish design

[jira] [Created] (HUDI-2602) Publish design doc/RFC for metadata based range index

2021-10-22 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-2602: - Summary: Publish design doc/RFC for metadata based range index Key: HUDI-2602 URL: https://issues.apache.org/jira/browse/HUDI-2602 Project: Apache Hudi

[jira] [Updated] (HUDI-2502) Refactor index in hudi-client module

2021-10-22 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-2502: Priority: Blocker (was: Major) > Refactor index in hudi-client module >

[jira] [Updated] (HUDI-2602) Publish design doc/RFC for metadata based range index

2021-10-22 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2602: -- Fix Version/s: 0.10.0 > Publish design doc/RFC for metadata based range index >

[jira] [Updated] (HUDI-2602) Publish design doc/RFC for metadata based range index

2021-10-22 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2602: -- Priority: Blocker (was: Major) > Publish design doc/RFC for metadata based range index

[jira] [Updated] (HUDI-2501) Refactor compaction actions in hudi-client module

2021-10-22 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-2501: Priority: Blocker (was: Major) > Refactor compaction actions in hudi-client module >

[GitHub] [hudi] vinothchandar commented on pull request #3741: [HUDI-2501] Add HoodieData abstraction and refactor compaction actions in hudi-client module

2021-10-22 Thread GitBox
vinothchandar commented on pull request #3741: URL: https://github.com/apache/hudi/pull/3741#issuecomment-949771866 @nsivabalan you can land this when you are happy with the changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] vinothchandar commented on pull request #3741: [HUDI-2501] Add HoodieData abstraction and refactor compaction actions in hudi-client module

2021-10-22 Thread GitBox
vinothchandar commented on pull request #3741: URL: https://github.com/apache/hudi/pull/3741#issuecomment-949771555 Seems to have passed. https://dev.azure.com/apache-hudi-ci-org/apache-hudi-ci/_build?definitionId=5&_a=summary=2=2139 -- This is an automated message from the Apache Git

[GitHub] [hudi] codope commented on pull request #3833: [HUDI-1877] Support records staying in same fileId after clustering

2021-10-22 Thread GitBox
codope commented on pull request #3833: URL: https://github.com/apache/hudi/pull/3833#issuecomment-949768742 @satishkotha Could you please take a look at this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hudi] nsivabalan commented on a change in pull request #3802: [HUDI-1500] Support replace commit in DeltaSync with commit metadata preserved

2021-10-22 Thread GitBox
nsivabalan commented on a change in pull request #3802: URL: https://github.com/apache/hudi/pull/3802#discussion_r734671583 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/functional/TestHoodieDeltaStreamer.java ## @@ -1137,8 +1144,9 @@ public void

[hudi] branch master updated: [HUDI-2489]Tuning HoodieROTablePathFilter by caching hoodieTableFileSystemView, aiming to reduce unnecessary list/get requests (#3719)

2021-10-22 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 1e285dc [HUDI-2489]Tuning

[GitHub] [hudi] nsivabalan merged pull request #3719: [HUDI-2489]Tuning HoodieROTablePathFilter by caching hoodieTableFileSystemView, aiming to reduce unnecessary list/get requests

2021-10-22 Thread GitBox
nsivabalan merged pull request #3719: URL: https://github.com/apache/hudi/pull/3719 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] yihua commented on a change in pull request #3746: [HUDI-2515] Add close when producing records failed

2021-10-22 Thread GitBox
yihua commented on a change in pull request #3746: URL: https://github.com/apache/hudi/pull/3746#discussion_r734653163 ## File path: hudi-common/src/test/java/org/apache/hudi/common/util/TestParquetReaderIterator.java ## @@ -59,6 +59,6 @@ public void testParquetIterator()

[GitHub] [hudi] hudi-bot edited a comment on pull request #3833: [HUDI-1877] Support records staying in same fileId after clustering

2021-10-22 Thread GitBox
hudi-bot edited a comment on pull request #3833: URL: https://github.com/apache/hudi/pull/3833#issuecomment-947674154 ## CI report: * d5f4923465a8960ce35f5b1ae6980ff3f2c5390d Azure:

[jira] [Assigned] (HUDI-1822) [Umbrella] Multi Modal Indexing

2021-10-22 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-1822: Assignee: sivabalan narayanan (was: satish) > [Umbrella] Multi Modal Indexing >

[jira] [Updated] (HUDI-2601) Support point lookup queries leveraging the bloom filter indexing

2021-10-22 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-2601: - Description: dev list thread

[jira] [Created] (HUDI-2601) Support point lookup queries leveraging the bloom filter indexing

2021-10-22 Thread Vinoth Chandar (Jira)
Vinoth Chandar created HUDI-2601: Summary: Support point lookup queries leveraging the bloom filter indexing Key: HUDI-2601 URL: https://issues.apache.org/jira/browse/HUDI-2601 Project: Apache Hudi

[GitHub] [hudi] hudi-bot edited a comment on pull request #3833: [HUDI-1877] Support records staying in same fileId after clustering

2021-10-22 Thread GitBox
hudi-bot edited a comment on pull request #3833: URL: https://github.com/apache/hudi/pull/3833#issuecomment-947674154 ## CI report: * 24834426e81de8cd3e96e317d2a7d3dcbaf07117 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3833: [HUDI-1877] Support records staying in same fileId after clustering

2021-10-22 Thread GitBox
hudi-bot edited a comment on pull request #3833: URL: https://github.com/apache/hudi/pull/3833#issuecomment-947674154 ## CI report: * 24834426e81de8cd3e96e317d2a7d3dcbaf07117 Azure:

[jira] [Commented] (HUDI-2592) NumberFormatException: Zero length BigInteger when write.precombine.field is decimal type

2021-10-22 Thread Matrix42 (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17432982#comment-17432982 ] Matrix42 commented on HUDI-2592: [~yanghua]Thanks > NumberFormatException: Zero length BigInteger when

[GitHub] [hudi] BenjMaq opened a new issue #3848: [SUPPORT] Cannot write to null outputStream error

2021-10-22 Thread GitBox
BenjMaq opened a new issue #3848: URL: https://github.com/apache/hudi/issues/3848 **_Tips before filing an issue_** - Have you gone through our [FAQs](https://cwiki.apache.org/confluence/display/HUDI/FAQ)? - Join the mailing list to engage in conversations and get faster

[GitHub] [hudi] BenjMaq commented on issue #3845: [SUPPORT]`if not exists` doesn't work on create table in spark-sql

2021-10-22 Thread GitBox
BenjMaq commented on issue #3845: URL: https://github.com/apache/hudi/issues/3845#issuecomment-949640297 I am facing the same issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] nsivabalan commented on issue #3841: Schema evolution improvement in 0.9.0 brakes existing applications

2021-10-22 Thread GitBox
nsivabalan commented on issue #3841: URL: https://github.com/apache/hudi/issues/3841#issuecomment-949639563 hmmm. may I know whats the value for `hoodie.deltastreamer.schemaprovider.spark_avro_post_processor.enable` config ? Any changes in any other config values in general?

[GitHub] [hudi] nsivabalan commented on a change in pull request #3843: [HUDI-2468] Metadata table support for rolling back the first commit

2021-10-22 Thread GitBox
nsivabalan commented on a change in pull request #3843: URL: https://github.com/apache/hudi/pull/3843#discussion_r734526072 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTable.java ## @@ -706,11 +707,23 @@ public HoodieEngineContext

[GitHub] [hudi] nsivabalan commented on a change in pull request #3803: [HUDI-2472] Enabling Metadata table for TestCleaner unit tests

2021-10-22 Thread GitBox
nsivabalan commented on a change in pull request #3803: URL: https://github.com/apache/hudi/pull/3803#discussion_r734524064 ## File path: hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/table/TestCleaner.java ## @@ -1319,15 +1319,15 @@ public void

[GitHub] [hudi] nsivabalan commented on pull request #3630: [HUDI-313] NPE when select count start from a realtime table

2021-10-22 Thread GitBox
nsivabalan commented on pull request #3630: URL: https://github.com/apache/hudi/pull/3630#issuecomment-949613024 @codope : cool. thanks for reproducing it. @uncleGen : Can you add a unit test. Once added, we can land this in. -- This is an automated message from the Apache Git

[GitHub] [hudi] hudi-bot edited a comment on pull request #3847: [HUDI-2600][WIP] Remove duplicated hadoop-common with tests classifier exists in bundles

2021-10-22 Thread GitBox
hudi-bot edited a comment on pull request #3847: URL: https://github.com/apache/hudi/pull/3847#issuecomment-949419768 ## CI report: * 09bec16fc3663bca8d0ddb1563fd02d58b723915 Azure:

[jira] [Closed] (HUDI-2592) NumberFormatException: Zero length BigInteger when write.precombine.field is decimal type

2021-10-22 Thread vinoyang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang closed HUDI-2592. -- Resolution: Fixed > NumberFormatException: Zero length BigInteger when write.precombine.field is > decimal

[jira] [Reopened] (HUDI-2592) NumberFormatException: Zero length BigInteger when write.precombine.field is decimal type

2021-10-22 Thread vinoyang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang reopened HUDI-2592: > NumberFormatException: Zero length BigInteger when write.precombine.field is > decimal type >

[jira] [Commented] (HUDI-2592) NumberFormatException: Zero length BigInteger when write.precombine.field is decimal type

2021-10-22 Thread vinoyang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17432925#comment-17432925 ] vinoyang commented on HUDI-2592: [~Matrix42] I have given you Jira contributor permission. Thanks for your

[jira] [Assigned] (HUDI-2592) NumberFormatException: Zero length BigInteger when write.precombine.field is decimal type

2021-10-22 Thread vinoyang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang reassigned HUDI-2592: -- Assignee: Matrix42 > NumberFormatException: Zero length BigInteger when write.precombine.field is >

[GitHub] [hudi] hudi-bot edited a comment on pull request #3847: [HUDI-2600][WIP] Remove duplicated hadoop-common with tests classifier exists in bundles

2021-10-22 Thread GitBox
hudi-bot edited a comment on pull request #3847: URL: https://github.com/apache/hudi/pull/3847#issuecomment-949419768 ## CI report: * 09bec16fc3663bca8d0ddb1563fd02d58b723915 Azure:

[jira] [Updated] (HUDI-2592) NumberFormatException: Zero length BigInteger when write.precombine.field is decimal type

2021-10-22 Thread vinoyang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang updated HUDI-2592: --- Status: Closed (was: Patch Available) > NumberFormatException: Zero length BigInteger when

[hudi] branch master updated (84ca981 -> 499af7c)

2021-10-22 Thread vinoyang
This is an automated email from the ASF dual-hosted git repository. vinoyang pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 84ca981 [HUDI-2553] Metadata table compaction trigger max delta commits (#3794) add 499af7c [HUDI-2592] Fix

[GitHub] [hudi] yanghua merged pull request #3837: [HUDI-2592] Fix write empty array when write.precombine.field is decimal type

2021-10-22 Thread GitBox
yanghua merged pull request #3837: URL: https://github.com/apache/hudi/pull/3837 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] yanghua commented on pull request #3847: [HUDI-2600][WIP] Remove duplicated hadoop-common with tests classifier exists in bundles

2021-10-22 Thread GitBox
yanghua commented on pull request #3847: URL: https://github.com/apache/hudi/pull/3847#issuecomment-949551190 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] hudi-bot edited a comment on pull request #3844: [HUDI-1869] Upgrading Spark3 To 3.1

2021-10-22 Thread GitBox
hudi-bot edited a comment on pull request #3844: URL: https://github.com/apache/hudi/pull/3844#issuecomment-949285568 ## CI report: * 4bee60d825d0229ba438928463200ccd973003fa Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3847: [HUDI-2600] Remove duplicated hadoop-common with tests classifier exists in bundles

2021-10-22 Thread GitBox
hudi-bot edited a comment on pull request #3847: URL: https://github.com/apache/hudi/pull/3847#issuecomment-949419768 ## CI report: * 09bec16fc3663bca8d0ddb1563fd02d58b723915 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3846: [MINOR] Fix typo,'deseralized' corrected to 'deserialized' & 'Kyro' corrected to 'Kryo'

2021-10-22 Thread GitBox
hudi-bot edited a comment on pull request #3846: URL: https://github.com/apache/hudi/pull/3846#issuecomment-949417027 ## CI report: * 4926acaa4b2922292638a775f7d518d4e4826194 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3803: [HUDI-2472] Enabling Metadata table for TestCleaner unit tests

2021-10-22 Thread GitBox
hudi-bot edited a comment on pull request #3803: URL: https://github.com/apache/hudi/pull/3803#issuecomment-943565382 ## CI report: * cbb003d33eb6ca93122dbe49578a85abe0413d05 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3844: [HUDI-1869] Upgrading Spark3 To 3.1

2021-10-22 Thread GitBox
hudi-bot edited a comment on pull request #3844: URL: https://github.com/apache/hudi/pull/3844#issuecomment-949285568 ## CI report: * 6dc22049caf4e7a04c1d89541f9e633a524c701c Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3847: [HUDI-2600] Remove duplicated hadoop-common with tests classifier exists in bundles

2021-10-22 Thread GitBox
hudi-bot edited a comment on pull request #3847: URL: https://github.com/apache/hudi/pull/3847#issuecomment-949419768 ## CI report: * 09bec16fc3663bca8d0ddb1563fd02d58b723915 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3844: [HUDI-1869] Upgrading Spark3 To 3.1

2021-10-22 Thread GitBox
hudi-bot edited a comment on pull request #3844: URL: https://github.com/apache/hudi/pull/3844#issuecomment-949285568 ## CI report: * 6dc22049caf4e7a04c1d89541f9e633a524c701c Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3746: [HUDI-2515] Add close when producing records failed

2021-10-22 Thread GitBox
hudi-bot edited a comment on pull request #3746: URL: https://github.com/apache/hudi/pull/3746#issuecomment-933086527 ## CI report: * bc2c849af2a376bae8c46f10a9386963faf0ccbf Azure:

[GitHub] [hudi] dongkelun commented on pull request #3746: [HUDI-2515] Add close when producing records failed

2021-10-22 Thread GitBox
dongkelun commented on pull request #3746: URL: https://github.com/apache/hudi/pull/3746#issuecomment-949440528 @yihua Hello, the Azure CI failure has been solved, but I don't know how to write unit test cases, but I have tested it on the server before.Do you have any ideas? -- This is

[GitHub] [hudi] codope edited a comment on issue #3838: [SUPPORT]Can Hudi support more hive version?

2021-10-22 Thread GitBox
codope edited a comment on issue #3838: URL: https://github.com/apache/hudi/issues/3838#issuecomment-949440451 > guess 2.1.1-cdh6.3.2 is not certified to work with hudi? No. We have only ceertified 2.3.1, 2.3.3, 2.3.7. Certification for Hive 3 is in progress.

[GitHub] [hudi] codope commented on issue #3838: [SUPPORT]Can Hudi support more hive version?

2021-10-22 Thread GitBox
codope commented on issue #3838: URL: https://github.com/apache/hudi/issues/3838#issuecomment-949440451 > guess 2.1.1-cdh6.3.2 is not certified to work with hudi? No. We have only ceertified 2.3.1, 2.3.3, 2.3.7. Certification for Hive 3 is in progress. `ThriftHiveMetastore` does

[GitHub] [hudi] codope commented on a change in pull request #3757: [HUDI-2005] Avoiding direct fs calls in HoodieLogFileReader

2021-10-22 Thread GitBox
codope commented on a change in pull request #3757: URL: https://github.com/apache/hudi/pull/3757#discussion_r734361882 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/realtime/RealtimeSplit.java ## @@ -41,6 +42,8 @@ */ List getDeltaLogPaths();

[GitHub] [hudi] hanson2021 commented on issue #3838: [SUPPORT]Can Hudi support more hive version?

2021-10-22 Thread GitBox
hanson2021 commented on issue #3838: URL: https://github.com/apache/hudi/issues/3838#issuecomment-949420628 > @codope do we have a list of supported hive versions? guess `2.1.1-cdh6.3.2` is not certified to work with hudi? ``` Caused by: org.apache.thrift.TApplicationException:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3846: [MINOR] Fix typo,'deseralized' corrected to 'deserialized' & 'Kyro' corrected to 'Kryo'

2021-10-22 Thread GitBox
hudi-bot edited a comment on pull request #3846: URL: https://github.com/apache/hudi/pull/3846#issuecomment-949417027 ## CI report: * 4926acaa4b2922292638a775f7d518d4e4826194 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3847: [HUDI-2600] Remove duplicated hadoop-common with tests classifier exists in bundles

2021-10-22 Thread GitBox
hudi-bot commented on pull request #3847: URL: https://github.com/apache/hudi/pull/3847#issuecomment-949419768 ## CI report: * 09bec16fc3663bca8d0ddb1563fd02d58b723915 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[GitHub] [hudi] hudi-bot commented on pull request #3846: [MINOR] Fix typo,'deseralized' corrected to 'deserialized' & 'Kyro' corrected to 'Kryo'

2021-10-22 Thread GitBox
hudi-bot commented on pull request #3846: URL: https://github.com/apache/hudi/pull/3846#issuecomment-949417027 ## CI report: * 4926acaa4b2922292638a775f7d518d4e4826194 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[jira] [Updated] (HUDI-2600) Remove duplicated hadoop-common with tests classifier exists in bundles

2021-10-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2600: - Labels: pull-request-available (was: ) > Remove duplicated hadoop-common with tests classifier

[GitHub] [hudi] yanghua opened a new pull request #3847: [HUDI-2600] Remove duplicated hadoop-common with tests classifier exists in bundles

2021-10-22 Thread GitBox
yanghua opened a new pull request #3847: URL: https://github.com/apache/hudi/pull/3847 … ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is

[GitHub] [hudi] dongkelun opened a new pull request #3846: [MINOR] Fix typo,'deseralized' corrected to 'deserialized' & 'Kyro' corrected to 'Kryo'

2021-10-22 Thread GitBox
dongkelun opened a new pull request #3846: URL: https://github.com/apache/hudi/pull/3846 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the

[GitHub] [hudi] hudi-bot edited a comment on pull request #3803: [HUDI-2472] Enabling Metadata table for TestCleaner unit tests

2021-10-22 Thread GitBox
hudi-bot edited a comment on pull request #3803: URL: https://github.com/apache/hudi/pull/3803#issuecomment-943565382 ## CI report: * e5b8d0dbf590d70e4836b4194b06ea6db3821dfd Azure:

[GitHub] [hudi] codope commented on pull request #3630: [HUDI-313] NPE when select count start from a realtime table

2021-10-22 Thread GitBox
codope commented on pull request #3630: URL: https://github.com/apache/hudi/pull/3630#issuecomment-949411126 > @codope : Did you get a chance to repro this as vinoth suggested? It does not reproduce in local docker setup. It requires Tez. So, I setup on EMR and I could reproduce the

  1   2   >