[GitHub] [hudi] vinothchandar commented on issue #2592: [SUPPORT] Does latest versions of Hudi (0.7.0, 0.6.0) work with Spark 2.3.0 when reading orc files?

2021-06-04 Thread GitBox
vinothchandar commented on issue #2592: URL: https://github.com/apache/hudi/issues/2592#issuecomment-855195674 Closing due to inactivity and since the issue goes away with spark 2.4 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

[jira] [Updated] (HUDI-1974) Run pyspark and validate that it works correctly with all hudi versions

2021-06-04 Thread Nishith Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishith Agarwal updated HUDI-1974: -- Fix Version/s: 0.9.0 > Run pyspark and validate that it works correctly with all hudi versions >

[GitHub] [hudi] vinothchandar closed issue #2592: [SUPPORT] Does latest versions of Hudi (0.7.0, 0.6.0) work with Spark 2.3.0 when reading orc files?

2021-06-04 Thread GitBox
vinothchandar closed issue #2592: URL: https://github.com/apache/hudi/issues/2592 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please c

[jira] [Updated] (HUDI-1974) Run pyspark and validate that it works correctly with all hudi versions

2021-06-04 Thread Nishith Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishith Agarwal updated HUDI-1974: -- Priority: Blocker (was: Major) > Run pyspark and validate that it works correctly with all hudi

[jira] [Created] (HUDI-1974) Run pyspark and validate that it works correctly with all hudi versions

2021-06-04 Thread Nishith Agarwal (Jira)
Nishith Agarwal created HUDI-1974: - Summary: Run pyspark and validate that it works correctly with all hudi versions Key: HUDI-1974 URL: https://issues.apache.org/jira/browse/HUDI-1974 Project: Apache

[jira] [Updated] (HUDI-1936) Introduce a optional property for conditional upsert

2021-06-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1936: - Labels: features pull-request-available (was: features) > Introduce a optional property for condi

[GitHub] [hudi] vinothchandar commented on issue #2564: Hoodie clean is not deleting old files

2021-06-04 Thread GitBox
vinothchandar commented on issue #2564: URL: https://github.com/apache/hudi/issues/2564#issuecomment-855195284 lets run the integ-test-suite with large number of commit rounds and see everything is healthy. We should use this opp to add a few serious YAML files to the code base. -- Thi

[GitHub] [hudi] fanaticjo opened a new pull request #3035: HUDI-1936 Introduce a optional property for conditional upsert

2021-06-04 Thread GitBox
fanaticjo opened a new pull request #3035: URL: https://github.com/apache/hudi/pull/3035 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the p

[GitHub] [hudi] vinothchandar commented on issue #2517: [SUPPORT]Version information found in metastore differs

2021-06-04 Thread GitBox
vinothchandar commented on issue #2517: URL: https://github.com/apache/hudi/issues/2517#issuecomment-855195036 Can you try with a higher hive version or reproduce this in the docker setup? We run an integ test on each commit, that works on Apache Hive 2.3.1. So I am a bit confused as well

[GitHub] [hudi] vinothchandar commented on issue #2544: [SUPPORT]failed to read timestamp column in version 0.7.0 even when HIVE_SUPPORT_TIMESTAMP is enabled

2021-06-04 Thread GitBox
vinothchandar commented on issue #2544: URL: https://github.com/apache/hudi/issues/2544#issuecomment-855194752 Few things. On master now, we fallback to the datasource when reading a Hudi/Hive table. And the MOR snapshot query uses Spark's reading (not the Hive SerDes). So worth reproducin

[GitHub] [hudi] vinothchandar commented on issue #2509: [SUPPORT] Hudi Spark DataSource saves TimestampType as bigInt

2021-06-04 Thread GitBox
vinothchandar commented on issue #2509: URL: https://github.com/apache/hudi/issues/2509#issuecomment-855194302 Just reading through this again. We def need to understand if this is an issue even when using Spark as the only engine (i.e no registration to HMS). and understand if parquet-avr

[GitHub] [hudi] n3nash commented on issue #2284: [SUPPORT] : Is there a option to achieve SCD 2 in Hudi?

2021-06-04 Thread GitBox
n3nash commented on issue #2284: URL: https://github.com/apache/hudi/issues/2284#issuecomment-855192344 Filed a JIRA for us to document this on Hudi -> https://issues.apache.org/jira/browse/HUDI-1973 -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hudi] n3nash closed issue #2284: [SUPPORT] : Is there a option to achieve SCD 2 in Hudi?

2021-06-04 Thread GitBox
n3nash closed issue #2284: URL: https://github.com/apache/hudi/issues/2284 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact

[jira] [Created] (HUDI-1973) Support SCD2 types in Hudi tables

2021-06-04 Thread Nishith Agarwal (Jira)
Nishith Agarwal created HUDI-1973: - Summary: Support SCD2 types in Hudi tables Key: HUDI-1973 URL: https://issues.apache.org/jira/browse/HUDI-1973 Project: Apache Hudi Issue Type: Improvement

[GitHub] [hudi] vinothchandar commented on issue #1982: [SUPPORT] Not able to write to ADLS Gen2 in Azure Databricks, with error has invalid authority.

2021-06-04 Thread GitBox
vinothchandar commented on issue #1982: URL: https://github.com/apache/hudi/issues/1982#issuecomment-855191779 Next step: Can we test this setup and reproduce this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hudi] vinothchandar closed issue #1981: [SUPPORT] Huge performance Difference Between Hudi and Regular Parquet in Athena

2021-06-04 Thread GitBox
vinothchandar closed issue #1981: URL: https://github.com/apache/hudi/issues/1981 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please c

[GitHub] [hudi] vinothchandar commented on issue #1981: [SUPPORT] Huge performance Difference Between Hudi and Regular Parquet in Athena

2021-06-04 Thread GitBox
vinothchandar commented on issue #1981: URL: https://github.com/apache/hudi/issues/1981#issuecomment-855191697 Closing this issue. 0.7 is at-least in EMR now. But ultimately, this issue boils down to Athena and we have little idea what exact prestoDB (or trino) they are running. -- Thi

[GitHub] [hudi] vinothchandar commented on issue #1962: [SUPPORT] Unable to filter hudi table in hive on partition column

2021-06-04 Thread GitBox
vinothchandar commented on issue #1962: URL: https://github.com/apache/hudi/issues/1962#issuecomment-855191281 https://issues.apache.org/jira/browse/HUDI-1972 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [hudi] vinothchandar closed issue #1962: [SUPPORT] Unable to filter hudi table in hive on partition column

2021-06-04 Thread GitBox
vinothchandar closed issue #1962: URL: https://github.com/apache/hudi/issues/1962 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please c

[jira] [Created] (HUDI-1972) Document set hive.conversion=none when using clause like LIMIT

2021-06-04 Thread Nishith Agarwal (Jira)
Nishith Agarwal created HUDI-1972: - Summary: Document set hive.conversion=none when using clause like LIMIT Key: HUDI-1972 URL: https://issues.apache.org/jira/browse/HUDI-1972 Project: Apache Hudi

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2833: [HUDI-89] Add configOption & refactor Hudi configuration framework

2021-06-04 Thread GitBox
codecov-commenter edited a comment on pull request #2833: URL: https://github.com/apache/hudi/pull/2833#issuecomment-828792354 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2833?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] vinothchandar commented on issue #1962: [SUPPORT] Unable to filter hudi table in hive on partition column

2021-06-04 Thread GitBox
vinothchandar commented on issue #1962: URL: https://github.com/apache/hudi/issues/1962#issuecomment-855191204 I think we should doc better, when `set hive.fetch.task.conversion=none;` is to be set with Hudi tables. Closing this -- This is an automated message from the Apache Git Servic

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2833: [HUDI-89] Add configOption & refactor Hudi configuration framework

2021-06-04 Thread GitBox
codecov-commenter edited a comment on pull request #2833: URL: https://github.com/apache/hudi/pull/2833#issuecomment-828792354 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2833?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] vinothchandar commented on issue #1845: [SUPPORT] Support for Schema evolution. Facing an error

2021-06-04 Thread GitBox
vinothchandar commented on issue #1845: URL: https://github.com/apache/hudi/issues/1845#issuecomment-855190740 @n3nash The concrete thing to do here, is to just take and repro this scenario with master branch and see if its still an issue. -- This is an automated message from the Apache

[GitHub] [hudi] vinothchandar commented on issue #1845: [SUPPORT] Support for Schema evolution. Facing an error

2021-06-04 Thread GitBox
vinothchandar commented on issue #1845: URL: https://github.com/apache/hudi/issues/1845#issuecomment-855190666 cc @nsivabalan as well. This is related to the schema evolution PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[jira] [Commented] (HUDI-1292) [Umbrella] RFC-15 : File Listing and Query Planning Optimizations

2021-06-04 Thread Nishith Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17357765#comment-17357765 ] Nishith Agarwal commented on HUDI-1292: --- Potential issue that can be solved by metad

[GitHub] [hudi] vinothchandar commented on issue #1837: [SUPPORT]S3 file listing causing compaction to get eventually slow

2021-06-04 Thread GitBox
vinothchandar commented on issue #1837: URL: https://github.com/apache/hudi/issues/1837#issuecomment-855190520 Metadata table is out for couple releases now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [hudi] vinothchandar closed issue #1837: [SUPPORT]S3 file listing causing compaction to get eventually slow

2021-06-04 Thread GitBox
vinothchandar closed issue #1837: URL: https://github.com/apache/hudi/issues/1837 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please c

[GitHub] [hudi] vinothchandar closed issue #1552: Time taken for upserting hudi table is increasing with increase in number of partitions

2021-06-04 Thread GitBox
vinothchandar closed issue #1552: URL: https://github.com/apache/hudi/issues/1552 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please c

[jira] [Commented] (HUDI-818) Optimize the default value of hoodie.memory.merge.max.size option

2021-06-04 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17357763#comment-17357763 ] Vinoth Chandar commented on HUDI-818: - Potentially related: [https://github.com/apache

[jira] [Commented] (HUDI-1965) Add a FAQ around upgrading Hudi version in EMR cluster

2021-06-04 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17357762#comment-17357762 ] Vinoth Chandar commented on HUDI-1965: -- [https://dev.to/bytearray/using-your-own-apac

[GitHub] [hudi] vinothchandar closed issue #1491: [SUPPORT] OutOfMemoryError during upsert 53M records

2021-06-04 Thread GitBox
vinothchandar closed issue #1491: URL: https://github.com/apache/hudi/issues/1491 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please c

[jira] [Commented] (HUDI-1281) deltacommit is not part of ActionType used in HoodieArchivedMetaEntry

2021-06-04 Thread Vinay (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17357761#comment-17357761 ] Vinay commented on HUDI-1281: - cf90f17732e313cf71248a8baaf10307463d9b6e > deltacommit is not

[jira] [Resolved] (HUDI-1281) deltacommit is not part of ActionType used in HoodieArchivedMetaEntry

2021-06-04 Thread Vinay (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinay resolved HUDI-1281. - Resolution: Fixed > deltacommit is not part of ActionType used in HoodieArchivedMetaEntry > -

[jira] [Closed] (HUDI-1281) deltacommit is not part of ActionType used in HoodieArchivedMetaEntry

2021-06-04 Thread Vinay (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinay closed HUDI-1281. --- > deltacommit is not part of ActionType used in HoodieArchivedMetaEntry > ---

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2833: [HUDI-89] Add configOption & refactor Hudi configuration framework

2021-06-04 Thread GitBox
codecov-commenter edited a comment on pull request #2833: URL: https://github.com/apache/hudi/pull/2833#issuecomment-828792354 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2833?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[jira] [Updated] (HUDI-1958) [Umbrella] Follow up items from 1 pass over GH issues

2021-06-04 Thread Nishith Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishith Agarwal updated HUDI-1958: -- Priority: Blocker (was: Major) > [Umbrella] Follow up items from 1 pass over GH issues > --

[jira] [Updated] (HUDI-1958) [Umbrella] Follow up items from 1 pass over GH issues

2021-06-04 Thread Nishith Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishith Agarwal updated HUDI-1958: -- Fix Version/s: 0.9.0 > [Umbrella] Follow up items from 1 pass over GH issues > -

[jira] [Updated] (HUDI-1958) [Umbrella] Follow up items from 1 pass over GH issues

2021-06-04 Thread Nishith Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishith Agarwal updated HUDI-1958: -- Labels: release-blocker (was: ) > [Umbrella] Follow up items from 1 pass over GH issues > -

[GitHub] [hudi] n3nash commented on issue #2905: Website redesign

2021-06-04 Thread GitBox
n3nash commented on issue #2905: URL: https://github.com/apache/hudi/issues/2905#issuecomment-855188325 @xushiyan LGTM! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2833: [HUDI-89] Add configOption & refactor Hudi configuration framework

2021-06-04 Thread GitBox
codecov-commenter edited a comment on pull request #2833: URL: https://github.com/apache/hudi/pull/2833#issuecomment-828792354 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2833?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] yuzhaojing commented on pull request #3024: [HUDI-1924] Add BootstrapFunction to support index bootstrap

2021-06-04 Thread GitBox
yuzhaojing commented on pull request #3024: URL: https://github.com/apache/hudi/pull/3024#issuecomment-855187253 @wangxianghu @danny0405 Thanks for review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hudi] wangxianghu commented on a change in pull request #3024: [HUDI-1924] Add BootstrapFunction to support index bootstrap

2021-06-04 Thread GitBox
wangxianghu commented on a change in pull request #3024: URL: https://github.com/apache/hudi/pull/3024#discussion_r645943833 ## File path: hudi-flink/src/main/java/org/apache/hudi/streamer/HoodieFlinkStreamer.java ## @@ -93,8 +95,15 @@ public static void main(String[] args) th

[hudi] branch master updated (c4a2ad2 -> cf90f17)

2021-06-04 Thread vinoth
This is an automated email from the ASF dual-hosted git repository. vinoth pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from c4a2ad2 [HUDI-1954] only reset bucket when flush bucket success (#3029) add cf90f17 [HUDI-1281] Add deltacommit

[GitHub] [hudi] vinothchandar merged pull request #3018: [HUDI-1281] Add deltacommit to ActionType

2021-06-04 Thread GitBox
vinothchandar merged pull request #3018: URL: https://github.com/apache/hudi/pull/3018 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, ple

[GitHub] [hudi] n3nash commented on issue #3022: [FEATURE REQUEST] Better support for Trino

2021-06-04 Thread GitBox
n3nash commented on issue #3022: URL: https://github.com/apache/hudi/issues/3022#issuecomment-855186869 @rubenssoto You should be able to use CopyOnWrite with Trino + Hudi. I'm aware that MOR is not supported. Full Hudi support on Trino is on the roadmap. -- This is an automated message

[GitHub] [hudi] wangxianghu commented on pull request #3024: [HUDI-1924] Add BootstrapFunction to support index bootstrap

2021-06-04 Thread GitBox
wangxianghu commented on pull request #3024: URL: https://github.com/apache/hudi/pull/3024#issuecomment-855186725 The ci failed because of this ``` [ERROR] TestHoodieMultiTableDeltaStreamer>TestHoodieDeltaStreamer.testUpsertsMORContinuousModeWithMultipleWriters:716->TestHoodieDelta

[GitHub] [hudi] n3nash commented on issue #2849: [SUPPORT] - org.apache.hudi.exception.HoodieIOException: Could not load Hoodie properties from file:/tmp/hudi_trips_cow/.hoodie/hoodie.properties

2021-06-04 Thread GitBox
n3nash commented on issue #2849: URL: https://github.com/apache/hudi/issues/2849#issuecomment-855186466 @AdeloreSimiloluwa Closing this issue. Please re-open if you see the same issue or need assistance. -- This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [hudi] n3nash closed issue #2849: [SUPPORT] - org.apache.hudi.exception.HoodieIOException: Could not load Hoodie properties from file:/tmp/hudi_trips_cow/.hoodie/hoodie.properties

2021-06-04 Thread GitBox
n3nash closed issue #2849: URL: https://github.com/apache/hudi/issues/2849 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact

[GitHub] [hudi] n3nash commented on issue #2924: [SUPPORT]presto query:could not initialize class org.apache.hudi.common.util.HoodieAvroUtils

2021-06-04 Thread GitBox
n3nash commented on issue #2924: URL: https://github.com/apache/hudi/issues/2924#issuecomment-855186102 @root18039532923 Can you confirm if hudi-hadoop-mr bundle is in the classpath ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] n3nash commented on issue #2941: the step of get small file has very poor performance by spark struct streaming

2021-06-04 Thread GitBox
n3nash commented on issue #2941: URL: https://github.com/apache/hudi/issues/2941#issuecomment-855185973 @dong0808 Can you provide more details to help debug this issue ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[jira] [Assigned] (HUDI-1971) Debug small file handling taking a long time

2021-06-04 Thread Nishith Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishith Agarwal reassigned HUDI-1971: - Assignee: Nishith Agarwal > Debug small file handling taking a long time > --

[jira] [Created] (HUDI-1971) Debug small file handling taking a long time

2021-06-04 Thread Nishith Agarwal (Jira)
Nishith Agarwal created HUDI-1971: - Summary: Debug small file handling taking a long time Key: HUDI-1971 URL: https://issues.apache.org/jira/browse/HUDI-1971 Project: Apache Hudi Issue Type:

[jira] [Updated] (HUDI-1958) [Umbrella] Follow up items from 1 pass over GH issues

2021-06-04 Thread Nishith Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishith Agarwal updated HUDI-1958: -- Summary: [Umbrella] Follow up items from 1 pass over GH issues (was: [Umbrella] Improve documen

[GitHub] [hudi] wangxianghu commented on a change in pull request #3024: [HUDI-1924] Add BootstrapFunction to support index bootstrap

2021-06-04 Thread GitBox
wangxianghu commented on a change in pull request #3024: URL: https://github.com/apache/hudi/pull/3024#discussion_r645943934 ## File path: hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableSink.java ## @@ -73,10 +76,20 @@ public SinkRuntimeProvider getSinkRuntimeProvide

[GitHub] [hudi] wangxianghu commented on a change in pull request #3024: [HUDI-1924] Add BootstrapFunction to support index bootstrap

2021-06-04 Thread GitBox
wangxianghu commented on a change in pull request #3024: URL: https://github.com/apache/hudi/pull/3024#discussion_r645943833 ## File path: hudi-flink/src/main/java/org/apache/hudi/streamer/HoodieFlinkStreamer.java ## @@ -93,8 +95,15 @@ public static void main(String[] args) th

[GitHub] [hudi] n3nash commented on issue #2973: [SUPPORT]Fail to run compaction in Docker Demo

2021-06-04 Thread GitBox
n3nash commented on issue #2973: URL: https://github.com/apache/hudi/issues/2973#issuecomment-855185577 @fengjian428 Are you running it off the master branch ? I did not see any problems with it. -- This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [hudi] n3nash commented on issue #3009: Dependency error when attempt to build Hudi from git source ..

2021-06-04 Thread GitBox
n3nash commented on issue #3009: URL: https://github.com/apache/hudi/issues/3009#issuecomment-855184359 @arun990 A fix for this was landed today -> https://github.com/apache/hudi/pull/3034 Can you check if the master works for you now ? -- This is an automated message from the Apache Gi

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2963: [HUDI-1904] Make SchemaProvider spark free and move it to hudi-client-common

2021-06-04 Thread GitBox
codecov-commenter edited a comment on pull request #2963: URL: https://github.com/apache/hudi/pull/2963#issuecomment-843155329 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2963?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2963: [HUDI-1904] Make SchemaProvider spark free and move it to hudi-client-common

2021-06-04 Thread GitBox
codecov-commenter edited a comment on pull request #2963: URL: https://github.com/apache/hudi/pull/2963#issuecomment-843155329 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[hudi] branch master updated: [HUDI-1954] only reset bucket when flush bucket success (#3029)

2021-06-04 Thread garyli
This is an automated email from the ASF dual-hosted git repository. garyli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new c4a2ad2 [HUDI-1954] only reset bucket when flush

[GitHub] [hudi] garyli1019 merged pull request #3029: [HUDI-1954] Only reset bucket when flush bucket success

2021-06-04 Thread GitBox
garyli1019 merged pull request #3029: URL: https://github.com/apache/hudi/pull/3029 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3018: [HUDI-1281] Add deltacommit to ActionType

2021-06-04 Thread GitBox
codecov-commenter edited a comment on pull request #3018: URL: https://github.com/apache/hudi/pull/3018#issuecomment-851564696 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2963: [HUDI-1904] Make SchemaProvider spark free and move it to hudi-client-common

2021-06-04 Thread GitBox
codecov-commenter edited a comment on pull request #2963: URL: https://github.com/apache/hudi/pull/2963#issuecomment-843155329 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2963?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2963: [HUDI-1904] Make SchemaProvider spark free and move it to hudi-client-common

2021-06-04 Thread GitBox
codecov-commenter edited a comment on pull request #2963: URL: https://github.com/apache/hudi/pull/2963#issuecomment-843155329 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2963?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] wangxianghu commented on pull request #2993: [HUDI-1929] Support configure KeyGenerator by type

2021-06-04 Thread GitBox
wangxianghu commented on pull request #2993: URL: https://github.com/apache/hudi/pull/2993#issuecomment-855174332 @nsivabalan this pr is ready now, please take a look when free -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] wangxianghu commented on a change in pull request #2963: [HUDI-1904] Make SchemaProvider spark free and move it to hudi-client-common

2021-06-04 Thread GitBox
wangxianghu commented on a change in pull request #2963: URL: https://github.com/apache/hudi/pull/2963#discussion_r645932010 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/schema/SchemaProvider.java ## @@ -34,18 +32,9 @@ @PublicAPIClass(maturity =

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3029: [HUDI-1954] Only reset bucket when flush bucket success

2021-06-04 Thread GitBox
codecov-commenter edited a comment on pull request #3029: URL: https://github.com/apache/hudi/pull/3029#issuecomment-853731624 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3029?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] vinothchandar commented on pull request #3018: [HUDI-1281] Add deltacommit to ActionType

2021-06-04 Thread GitBox
vinothchandar commented on pull request #3018: URL: https://github.com/apache/hudi/pull/3018#issuecomment-855172998 cc @prashantwason as FYI . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [hudi] vinothchandar commented on pull request #3018: [HUDI-1281] Add deltacommit to ActionType

2021-06-04 Thread GitBox
vinothchandar commented on pull request #3018: URL: https://github.com/apache/hudi/pull/3018#issuecomment-855172971 LGTM. Re-kicked the CI. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [hudi] garyli1019 commented on a change in pull request #3026: [HUDI-1931] BucketAssignFunction use ValueState instead of MapState

2021-06-04 Thread GitBox
garyli1019 commented on a change in pull request #3026: URL: https://github.com/apache/hudi/pull/3026#discussion_r645929404 ## File path: hudi-flink/src/main/java/org/apache/hudi/sink/partitioner/BucketAssignFunction.java ## @@ -138,7 +139,7 @@ public void open(Configuration p

[GitHub] [hudi] wangxianghu commented on a change in pull request #2963: [HUDI-1904] Make SchemaProvider spark free and move it to hudi-client-common

2021-06-04 Thread GitBox
wangxianghu commented on a change in pull request #2963: URL: https://github.com/apache/hudi/pull/2963#discussion_r645924017 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/schema/SchemaProvider.java ## @@ -34,18 +32,9 @@ @PublicAPIClass(maturity =

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2993: [HUDI-1929] Support configure KeyGenerator by type

2021-06-04 Thread GitBox
codecov-commenter edited a comment on pull request #2993: URL: https://github.com/apache/hudi/pull/2993#issuecomment-848384059 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2993?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2993: [HUDI-1929] Support configure KeyGenerator by type

2021-06-04 Thread GitBox
codecov-commenter edited a comment on pull request #2993: URL: https://github.com/apache/hudi/pull/2993#issuecomment-848384059 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2993?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2993: [HUDI-1929] Support configure KeyGenerator by type

2021-06-04 Thread GitBox
codecov-commenter edited a comment on pull request #2993: URL: https://github.com/apache/hudi/pull/2993#issuecomment-848384059 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2993?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] wangxianghu commented on a change in pull request #2963: [HUDI-1904] Make SchemaProvider spark free and move it to hudi-client-common

2021-06-04 Thread GitBox
wangxianghu commented on a change in pull request #2963: URL: https://github.com/apache/hudi/pull/2963#discussion_r645925416 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/schema/SchemaProvider.java ## @@ -34,18 +32,9 @@ @PublicAPIClass(maturity =

[GitHub] [hudi] wangxianghu commented on a change in pull request #2963: [HUDI-1904] Make SchemaProvider spark free and move it to hudi-client-common

2021-06-04 Thread GitBox
wangxianghu commented on a change in pull request #2963: URL: https://github.com/apache/hudi/pull/2963#discussion_r645925416 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/schema/SchemaProvider.java ## @@ -34,18 +32,9 @@ @PublicAPIClass(maturity =

[GitHub] [hudi] wangxianghu commented on a change in pull request #2963: [HUDI-1904] Make SchemaProvider spark free and move it to hudi-client-common

2021-06-04 Thread GitBox
wangxianghu commented on a change in pull request #2963: URL: https://github.com/apache/hudi/pull/2963#discussion_r645925416 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/schema/SchemaProvider.java ## @@ -34,18 +32,9 @@ @PublicAPIClass(maturity =

[GitHub] [hudi] wangxianghu commented on a change in pull request #2963: [HUDI-1904] Make SchemaProvider spark free and move it to hudi-client-common

2021-06-04 Thread GitBox
wangxianghu commented on a change in pull request #2963: URL: https://github.com/apache/hudi/pull/2963#discussion_r645924017 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/schema/SchemaProvider.java ## @@ -34,18 +32,9 @@ @PublicAPIClass(maturity =

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2993: [HUDI-1929] Support configure KeyGenerator by type

2021-06-04 Thread GitBox
codecov-commenter edited a comment on pull request #2993: URL: https://github.com/apache/hudi/pull/2993#issuecomment-848384059 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2993?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2993: [HUDI-1929] Support configure KeyGenerator by type

2021-06-04 Thread GitBox
codecov-commenter edited a comment on pull request #2993: URL: https://github.com/apache/hudi/pull/2993#issuecomment-848384059 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2993?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2833: [HUDI-89] Add configOption & refactor Hudi configuration framework

2021-06-04 Thread GitBox
codecov-commenter edited a comment on pull request #2833: URL: https://github.com/apache/hudi/pull/2833#issuecomment-828792354 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2833?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2833: [HUDI-89] Add configOption & refactor Hudi configuration framework

2021-06-04 Thread GitBox
codecov-commenter edited a comment on pull request #2833: URL: https://github.com/apache/hudi/pull/2833#issuecomment-828792354 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2833?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2833: [HUDI-89] Add configOption & refactor Hudi configuration framework

2021-06-04 Thread GitBox
codecov-commenter edited a comment on pull request #2833: URL: https://github.com/apache/hudi/pull/2833#issuecomment-828792354 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2833?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] vinothchandar commented on a change in pull request #2833: [HUDI-89] Add configOption & refactor Hudi configuration framework

2021-06-04 Thread GitBox
vinothchandar commented on a change in pull request #2833: URL: https://github.com/apache/hudi/pull/2833#discussion_r645869201 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/lock/LockManager.java ## @@ -61,8 +61,8 @@ public void l

[GitHub] [hudi] vinothchandar commented on a change in pull request #2833: [HUDI-89] Add configOption & refactor Hudi configuration framework

2021-06-04 Thread GitBox
vinothchandar commented on a change in pull request #2833: URL: https://github.com/apache/hudi/pull/2833#discussion_r645552783 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieHBaseIndexConfig.java ## @@ -18,107 +18,133 @@ package org.

[GitHub] [hudi] prashantwason commented on a change in pull request #2999: [HUDI-764] [HUDI-765] Orc reader writer Implementation

2021-06-04 Thread GitBox
prashantwason commented on a change in pull request #2999: URL: https://github.com/apache/hudi/pull/2999#discussion_r645677619 ## File path: hudi-common/src/test/java/org/apache/hudi/io/storage/TestHoodieOrcReader.java ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Softwa

[GitHub] [hudi] hudi-bot edited a comment on pull request #2984: (Azure CI) test PR

2021-06-04 Thread GitBox
hudi-bot edited a comment on pull request #2984: URL: https://github.com/apache/hudi/pull/2984#issuecomment-846794102 ## CI report: * 24baac9967e7f0d3e1c814e41583092068156146 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[hudi] branch master updated (a658328 -> d02c0e5)

2021-06-04 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from a658328 [HUDI-1961] Add a debezium json integration test case for flink (#3030) add d02c0e5 [MINOR] Resolve

[GitHub] [hudi] nsivabalan merged pull request #3034: [MINOR] Resolve build issue arising from inaccessible pentaho jar

2021-06-04 Thread GitBox
nsivabalan merged pull request #3034: URL: https://github.com/apache/hudi/pull/3034 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [hudi] nsivabalan closed issue #160: Failure to find org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde

2021-06-04 Thread GitBox
nsivabalan closed issue #160: URL: https://github.com/apache/hudi/issues/160 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contac

[GitHub] [hudi] n3nash commented on a change in pull request #2542: [WIP] Global consistency

2021-06-04 Thread GitBox
n3nash commented on a change in pull request #2542: URL: https://github.com/apache/hudi/pull/2542#discussion_r645800278 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/InputPathHandler.java ## @@ -50,20 +59,22 @@ private final Map> groupedIncrementalPaths;

[GitHub] [hudi] n3nash commented on a change in pull request #2542: [WIP] Global consistency

2021-06-04 Thread GitBox
n3nash commented on a change in pull request #2542: URL: https://github.com/apache/hudi/pull/2542#discussion_r645800278 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/InputPathHandler.java ## @@ -50,20 +59,22 @@ private final Map> groupedIncrementalPaths;

[GitHub] [hudi] n3nash commented on a change in pull request #2542: [WIP] Global consistency

2021-06-04 Thread GitBox
n3nash commented on a change in pull request #2542: URL: https://github.com/apache/hudi/pull/2542#discussion_r645799116 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableMetaClient.java ## @@ -96,9 +96,9 @@ private HoodieArchivedTimeline archiv

[GitHub] [hudi] n3nash commented on a change in pull request #2542: [WIP] Global consistency

2021-06-04 Thread GitBox
n3nash commented on a change in pull request #2542: URL: https://github.com/apache/hudi/pull/2542#discussion_r645798650 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/InputPathHandler.java ## @@ -50,20 +59,22 @@ private final Map> groupedIncrementalPaths;

[GitHub] [hudi] n3nash commented on a change in pull request #2542: [WIP] Global consistency

2021-06-04 Thread GitBox
n3nash commented on a change in pull request #2542: URL: https://github.com/apache/hudi/pull/2542#discussion_r645797595 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableGloballyConsistentMetaClient.java ## @@ -0,0 +1,114 @@ +/* + * Licensed to th

[GitHub] [hudi] vinothchandar commented on issue #2905: Website redesign

2021-06-04 Thread GitBox
vinothchandar commented on issue #2905: URL: https://github.com/apache/hudi/issues/2905#issuecomment-854946774 The new nav makes sense! How do we proceed? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hudi] hudi-bot edited a comment on pull request #2984: (Azure CI) test PR

2021-06-04 Thread GitBox
hudi-bot edited a comment on pull request #2984: URL: https://github.com/apache/hudi/pull/2984#issuecomment-846794102 ## CI report: * 310f5de82e1cdd52fd1c3bbdd2f6496bd6890415 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot edited a comment on pull request #2984: (Azure CI) test PR

2021-06-04 Thread GitBox
hudi-bot edited a comment on pull request #2984: URL: https://github.com/apache/hudi/pull/2984#issuecomment-846794102 ## CI report: * bd3a9dd87c5db7cf54334dc399a38694505e93db Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot edited a comment on pull request #2984: (Azure CI) test PR

2021-06-04 Thread GitBox
hudi-bot edited a comment on pull request #2984: URL: https://github.com/apache/hudi/pull/2984#issuecomment-846794102 ## CI report: * bd3a9dd87c5db7cf54334dc399a38694505e93db Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

  1   2   3   >