[jira] [Updated] (HUDI-2066) Fix flaky test: HoodieSparkSqlWriterSuite.test schema evolution

2021-06-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2066: -- Status: In Progress (was: Open) > Fix flaky test: HoodieSparkSqlWriterSuite.test schema

[jira] [Updated] (HUDI-2066) Fix flaky test: HoodieSparkSqlWriterSuite.test schema evolution

2021-06-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2066: -- Fix Version/s: 0.9.0 > Fix flaky test: HoodieSparkSqlWriterSuite.test schema evolution >

[jira] [Reopened] (HUDI-2066) Fix flaky test: HoodieSparkSqlWriterSuite.test schema evolution

2021-06-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reopened HUDI-2066: --- > Fix flaky test: HoodieSparkSqlWriterSuite.test schema evolution > --

[jira] [Updated] (HUDI-2066) Fix flaky test: HoodieSparkSqlWriterSuite.test schema evolution

2021-06-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2066: -- Status: Patch Available (was: In Progress) > Fix flaky test: HoodieSparkSqlWriterSuite.

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3100: [HUDI-2034] Support explicit partition compaction strategy for flink …

2021-06-24 Thread GitBox
codecov-commenter edited a comment on pull request #3100: URL: https://github.com/apache/hudi/pull/3100#issuecomment-862906759 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3100?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] nsivabalan commented on pull request #3134: [HUDI-2052] Support load logFile in BootstrapFunction

2021-06-24 Thread GitBox
nsivabalan commented on pull request #3134: URL: https://github.com/apache/hudi/pull/3134#issuecomment-867786135 There are some checkstyle issues reported by CI. Can you please look into fixing -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [hudi] nsivabalan commented on pull request #3128: [HUDI-2029] Implement compression for DiskBasedMap in Spillable Map

2021-06-24 Thread GitBox
nsivabalan commented on pull request #3128: URL: https://github.com/apache/hudi/pull/3128#issuecomment-867784936 Looks like there are some checkstyle issues. Can you please look into it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [hudi] nsivabalan commented on pull request #3120: [HUDI-2045] Support Read Hoodie As DataSource Table For Flink And Del…

2021-06-24 Thread GitBox
nsivabalan commented on pull request #3120: URL: https://github.com/apache/hudi/pull/3120#issuecomment-867783069 @pengzhiwei2018 : Looks like an integ test is failing. Can you please check that out ITTestHoodieDemo.testParquetDemo -- This is an automated message from the Apache G

[GitHub] [hudi] nsivabalan commented on pull request #3108: [HUDI-2042] Compare the field object directly in OverwriteWithLatestA…

2021-06-24 Thread GitBox
nsivabalan commented on pull request #3108: URL: https://github.com/apache/hudi/pull/3108#issuecomment-867775132 Looks like existing tests are failing. Can please you fix those. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [hudi] sbernauer commented on a change in pull request #3111: [HUDI-2069] Fix KafkaAvroSchemaDeserializer to not rely on reflection

2021-06-24 Thread GitBox
sbernauer commented on a change in pull request #3111: URL: https://github.com/apache/hudi/pull/3111#discussion_r658093617 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/KafkaOffsetGen.java ## @@ -176,7 +176,9 @@ public KafkaOffsetGen(Type

[GitHub] [hudi] sbernauer commented on a change in pull request #3111: [HUDI-2069] Fix KafkaAvroSchemaDeserializer to not rely on reflection

2021-06-24 Thread GitBox
sbernauer commented on a change in pull request #3111: URL: https://github.com/apache/hudi/pull/3111#discussion_r658090982 ## File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala ## @@ -382,8 +382,6 @@ object DataSourceWrite

[GitHub] [hudi] sbernauer commented on a change in pull request #3111: [HUDI-2069] Fix KafkaAvroSchemaDeserializer to not rely on reflection

2021-06-24 Thread GitBox
sbernauer commented on a change in pull request #3111: URL: https://github.com/apache/hudi/pull/3111#discussion_r658090982 ## File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala ## @@ -382,8 +382,6 @@ object DataSourceWrite

[GitHub] [hudi] hudi-bot edited a comment on pull request #2542: Add ability to provide multi-region (global) data consistency across HMS in different regions

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #2542: URL: https://github.com/apache/hudi/pull/2542#issuecomment-861653385 ## CI report: * 436fadc5b76fce640dd8f71764fbfad32de758d9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3134: [HUDI-2052] Support load logFile in BootstrapFunction

2021-06-24 Thread GitBox
codecov-commenter edited a comment on pull request #3134: URL: https://github.com/apache/hudi/pull/3134#issuecomment-866110119 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3134?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] nsivabalan commented on a change in pull request #3111: [HUDI-2069] Fix KafkaAvroSchemaDeserializer to not rely on reflection

2021-06-24 Thread GitBox
nsivabalan commented on a change in pull request #3111: URL: https://github.com/apache/hudi/pull/3111#discussion_r658063310 ## File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala ## @@ -382,8 +382,6 @@ object DataSourceWrit

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3097: [HUDI-1996] Adding functionality to allow the providing of basic auth creds for confluent cloud schema registry

2021-06-24 Thread GitBox
codecov-commenter edited a comment on pull request #3097: URL: https://github.com/apache/hudi/pull/3097#issuecomment-862839067 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3097?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] vinothchandar commented on a change in pull request #3111: [HUDI-2069] Fix KafkaAvroSchemaDeserializer to not rely on reflection

2021-06-24 Thread GitBox
vinothchandar commented on a change in pull request #3111: URL: https://github.com/apache/hudi/pull/3111#discussion_r658037497 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/KafkaOffsetGen.java ## @@ -176,7 +176,9 @@ public KafkaOffsetGen(

[GitHub] [hudi] hudi-bot edited a comment on pull request #2542: Add ability to provide multi-region (global) data consistency across HMS in different regions

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #2542: URL: https://github.com/apache/hudi/pull/2542#issuecomment-861653385 ## CI report: * 77e1ba1df83a0315c197120f36b3910ba45072b7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] vinothchandar commented on a change in pull request #3111: [HUDI-2069] Fix KafkaAvroSchemaDeserializer to not rely on reflection

2021-06-24 Thread GitBox
vinothchandar commented on a change in pull request #3111: URL: https://github.com/apache/hudi/pull/3111#discussion_r658034007 ## File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala ## @@ -382,8 +382,6 @@ object DataSourceW

[GitHub] [hudi] hudi-bot edited a comment on pull request #3097: [HUDI-1996] Adding functionality to allow the providing of basic auth creds for confluent cloud schema registry

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #3097: URL: https://github.com/apache/hudi/pull/3097#issuecomment-862822918 ## CI report: * 5cfa5f10483f81338814037045dc8974cd3a50f8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] hudi-bot edited a comment on pull request #2542: Add ability to provide multi-region (global) data consistency across HMS in different regions

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #2542: URL: https://github.com/apache/hudi/pull/2542#issuecomment-861653385 ## CI report: * 77e1ba1df83a0315c197120f36b3910ba45072b7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] veenaypatil edited a comment on pull request #3136: [HUDI-2060] Added tests for KafkaOffsetGen

2021-06-24 Thread GitBox
veenaypatil edited a comment on pull request #3136: URL: https://github.com/apache/hudi/pull/3136#issuecomment-867543880 @vinothchandar added tests for multiple partitions > adding a new partition between two fetches and seeing that the new partition is picked up not able to

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3149: [HUDI-1104] Adding support for UserDefinedPartitioners and SortModes to BulkInsert with Rows

2021-06-24 Thread GitBox
codecov-commenter edited a comment on pull request #3149: URL: https://github.com/apache/hudi/pull/3149#issuecomment-867680176 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3149?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3139: [HUDI-2058]support incremental query for insert_overwrite_table/insert_overwrite operation on cow table

2021-06-24 Thread GitBox
codecov-commenter edited a comment on pull request #3139: URL: https://github.com/apache/hudi/pull/3139#issuecomment-866741631 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3148: [HUDI-2068] Skip the assign state for SmallFileAssign when the state …

2021-06-24 Thread GitBox
codecov-commenter edited a comment on pull request #3148: URL: https://github.com/apache/hudi/pull/3148#issuecomment-867652777 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3148?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3148: [HUDI-2068] Skip the assign state for SmallFileAssign when the state …

2021-06-24 Thread GitBox
codecov-commenter edited a comment on pull request #3148: URL: https://github.com/apache/hudi/pull/3148#issuecomment-867652777 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3136: [HUDI-2060] Added tests for KafkaOffsetGen

2021-06-24 Thread GitBox
codecov-commenter edited a comment on pull request #3136: URL: https://github.com/apache/hudi/pull/3136#issuecomment-866609218 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3139: [HUDI-2058]support incremental query for insert_overwrite_table/insert_overwrite operation on cow table

2021-06-24 Thread GitBox
codecov-commenter edited a comment on pull request #3139: URL: https://github.com/apache/hudi/pull/3139#issuecomment-866741631 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3139?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3148: [HUDI-2068] Skip the assign state for SmallFileAssign when the state …

2021-06-24 Thread GitBox
codecov-commenter edited a comment on pull request #3148: URL: https://github.com/apache/hudi/pull/3148#issuecomment-867652777 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3148?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3136: [HUDI-2060] Added tests for KafkaOffsetGen

2021-06-24 Thread GitBox
codecov-commenter edited a comment on pull request #3136: URL: https://github.com/apache/hudi/pull/3136#issuecomment-866609218 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3136?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] sbernauer commented on pull request #3111: [HUDI-2069] Fix KafkaAvroSchemaDeserializer to not rely on reflection

2021-06-24 Thread GitBox
sbernauer commented on pull request #3111: URL: https://github.com/apache/hudi/pull/3111#issuecomment-867681473 Many thanks @nsivabalan! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[GitHub] [hudi] codecov-commenter commented on pull request #3149: [HUDI-1104] Adding support for UserDefinedPartitioners and SortModes to BulkInsert with Rows

2021-06-24 Thread GitBox
codecov-commenter commented on pull request #3149: URL: https://github.com/apache/hudi/pull/3149#issuecomment-867680176 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3149?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache

[GitHub] [hudi] hudi-bot edited a comment on pull request #3134: [HUDI-2052] Support load logFile in BootstrapFunction

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #3134: URL: https://github.com/apache/hudi/pull/3134#issuecomment-865885733 ## CI report: * b428ad50cd2a09bd5b9201acbd007e3a155cfc69 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] hudi-bot edited a comment on pull request #3097: [HUDI-1996] Adding functionality to allow the providing of basic auth creds for confluent cloud schema registry

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #3097: URL: https://github.com/apache/hudi/pull/3097#issuecomment-862822918 ## CI report: * 3e3d2c2b01fbfbf0d1906d647187cb8313f3ca94 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot edited a comment on pull request #3097: [HUDI-1996] Adding functionality to allow the providing of basic auth creds for confluent cloud schema registry

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #3097: URL: https://github.com/apache/hudi/pull/3097#issuecomment-862822918 ## CI report: * 3e3d2c2b01fbfbf0d1906d647187cb8313f3ca94 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] leesf commented on a change in pull request #3140: [HUDI-2063] Add Doc For Spark Sql Integrates With Hudi

2021-06-24 Thread GitBox
leesf commented on a change in pull request #3140: URL: https://github.com/apache/hudi/pull/3140#discussion_r657974442 ## File path: docs/_docs/1_1_spark_quick_start_guide.md ## @@ -300,6 +300,221 @@ spark. show(100, false) ``` +# Spark-Sql example +## Setup +Hudi support

[GitHub] [hudi] leesf commented on a change in pull request #3140: [HUDI-2063] Add Doc For Spark Sql Integrates With Hudi

2021-06-24 Thread GitBox
leesf commented on a change in pull request #3140: URL: https://github.com/apache/hudi/pull/3140#discussion_r657973304 ## File path: docs/_docs/1_1_spark_quick_start_guide.md ## @@ -300,6 +300,221 @@ spark. show(100, false) ``` +# Spark-Sql example +## Setup +Hudi support

[GitHub] [hudi] leesf commented on a change in pull request #3140: [HUDI-2063] Add Doc For Spark Sql Integrates With Hudi

2021-06-24 Thread GitBox
leesf commented on a change in pull request #3140: URL: https://github.com/apache/hudi/pull/3140#discussion_r657972394 ## File path: docs/_docs/1_1_spark_quick_start_guide.md ## @@ -300,6 +300,221 @@ spark. show(100, false) ``` +# Spark-Sql example +## Setup +Hudi support

[GitHub] [hudi] leesf commented on a change in pull request #3140: [HUDI-2063] Add Doc For Spark Sql Integrates With Hudi

2021-06-24 Thread GitBox
leesf commented on a change in pull request #3140: URL: https://github.com/apache/hudi/pull/3140#discussion_r657971685 ## File path: docs/_docs/1_1_spark_quick_start_guide.md ## @@ -300,6 +300,221 @@ spark. show(100, false) ``` +# Spark-Sql example +## Setup +Hudi support

[GitHub] [hudi] hudi-bot edited a comment on pull request #3097: [HUDI-1996] Adding functionality to allow the providing of basic auth creds for confluent cloud schema registry

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #3097: URL: https://github.com/apache/hudi/pull/3097#issuecomment-862822918 ## CI report: * 686539191a516b704e43e4a5edc1d3e353285371 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3139: [HUDI-2058]support incremental query for insert_overwrite_table/insert_overwrite operation on cow table

2021-06-24 Thread GitBox
codecov-commenter edited a comment on pull request #3139: URL: https://github.com/apache/hudi/pull/3139#issuecomment-866741631 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3139?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] yuzhaojing commented on pull request #3134: [HUDI-2052] Support load logFile in BootstrapFunction

2021-06-24 Thread GitBox
yuzhaojing commented on pull request #3134: URL: https://github.com/apache/hudi/pull/3134#issuecomment-867652914 Already modify. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[GitHub] [hudi] codecov-commenter commented on pull request #3148: [HUDI-2068] Skip the assign state for SmallFileAssign when the state …

2021-06-24 Thread GitBox
codecov-commenter commented on pull request #3148: URL: https://github.com/apache/hudi/pull/3148#issuecomment-867652777 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3148?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache

[GitHub] [hudi] hudi-bot edited a comment on pull request #3134: [HUDI-2052] Support load logFile in BootstrapFunction

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #3134: URL: https://github.com/apache/hudi/pull/3134#issuecomment-865885733 ## CI report: * 56a9c154b84601205b7728b821e98ed6a474b89a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3136: [HUDI-2060] Added tests for KafkaOffsetGen

2021-06-24 Thread GitBox
codecov-commenter edited a comment on pull request #3136: URL: https://github.com/apache/hudi/pull/3136#issuecomment-866609218 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3136?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] yuzhaojing commented on pull request #3138: [HUDI-2062] Catch FileNotFoundException in WriteProfiles #getCommitMetadataSafely

2021-06-24 Thread GitBox
yuzhaojing commented on pull request #3138: URL: https://github.com/apache/hudi/pull/3138#issuecomment-867650673 > @yuzhaojing The flaky test cases seem to be fixed yesterday. You can try to `pull --rebase` with the master to see if we can make the CI pass. WDYT? Ok, CI is passed.

[GitHub] [hudi] hudi-bot edited a comment on pull request #3149: [HUDI-1104] Adding support for UserDefinedPartitioners and SortModes to BulkInsert with Rows

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #3149: URL: https://github.com/apache/hudi/pull/3149#issuecomment-867609021 ## CI report: * b12f4279ca83456ec0517792cfb568f39d48a5e8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] hudi-bot edited a comment on pull request #3134: [HUDI-2052] Support load logFile in BootstrapFunction

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #3134: URL: https://github.com/apache/hudi/pull/3134#issuecomment-865885733 ## CI report: * 56a9c154b84601205b7728b821e98ed6a474b89a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[jira] [Commented] (HUDI-1248) [UMBRELLA] Tests cleanup and fixes

2021-06-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368857#comment-17368857 ] sivabalan narayanan commented on HUDI-1248: ---   TestCleaner.testKeepLatestFileVe

[GitHub] [hudi] nsivabalan commented on a change in pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2021-06-24 Thread GitBox
nsivabalan commented on a change in pull request #2438: URL: https://github.com/apache/hudi/pull/2438#discussion_r657945070 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/DeltaSync.java ## @@ -312,13 +313,13 @@ public void refreshTimeline()

[GitHub] [hudi] hudi-bot edited a comment on pull request #3097: [HUDI-1996] Adding functionality to allow the providing of basic auth creds for confluent cloud schema registry

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #3097: URL: https://github.com/apache/hudi/pull/3097#issuecomment-862822918 ## CI report: * 686539191a516b704e43e4a5edc1d3e353285371 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] hudi-bot edited a comment on pull request #3142: [HUDI-1483] Support async clustering for deltastreamer and Spark streaming

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #3142: URL: https://github.com/apache/hudi/pull/3142#issuecomment-866996072 ## CI report: * 4a651f8d6b32f63838d8317c9f083508c17e4458 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] hudi-bot edited a comment on pull request #3097: [HUDI-1996] Adding functionality to allow the providing of basic auth creds for confluent cloud schema registry

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #3097: URL: https://github.com/apache/hudi/pull/3097#issuecomment-862822918 ## CI report: * 686539191a516b704e43e4a5edc1d3e353285371 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[jira] [Commented] (HUDI-1248) [UMBRELLA] Tests cleanup and fixes

2021-06-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368838#comment-17368838 ] sivabalan narayanan commented on HUDI-1248: --- Formatting using clusterid: testClu

[GitHub] [hudi] hudi-bot edited a comment on pull request #3142: [HUDI-1483] Support async clustering for deltastreamer and Spark streaming

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #3142: URL: https://github.com/apache/hudi/pull/3142#issuecomment-866996072 ## CI report: * 4a651f8d6b32f63838d8317c9f083508c17e4458 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] nsivabalan commented on pull request #3142: [HUDI-1483] Support async clustering for deltastreamer and Spark streaming

2021-06-24 Thread GitBox
nsivabalan commented on pull request #3142: URL: https://github.com/apache/hudi/pull/3142#issuecomment-867624272 @hudi-bot run travis -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[jira] [Updated] (HUDI-2069) Fix KafkaAvroSchemaDeserializer to not rely on reflection

2021-06-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2069: - Labels: pull-request-available (was: ) > Fix KafkaAvroSchemaDeserializer to not rely on reflectio

[hudi] branch master updated (84dd3ca -> b328555)

2021-06-24 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 84dd3ca [HUDI-2053] Insert Static Partition With DateType Return Incorrect Partition Value (#3133) add b3285

[GitHub] [hudi] nsivabalan merged pull request #3111: [HUDI-2069] Fix KafkaAvroSchemaDeserializer to not rely on reflection

2021-06-24 Thread GitBox
nsivabalan merged pull request #3111: URL: https://github.com/apache/hudi/pull/3111 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [hudi] nsivabalan commented on pull request #3111: [HUDI-2069] Fix KafkaAvroSchemaDeserializer to not rely on reflection

2021-06-24 Thread GitBox
nsivabalan commented on pull request #3111: URL: https://github.com/apache/hudi/pull/3111#issuecomment-867622402 We always have a practice of creating a jira for every patch. especially bugs. I have created one and prefixed the title. Will fix the commit msg as well. Something to remember

[jira] [Created] (HUDI-2069) Fix KafkaAvroSchemaDeserializer to not rely on reflection

2021-06-24 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-2069: - Summary: Fix KafkaAvroSchemaDeserializer to not rely on reflection Key: HUDI-2069 URL: https://issues.apache.org/jira/browse/HUDI-2069 Project: Apache Hudi

[GitHub] [hudi] hudi-bot edited a comment on pull request #3142: [HUDI-1483] Support async clustering for deltastreamer and Spark streaming

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #3142: URL: https://github.com/apache/hudi/pull/3142#issuecomment-866996072 ## CI report: * 4a651f8d6b32f63838d8317c9f083508c17e4458 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] nsivabalan commented on pull request #3142: [HUDI-1483] Support async clustering for deltastreamer and Spark streaming

2021-06-24 Thread GitBox
nsivabalan commented on pull request #3142: URL: https://github.com/apache/hudi/pull/3142#issuecomment-867615903 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] hudi-bot edited a comment on pull request #3149: [HUDI-1104] Adding support for UserDefinedPartitioners and SortModes to BulkInsert with Rows

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #3149: URL: https://github.com/apache/hudi/pull/3149#issuecomment-867609021 ## CI report: * b12f4279ca83456ec0517792cfb568f39d48a5e8 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] hudi-bot commented on pull request #3149: [HUDI-1104] Adding support for UserDefinedPartitioners and SortModes to BulkInsert with Rows

2021-06-24 Thread GitBox
hudi-bot commented on pull request #3149: URL: https://github.com/apache/hudi/pull/3149#issuecomment-867609021 ## CI report: * b12f4279ca83456ec0517792cfb568f39d48a5e8 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[GitHub] [hudi] nsivabalan commented on pull request #2049: [HUDI-1104] Adding support for UserDefinedPartitioners and SortModes to BulkInsert with Rows

2021-06-24 Thread GitBox
nsivabalan commented on pull request #2049: URL: https://github.com/apache/hudi/pull/2049#issuecomment-867606917 Closing this in favor of https://github.com/apache/hudi/pull/3149 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[GitHub] [hudi] nsivabalan closed pull request #2049: [HUDI-1104] Adding support for UserDefinedPartitioners and SortModes to BulkInsert with Rows

2021-06-24 Thread GitBox
nsivabalan closed pull request #2049: URL: https://github.com/apache/hudi/pull/2049 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [hudi] hudi-bot edited a comment on pull request #3139: [HUDI-2058]support incremental query for insert_overwrite_table/insert_overwrite operation on cow table

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #3139: URL: https://github.com/apache/hudi/pull/3139#issuecomment-866671675 ## CI report: * ebd7ae14e0fafb413abdd4561361b2fbbd398473 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] nsivabalan opened a new pull request #3149: [HUDI-1104] Adding support for UserDefinedPartitioners and SortModes to BulkInsert with Rows

2021-06-24 Thread GitBox
nsivabalan opened a new pull request #3149: URL: https://github.com/apache/hudi/pull/3149 ## What is the purpose of the pull request - Adding bulk_insert sort modes for row writer path *(for example:)* - Adding bulk_insert sort modes for row writer path - Caching Ro

[GitHub] [hudi] yanghua commented on pull request #2963: [HUDI-1904] Introduce SchemaProviderInterface to make SchemaProvider unified

2021-06-24 Thread GitBox
yanghua commented on pull request #2963: URL: https://github.com/apache/hudi/pull/2963#issuecomment-867599363 Hi @nsivabalan Any input? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [hudi] hudi-bot edited a comment on pull request #3148: [HUDI-2068] Skip the assign state for SmallFileAssign when the state …

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #3148: URL: https://github.com/apache/hudi/pull/3148#issuecomment-867564985 ## CI report: * 0c77a46a83ed5ac646120fd34197d8d3a6f7ae5d Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3138: [HUDI-2062] Catch FileNotFoundException in WriteProfiles #getCommitMetadataSafely

2021-06-24 Thread GitBox
codecov-commenter edited a comment on pull request #3138: URL: https://github.com/apache/hudi/pull/3138#issuecomment-866723821 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3138?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3138: [HUDI-2062] Catch FileNotFoundException in WriteProfiles #getCommitMetadataSafely

2021-06-24 Thread GitBox
codecov-commenter edited a comment on pull request #3138: URL: https://github.com/apache/hudi/pull/3138#issuecomment-866723821 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3138?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3134: [HUDI-2052] Support load logFile in BootstrapFunction

2021-06-24 Thread GitBox
codecov-commenter edited a comment on pull request #3134: URL: https://github.com/apache/hudi/pull/3134#issuecomment-866110119 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3134?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] hudi-bot edited a comment on pull request #3139: [HUDI-2058]support incremental query for insert_overwrite_table/insert_overwrite operation on cow table

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #3139: URL: https://github.com/apache/hudi/pull/3139#issuecomment-866671675 ## CI report: * 29536de9ee87bbc207daefef84b80b23c52fca9d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] hudi-bot edited a comment on pull request #3139: [HUDI-2058]support incremental query for insert_overwrite_table/insert_overwrite operation on cow table

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #3139: URL: https://github.com/apache/hudi/pull/3139#issuecomment-866671675 ## CI report: * 29536de9ee87bbc207daefef84b80b23c52fca9d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] hudi-bot edited a comment on pull request #3136: [HUDI-2060] Added tests for KafkaOffsetGen

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #3136: URL: https://github.com/apache/hudi/pull/3136#issuecomment-866573989 ## CI report: * a7a189f06db5da90d577e6b4bf9dae1948e3fcac Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] hudi-bot edited a comment on pull request #3148: [HUDI-2068] Skip the assign state for SmallFileAssign when the state …

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #3148: URL: https://github.com/apache/hudi/pull/3148#issuecomment-867564985 ## CI report: * 0c77a46a83ed5ac646120fd34197d8d3a6f7ae5d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3138: [HUDI-2062] Catch FileNotFoundException in WriteProfiles #getCommitMetadataSafely

2021-06-24 Thread GitBox
codecov-commenter edited a comment on pull request #3138: URL: https://github.com/apache/hudi/pull/3138#issuecomment-866723821 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3138?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] danny0405 commented on a change in pull request #3134: [HUDI-2052] Support load logFile in BootstrapFunction

2021-06-24 Thread GitBox
danny0405 commented on a change in pull request #3134: URL: https://github.com/apache/hudi/pull/3134#discussion_r657867608 ## File path: hudi-flink/src/test/java/org/apache/hudi/sink/TestWriteMergeOnRead.java ## @@ -46,6 +50,11 @@ import java.util.Map; import java.util.strea

[GitHub] [hudi] danny0405 commented on a change in pull request #3134: [HUDI-2052] Support load logFile in BootstrapFunction

2021-06-24 Thread GitBox
danny0405 commented on a change in pull request #3134: URL: https://github.com/apache/hudi/pull/3134#discussion_r657866899 ## File path: hudi-flink/src/test/java/org/apache/hudi/sink/TestWriteMergeOnRead.java ## @@ -83,10 +92,14 @@ protected void checkWrittenData(File baseFile

[GitHub] [hudi] danny0405 commented on a change in pull request #3134: [HUDI-2052] Support load logFile in BootstrapFunction

2021-06-24 Thread GitBox
danny0405 commented on a change in pull request #3134: URL: https://github.com/apache/hudi/pull/3134#discussion_r657866899 ## File path: hudi-flink/src/test/java/org/apache/hudi/sink/TestWriteMergeOnRead.java ## @@ -83,10 +92,14 @@ protected void checkWrittenData(File baseFile

[GitHub] [hudi] hudi-bot commented on pull request #3148: [HUDI-2068] Skip the assign state for SmallFileAssign when the state …

2021-06-24 Thread GitBox
hudi-bot commented on pull request #3148: URL: https://github.com/apache/hudi/pull/3148#issuecomment-867564985 ## CI report: * 0c77a46a83ed5ac646120fd34197d8d3a6f7ae5d UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[jira] [Updated] (HUDI-2068) Skip the assign state for SmallFileAssign when the state can not assign initially

2021-06-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2068: - Labels: pull-request-available (was: ) > Skip the assign state for SmallFileAssign when the state

[GitHub] [hudi] danny0405 opened a new pull request #3148: [HUDI-2068] Skip the assign state for SmallFileAssign when the state …

2021-06-24 Thread GitBox
danny0405 opened a new pull request #3148: URL: https://github.com/apache/hudi/pull/3148 …can not assign initially ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.*

[jira] [Created] (HUDI-2068) Skip the assign state for SmallFileAssign when the state can not assign initially

2021-06-24 Thread Danny Chen (Jira)
Danny Chen created HUDI-2068: Summary: Skip the assign state for SmallFileAssign when the state can not assign initially Key: HUDI-2068 URL: https://issues.apache.org/jira/browse/HUDI-2068 Project: Apache

[GitHub] [hudi] hudi-bot edited a comment on pull request #3097: [HUDI-1996] Adding functionality to allow the providing of basic auth creds for confluent cloud schema registry

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #3097: URL: https://github.com/apache/hudi/pull/3097#issuecomment-862822918 ## CI report: * 686539191a516b704e43e4a5edc1d3e353285371 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] sbernauer commented on pull request #3111: Fix KafkaAvroSchemaDeserializer to not rely on reflection

2021-06-24 Thread GitBox
sbernauer commented on pull request #3111: URL: https://github.com/apache/hudi/pull/3111#issuecomment-867560326 So, the tests passed with the third attempt :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[jira] [Closed] (HUDI-1405) Make HoodieFlinkStreamer support read props from local fileSystem

2021-06-24 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-1405. -- Resolution: Invalid > Make HoodieFlinkStreamer support read props from local fileSystem > --

[jira] [Updated] (HUDI-1405) Make HoodieFlinkStreamer support read props from local fileSystem

2021-06-24 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang updated HUDI-1405: --- Status: Open (was: New) > Make HoodieFlinkStreamer support read props from local fileSystem > ---

[hudi] branch master updated: [HUDI-2053] Insert Static Partition With DateType Return Incorrect Partition Value (#3133)

2021-06-24 Thread leesf
This is an automated email from the ASF dual-hosted git repository. leesf pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 84dd3ca [HUDI-2053] Insert Static Partition With

[GitHub] [hudi] leesf merged pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…

2021-06-24 Thread GitBox
leesf merged pull request #3133: URL: https://github.com/apache/hudi/pull/3133 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please cont

[GitHub] [hudi] hudi-bot edited a comment on pull request #3136: [HUDI-2060] Added tests for KafkaOffsetGen

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #3136: URL: https://github.com/apache/hudi/pull/3136#issuecomment-866573989 ## CI report: * cffc8398380b361be89ccf1bdaaa5259a3cc5856 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] veenaypatil commented on pull request #3136: [HUDI-2060] Added tests for KafkaOffsetGen

2021-06-24 Thread GitBox
veenaypatil commented on pull request #3136: URL: https://github.com/apache/hudi/pull/3136#issuecomment-867543880 @vinothchandar added tests for multiple partitions > adding a new partition between two fetches and seeing that the new partition is picked up not able to add new par

[GitHub] [hudi] hudi-bot edited a comment on pull request #3136: [HUDI-2060] Added tests for KafkaOffsetGen

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #3136: URL: https://github.com/apache/hudi/pull/3136#issuecomment-866573989 ## CI report: * cffc8398380b361be89ccf1bdaaa5259a3cc5856 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…

2021-06-24 Thread GitBox
codecov-commenter edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865868967 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…

2021-06-24 Thread GitBox
codecov-commenter edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865868967 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3133?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3146: [HUDI-2057] CTAS Generate An External Table When Create Managed Table

2021-06-24 Thread GitBox
codecov-commenter edited a comment on pull request #3146: URL: https://github.com/apache/hudi/pull/3146#issuecomment-867510590 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [hudi] hudi-bot edited a comment on pull request #3138: [HUDI-2062] Catch FileNotFoundException in WriteProfiles #getCommitMetadataSafely

2021-06-24 Thread GitBox
hudi-bot edited a comment on pull request #3138: URL: https://github.com/apache/hudi/pull/3138#issuecomment-87254 ## CI report: * 4346314677d2969e441545f796aff2a37126fa49 UNKNOWN * caae0d7a3b5e5d7206ec36bbe2b919e96c150e7e Azure: [SUCCESS](https://dev.azure.com/apache-hudi-

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3146: [HUDI-2057] CTAS Generate An External Table When Create Managed Table

2021-06-24 Thread GitBox
codecov-commenter edited a comment on pull request #3146: URL: https://github.com/apache/hudi/pull/3146#issuecomment-867510590 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3146?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The

<    1   2   3   >