[GitHub] [hudi] jsbali commented on issue #2934: [SUPPORT] Parquet file does not exist when trying to read hudi table incrementally

2021-09-27 Thread GitBox
jsbali commented on issue #2934: URL: https://github.com/apache/hudi/issues/2934#issuecomment-928877634 @t0il3ts0ap Do raise the PR for the same if it is not too much work and we can let the Hudi folks decide what makes sense for Hudi. My changes are mostly in IncrementalRelation. Will

[GitHub] [hudi] hudi-bot edited a comment on pull request #3693: [HUDI-2456] support 'show partitions' sql

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3693: URL: https://github.com/apache/hudi/pull/3693#issuecomment-922600059 ## CI report: * 5f539d60537c7f2b04aecf10a8e217110d7e8025 Azure:

[jira] [Closed] (HUDI-2487) An empty message in Kafka causes a task exception

2021-09-27 Thread vinoyang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang closed HUDI-2487. -- Fix Version/s: (was: 0.9.0) 0.10.0 Resolution: Implemented

[hudi] branch master updated: [HUDI-2487] Fix JsonKafkaSource cannot filter empty messages from kafka (#3715)

2021-09-27 Thread vinoyang
This is an automated email from the ASF dual-hosted git repository. vinoyang pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 9067657 [HUDI-2487] Fix JsonKafkaSource cannot

[GitHub] [hudi] yanghua merged pull request #3715: [HUDI-2487] Fix JsonKafkaSource cannot filter empty messages from kafka

2021-09-27 Thread GitBox
yanghua merged pull request #3715: URL: https://github.com/apache/hudi/pull/3715 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] t0il3ts0ap edited a comment on issue #2934: [SUPPORT] Parquet file does not exist when trying to read hudi table incrementally

2021-09-27 Thread GitBox
t0il3ts0ap edited a comment on issue #2934: URL: https://github.com/apache/hudi/issues/2934#issuecomment-928866105 I have also worked on the same changes already. @vingov, @jsbali Let me know, if you have not yet started then I can raise a pr for this. -- This is an automated message

[GitHub] [hudi] t0il3ts0ap commented on issue #2934: [SUPPORT] Parquet file does not exist when trying to read hudi table incrementally

2021-09-27 Thread GitBox
t0il3ts0ap commented on issue #2934: URL: https://github.com/apache/hudi/issues/2934#issuecomment-928866105 I have also worked on the same changes already. @jsbali Let me know, if you have not yet started then I can raise a pr for this. -- This is an automated message from the Apache

[GitHub] [hudi] mkk1490 commented on issue #3313: [SUPPORT] CoW: Hudi Upsert not working when there is a timestamp field in the composite key

2021-09-27 Thread GitBox
mkk1490 commented on issue #3313: URL: https://github.com/apache/hudi/issues/3313#issuecomment-928853775 > @mkk1490 : sorry the issue got lengthy and I have got a couple of clarifications. > Is your issue is with record key fields having one component as timestamp or is it

[GitHub] [hudi] xushiyan commented on issue #3624: Failed to delete the partition table record

2021-09-27 Thread GitBox
xushiyan commented on issue #3624: URL: https://github.com/apache/hudi/issues/3624#issuecomment-928837505 @melin could you provide more version info like Hudi, Spark versions. also if it's on cloud or other environment? > The reason for this error is because a partition was

[GitHub] [hudi] bvaradar commented on a change in pull request #3668: [RFC-33] [HUDI-2429][WIP] Full schema evolution

2021-09-27 Thread GitBox
bvaradar commented on a change in pull request #3668: URL: https://github.com/apache/hudi/pull/3668#discussion_r717230330 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/AbstractHoodieWriteClient.java ## @@ -213,6 +224,27 @@ protected void

[GitHub] [hudi] bvaradar commented on a change in pull request #3668: [RFC-33] [HUDI-2429][WIP] Full schema evolution

2021-09-27 Thread GitBox
bvaradar commented on a change in pull request #3668: URL: https://github.com/apache/hudi/pull/3668#discussion_r717230330 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/AbstractHoodieWriteClient.java ## @@ -213,6 +224,27 @@ protected void

[GitHub] [hudi] hudi-bot edited a comment on pull request #3693: [HUDI-2456] support 'show partitions' sql

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3693: URL: https://github.com/apache/hudi/pull/3693#issuecomment-922600059 ## CI report: * ca20de655b6b2da2bdc3d31cde6f8f2579b407ae Azure:

[GitHub] [hudi] bvaradar commented on pull request #3668: [RFC-33] [HUDI-2429][WIP] Full schema evolution

2021-09-27 Thread GitBox
bvaradar commented on pull request #3668: URL: https://github.com/apache/hudi/pull/3668#issuecomment-928824868 @xiarixiaoyao : Can you add commits to this PR instead of squashing. It makes things easy for us to find the delta changes. We can do final squash before landing the PR. --

[jira] [Updated] (HUDI-2390) KeyGenerator discrepancy between DataFrame writer and SQL

2021-09-27 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2390: - Labels: sev:critical (was: ) > KeyGenerator discrepancy between DataFrame writer and SQL >

[jira] [Updated] (HUDI-2390) KeyGenerator discrepancy between DataFrame writer and SQL

2021-09-27 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2390: - Priority: Critical (was: Minor) > KeyGenerator discrepancy between DataFrame writer and SQL >

[GitHub] [hudi] danny0405 commented on pull request #3203: [HUDI-2086] Redo the logical of mor_incremental_view for hive

2021-09-27 Thread GitBox
danny0405 commented on pull request #3203: URL: https://github.com/apache/hudi/pull/3203#issuecomment-928786962 Thanks, i will take a look tomorrow :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hudi] xushiyan edited a comment on pull request #3726: [MINOR] Add a RFC template and folder

2021-09-27 Thread GitBox
xushiyan edited a comment on pull request #3726: URL: https://github.com/apache/hudi/pull/3726#issuecomment-928184950 @vinothchandar ~RFCs should be kept in asf-site?~ ok just read the process PR and understood the idea. Yes looks easier to keep PRs and RFC docs updated altogether. And

[GitHub] [hudi] xushiyan edited a comment on pull request #3726: [MINOR] Add a RFC template and folder

2021-09-27 Thread GitBox
xushiyan edited a comment on pull request #3726: URL: https://github.com/apache/hudi/pull/3726#issuecomment-928184950 ~@vinothchandar RFCs should be kept in asf-site?~ Read the process PR and understood the idea. Yes looks easier to keep PRs and RFC docs updated altogether. And RFCs may

[GitHub] [hudi] xushiyan commented on a change in pull request #3725: [DOCS] New RFC Process

2021-09-27 Thread GitBox
xushiyan commented on a change in pull request #3725: URL: https://github.com/apache/hudi/pull/3725#discussion_r717211562 ## File path: website/contribute/rfc-process.md ## @@ -0,0 +1,56 @@ +--- +sidebar_position: 3 +title: "RFC Process" +toc: true +last_modified_at:

[GitHub] [hudi] xushiyan commented on a change in pull request #3725: [DOCS] New RFC Process

2021-09-27 Thread GitBox
xushiyan commented on a change in pull request #3725: URL: https://github.com/apache/hudi/pull/3725#discussion_r717208114 ## File path: website/contribute/rfc-process.md ## @@ -0,0 +1,56 @@ +--- +sidebar_position: 3 +title: "RFC Process" +toc: true +last_modified_at:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3693: [HUDI-2456] support 'show partitions' sql

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3693: URL: https://github.com/apache/hudi/pull/3693#issuecomment-922600059 ## CI report: * ca20de655b6b2da2bdc3d31cde6f8f2579b407ae Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3693: [HUDI-2456] support 'show partitions' sql

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3693: URL: https://github.com/apache/hudi/pull/3693#issuecomment-922600059 ## CI report: * 9ad6e667a40b60b1bbf129554dcc208827a18dd6 Azure:

[GitHub] [hudi] YannByron commented on a change in pull request #3693: [HUDI-2456] support 'show partitions' sql

2021-09-27 Thread GitBox
YannByron commented on a change in pull request #3693: URL: https://github.com/apache/hudi/pull/3693#discussion_r717206741 ## File path: hudi-common/src/main/java/org/apache/hudi/common/util/PartitionPathEncodeUtils.java ## @@ -71,7 +73,7 @@ public static String

[GitHub] [hudi] hudi-bot edited a comment on pull request #3693: [HUDI-2456] support 'show partitions' sql

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3693: URL: https://github.com/apache/hudi/pull/3693#issuecomment-922600059 ## CI report: * 9ad6e667a40b60b1bbf129554dcc208827a18dd6 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3693: [HUDI-2456] support 'show partitions' sql

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3693: URL: https://github.com/apache/hudi/pull/3693#issuecomment-922600059 ## CI report: * 9ad6e667a40b60b1bbf129554dcc208827a18dd6 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3203: [HUDI-2086] Redo the logical of mor_incremental_view for hive

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3203: URL: https://github.com/apache/hudi/pull/3203#issuecomment-872092745 ## CI report: * 0fa6297ce58eb877fd5c4eba59fef20ad9335d26 UNKNOWN * 423f047e00904b17735a5f0120959de87e4cb0a3 Azure:

[jira] [Updated] (HUDI-2496) Inserts are precombined even with dedup disabled

2021-09-27 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2496: - Priority: Critical (was: Major) > Inserts are precombined even with dedup disabled >

[jira] [Updated] (HUDI-2496) Inserts are precombined even with dedup disabled

2021-09-27 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2496: - Description: Original GH issue https://github.com/apache/hudi/issues/3709 Test case by [~xushiyan] :

[GitHub] [hudi] xushiyan commented on issue #3709: [SUPPORT] insert operation does not consistently insert duplicate records

2021-09-27 Thread GitBox
xushiyan commented on issue #3709: URL: https://github.com/apache/hudi/issues/3709#issuecomment-928732681 JIRA filed https://issues.apache.org/jira/browse/HUDI-2496 and we'll prioritize a fix. Thanks again @helanto -- This is an automated message from the Apache Git Service. To respond

[GitHub] [hudi] xushiyan closed issue #3709: [SUPPORT] insert operation does not consistently insert duplicate records

2021-09-27 Thread GitBox
xushiyan closed issue #3709: URL: https://github.com/apache/hudi/issues/3709 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Updated] (HUDI-2496) Inserts are precombined even with dedup disabled

2021-09-27 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2496: - Fix Version/s: 0.10.0 > Inserts are precombined even with dedup disabled >

[jira] [Updated] (HUDI-2496) Inserts are precombined even with dedup disabled

2021-09-27 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2496: - Labels: sev:critical (was: writer) > Inserts are precombined even with dedup disabled >

[jira] [Updated] (HUDI-2496) Inserts are precombined even with dedup disabled

2021-09-27 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2496: - Component/s: Writer Core > Inserts are precombined even with dedup disabled >

[GitHub] [hudi] hudi-bot edited a comment on pull request #3698: [HUDI-2474] Refreshing timeline for every operation in Hudi

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3698: URL: https://github.com/apache/hudi/pull/3698#issuecomment-924412554 ## CI report: * 88bb193962590b1aa8851893aeaf538d28a6e967 Azure:

[jira] [Created] (HUDI-2496) Inserts are precombined even with dedup disabled

2021-09-27 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-2496: - Summary: Inserts are precombined even with dedup disabled Key: HUDI-2496 URL: https://issues.apache.org/jira/browse/HUDI-2496 Project: Apache Hudi Issue Type: Bug

[GitHub] [hudi] hudi-bot edited a comment on pull request #3203: [HUDI-2086] Redo the logical of mor_incremental_view for hive

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3203: URL: https://github.com/apache/hudi/pull/3203#issuecomment-872092745 ## CI report: * 158bfbfd6991b583b3b708922459fb2c0eaf4f62 Azure:

[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #3668: [RFC-33] [HUDI-2429][WIP] Full schema evolution

2021-09-27 Thread GitBox
xiarixiaoyao commented on a change in pull request #3668: URL: https://github.com/apache/hudi/pull/3668#discussion_r717192565 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/AbstractHoodieWriteClient.java ## @@ -213,6 +224,27 @@ protected

[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #3668: [RFC-33] [HUDI-2429][WIP] Full schema evolution

2021-09-27 Thread GitBox
xiarixiaoyao commented on a change in pull request #3668: URL: https://github.com/apache/hudi/pull/3668#discussion_r717191016 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/AbstractHoodieWriteClient.java ## @@ -213,6 +224,27 @@ protected

[GitHub] [hudi] hudi-bot edited a comment on pull request #3203: [HUDI-2086] Redo the logical of mor_incremental_view for hive

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3203: URL: https://github.com/apache/hudi/pull/3203#issuecomment-872092745 ## CI report: * 158bfbfd6991b583b3b708922459fb2c0eaf4f62 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3698: [HUDI-2474] Refreshing timeline for every operation in Hudi

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3698: URL: https://github.com/apache/hudi/pull/3698#issuecomment-924412554 ## CI report: * c07cf243677b940fd58254e4e8f803ed3e597a12 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3203: [HUDI-2086] Redo the logical of mor_incremental_view for hive

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3203: URL: https://github.com/apache/hudi/pull/3203#issuecomment-872092745 ## CI report: * 158bfbfd6991b583b3b708922459fb2c0eaf4f62 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3698: [HUDI-2474] Refreshing timeline for every operation in Hudi

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3698: URL: https://github.com/apache/hudi/pull/3698#issuecomment-924412554 ## CI report: * c07cf243677b940fd58254e4e8f803ed3e597a12 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3203: [HUDI-2086] Redo the logical of mor_incremental_view for hive

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3203: URL: https://github.com/apache/hudi/pull/3203#issuecomment-872092745 ## CI report: * 158bfbfd6991b583b3b708922459fb2c0eaf4f62 Azure:

[GitHub] [hudi] nsivabalan commented on pull request #2691: [HUDI-1703] Fixing kafka auto.reset.offsets config param key

2021-09-27 Thread GitBox
nsivabalan commented on pull request #2691: URL: https://github.com/apache/hudi/pull/2691#issuecomment-928636419 @YannByron : yes. it was a oversight that we landed this. later this got fixed. so with 090, you can use "auto.offset.reset". -- This is an automated message from the Apache

[GitHub] [hudi] hudi-bot edited a comment on pull request #3203: [HUDI-2086] Redo the logical of mor_incremental_view for hive

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3203: URL: https://github.com/apache/hudi/pull/3203#issuecomment-872092745 ## CI report: * 29e6acb65044597e35f9d499385dad940580d9df Azure:

[GitHub] [hudi] nsivabalan edited a comment on pull request #3715: [HUDI-2487] fix JsonKafkaSource cannot filter empty messages from kafka

2021-09-27 Thread GitBox
nsivabalan edited a comment on pull request #3715: URL: https://github.com/apache/hudi/pull/3715#issuecomment-928632193 LGTM. will let @yanghua land this in. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] nsivabalan commented on pull request #3715: [HUDI-2487] fix JsonKafkaSource cannot filter empty messages from kafka

2021-09-27 Thread GitBox
nsivabalan commented on pull request #3715: URL: https://github.com/apache/hudi/pull/3715#issuecomment-928632193 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] hudi-bot edited a comment on pull request #3698: [HUDI-2474] Refreshing timeline for every operation in Hudi

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3698: URL: https://github.com/apache/hudi/pull/3698#issuecomment-924412554 ## CI report: * c07cf243677b940fd58254e4e8f803ed3e597a12 Azure:

[GitHub] [hudi] nsivabalan commented on a change in pull request #3413: [HUDI-2277] HoodieDeltaStreamer reading ORC files directly using ORCDFSSource

2021-09-27 Thread GitBox
nsivabalan commented on a change in pull request #3413: URL: https://github.com/apache/hudi/pull/3413#discussion_r717168485 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/functional/TestHoodieDeltaStreamer.java ## @@ -1398,6 +1399,34 @@ private void

[GitHub] [hudi] qianchutao commented on pull request #3715: [HUDI-2487] fix JsonKafkaSource cannot filter empty messages from kafka

2021-09-27 Thread GitBox
qianchutao commented on pull request #3715: URL: https://github.com/apache/hudi/pull/3715#issuecomment-928605496 @yanghua Mr. Yang, I have made corresponding modifications and added the unit test part. Please help me review it again ,thanks -- This is an automated message from the

[GitHub] [hudi] fuyun2024 commented on pull request #3722: HUDI-2491 hoodie.datasource.hive_sync.mode=hms mode is supported in s…

2021-09-27 Thread GitBox
fuyun2024 commented on pull request #3722: URL: https://github.com/apache/hudi/pull/3722#issuecomment-928586896 I'm sorry, I don't know this mistake. Who can give me some advice? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[jira] [Updated] (HUDI-2495) Difference in behavior between GenericRecord based key gen and Row based key gen

2021-09-27 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2495: - Component/s: Spark Integration > Difference in behavior between GenericRecord based key gen and Row based

[jira] [Updated] (HUDI-2495) Difference in behavior between GenericRecord based key gen and Row based key gen

2021-09-27 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2495: - Priority: Critical (was: Major) > Difference in behavior between GenericRecord based key gen and Row

[jira] [Updated] (HUDI-2495) Difference in behavior between GenericRecord based key gen and Row based key gen

2021-09-27 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2495: -- Description: when complex key gen is used and one of the field in record key is a

[GitHub] [hudi] hudi-bot edited a comment on pull request #3698: [HUDI-2474] Refreshing timeline for every operation in Hudi

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3698: URL: https://github.com/apache/hudi/pull/3698#issuecomment-924412554 ## CI report: * d0c4f3f59687cf74c47c3d41328264ae7d7daab9 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3698: [HUDI-2474] Refreshing timeline for every operation in Hudi

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3698: URL: https://github.com/apache/hudi/pull/3698#issuecomment-924412554 ## CI report: * d0c4f3f59687cf74c47c3d41328264ae7d7daab9 Azure:

[jira] [Updated] (HUDI-2495) Difference in behavior between GenericRecord based key gen and Row based key gen

2021-09-27 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2495: -- Labels: sev:critical (was: ) > Difference in behavior between GenericRecord based key

[GitHub] [hudi] nsivabalan closed issue #3313: [SUPPORT] CoW: Hudi Upsert not working when there is a timestamp field in the composite key

2021-09-27 Thread GitBox
nsivabalan closed issue #3313: URL: https://github.com/apache/hudi/issues/3313 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] nsivabalan commented on issue #3313: [SUPPORT] CoW: Hudi Upsert not working when there is a timestamp field in the composite key

2021-09-27 Thread GitBox
nsivabalan commented on issue #3313: URL: https://github.com/apache/hudi/issues/3313#issuecomment-928330326 I can confirm there is a bug here. GenericRecord based keygen converts the timestamp value, where as Row based one does not convert. Have filed a bug

[jira] [Assigned] (HUDI-2495) Difference in behavior between GenericRecord based key gen and Row based key gen

2021-09-27 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-2495: - Assignee: sivabalan narayanan > Difference in behavior between GenericRecord

[jira] [Created] (HUDI-2495) Difference in behavior between GenericRecord based key gen and Row based key gen

2021-09-27 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-2495: - Summary: Difference in behavior between GenericRecord based key gen and Row based key gen Key: HUDI-2495 URL: https://issues.apache.org/jira/browse/HUDI-2495

[GitHub] [hudi] nsivabalan commented on issue #3313: [SUPPORT] CoW: Hudi Upsert not working when there is a timestamp field in the composite key

2021-09-27 Thread GitBox
nsivabalan commented on issue #3313: URL: https://github.com/apache/hudi/issues/3313#issuecomment-928290740 @mkk1490 : sorry the issue got lengthy and I have got a couple of clarifications. Is your issue is with record key fields having one component as timestamp or is it about

[GitHub] [hudi] hudi-bot edited a comment on pull request #3698: [HUDI-2474] Refreshing timeline for every operation in Hudi

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3698: URL: https://github.com/apache/hudi/pull/3698#issuecomment-924412554 ## CI report: * d0c4f3f59687cf74c47c3d41328264ae7d7daab9 Azure:

[GitHub] [hudi] nsivabalan commented on issue #3533: [SUPPORT]How to use MOR Table to Merge small file?

2021-09-27 Thread GitBox
nsivabalan commented on issue #3533: URL: https://github.com/apache/hudi/issues/3533#issuecomment-928269532 @aresa7796 : if we dig in more if you can provide us w/ more info like file sizes, etc. As of now, we can't debug much without that info. appreciate if you can respond w/ details.

[GitHub] [hudi] hudi-bot edited a comment on pull request #3590: [HUDI-2285][HUDI-2476] Metadata table synchronous design. Rebased and Squashed from pull/3426

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3590: URL: https://github.com/apache/hudi/pull/3590#issuecomment-912237120 ## CI report: * aefac7ec2f2e40bdf3ad4365ea6aa825803a439d UNKNOWN * 3b5fbfa0cc69e82f7c23fc6aa5b4944e2dd77220 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3698: [HUDI-2474] Refreshing timeline for every operation in Hudi

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3698: URL: https://github.com/apache/hudi/pull/3698#issuecomment-924412554 ## CI report: * 78b26aa57880cd6ea6c20d9c0fc36109e959874f Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3698: [HUDI-2474] Refreshing timeline for every operation in Hudi

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3698: URL: https://github.com/apache/hudi/pull/3698#issuecomment-924412554 ## CI report: * 78b26aa57880cd6ea6c20d9c0fc36109e959874f Azure:

[GitHub] [hudi] nsivabalan commented on pull request #3698: [HUDI-2474] Refreshing timeline for every operation in Hudi

2021-09-27 Thread GitBox
nsivabalan commented on pull request #3698: URL: https://github.com/apache/hudi/pull/3698#issuecomment-928235206 @hudi-bot azure run -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] hudi-bot edited a comment on pull request #3590: [HUDI-2285][HUDI-2476] Metadata table synchronous design. Rebased and Squashed from pull/3426

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3590: URL: https://github.com/apache/hudi/pull/3590#issuecomment-912237120 ## CI report: * aefac7ec2f2e40bdf3ad4365ea6aa825803a439d UNKNOWN * 92634fb730e723a9bdeb165348a4c747794be7e3 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3698: [HUDI-2474] Refreshing timeline for every operation in Hudi

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3698: URL: https://github.com/apache/hudi/pull/3698#issuecomment-924412554 ## CI report: * 78b26aa57880cd6ea6c20d9c0fc36109e959874f Azure:

[GitHub] [hudi] nsivabalan commented on pull request #3590: [HUDI-2285][HUDI-2476] Metadata table synchronous design. Rebased and Squashed from pull/3426

2021-09-27 Thread GitBox
nsivabalan commented on pull request #3590: URL: https://github.com/apache/hudi/pull/3590#issuecomment-928195283 @hudi-bot azure run -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] xushiyan commented on pull request #3726: [MINOR] Add a RFC template and folder

2021-09-27 Thread GitBox
xushiyan commented on pull request #3726: URL: https://github.com/apache/hudi/pull/3726#issuecomment-928184950 @vinothchandar RFCs should be kept in asf-site? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] hudi-bot edited a comment on pull request #3698: [HUDI-2474] Refreshing timeline for every operation in Hudi

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3698: URL: https://github.com/apache/hudi/pull/3698#issuecomment-924412554 ## CI report: * 57b876e62a44a2430ba49f51de67f19f7eb5a70d Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3698: [HUDI-2474] Refreshing timeline for every operation in Hudi

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3698: URL: https://github.com/apache/hudi/pull/3698#issuecomment-924412554 ## CI report: * 57b876e62a44a2430ba49f51de67f19f7eb5a70d Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3590: [HUDI-2285][HUDI-2476] Metadata table synchronous design. Rebased and Squashed from pull/3426

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3590: URL: https://github.com/apache/hudi/pull/3590#issuecomment-912237120 ## CI report: * aefac7ec2f2e40bdf3ad4365ea6aa825803a439d UNKNOWN * 92634fb730e723a9bdeb165348a4c747794be7e3 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3671: [HUDI-2418] add HiveSchemaProvider

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3671: URL: https://github.com/apache/hudi/pull/3671#issuecomment-920215882 ## CI report: * 35e7f483618be007802784caa5c64b953cc0ffda Azure:

[GitHub] [hudi] vingov commented on issue #2934: [SUPPORT] Parquet file does not exist when trying to read hudi table incrementally

2021-09-27 Thread GitBox
vingov commented on issue #2934: URL: https://github.com/apache/hudi/issues/2934#issuecomment-928063194 @jsbali is working on a fix when we hit this edge case, the delta streamer will do a snapshot query and update the `deltastreamer.checkpoint.key` to the latest commit so that the next

[jira] [Updated] (HUDI-864) parquet schema conflict: optional binary (UTF8) is not a group

2021-09-27 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-864: Affects Version/s: 0.6.0 0.5.3 0.7.0

[jira] [Updated] (HUDI-864) parquet schema conflict: optional binary (UTF8) is not a group

2021-09-27 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-864: Affects Version/s: 0.9.0 > parquet schema conflict: optional binary (UTF8) is not a group >

[GitHub] [hudi] hudi-bot edited a comment on pull request #3671: [HUDI-2418] add HiveSchemaProvider

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3671: URL: https://github.com/apache/hudi/pull/3671#issuecomment-920215882 ## CI report: * 5c7664dd776e18d31f46db5eff91dd3d87bf590c Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3671: [HUDI-2418] add HiveSchemaProvider

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3671: URL: https://github.com/apache/hudi/pull/3671#issuecomment-920215882 ## CI report: * 5c7664dd776e18d31f46db5eff91dd3d87bf590c Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3715: [HUDI-2487] fix JsonKafkaSource cannot filter empty messages from kafka

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3715: URL: https://github.com/apache/hudi/pull/3715#issuecomment-927115196 ## CI report: * deca323a98fa197fdb70e7451c958200308703ce Azure:

[GitHub] [hudi] fengjian428 commented on a change in pull request #3671: [HUDI-2418] add HiveSchemaProvider

2021-09-27 Thread GitBox
fengjian428 commented on a change in pull request #3671: URL: https://github.com/apache/hudi/pull/3671#discussion_r716837947 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/functional/TestHiveSchemaProvider.java ## @@ -0,0 +1,124 @@ +/* + * Licensed to

[GitHub] [hudi] hudi-bot edited a comment on pull request #3698: [HUDI-2474] Refreshing timeline for every operation in Hudi

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3698: URL: https://github.com/apache/hudi/pull/3698#issuecomment-924412554 ## CI report: * 57b876e62a44a2430ba49f51de67f19f7eb5a70d Azure:

[GitHub] [hudi] xushiyan closed issue #3617: [SUPPORT] Hive Sync to Glue throws Failed to read data schema

2021-09-27 Thread GitBox
xushiyan closed issue #3617: URL: https://github.com/apache/hudi/issues/3617 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3726: [MINOR] Add a RFC template and folder

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3726: URL: https://github.com/apache/hudi/pull/3726#issuecomment-927945227 ## CI report: * f1b58045603fec113b12eba78884f6e9ad98d929 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3715: [HUDI-2487] fix JsonKafkaSource cannot filter empty messages from kafka

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3715: URL: https://github.com/apache/hudi/pull/3715#issuecomment-927115196 ## CI report: * ea7a6f04b3c60354589c24bdb70eb45883a56e14 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3698: [HUDI-2474] Refreshing timeline for every operation in Hudi

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3698: URL: https://github.com/apache/hudi/pull/3698#issuecomment-924412554 ## CI report: * dce615fd3058a817e839a852dc5de8b06d518658 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3715: [HUDI-2487] fix JsonKafkaSource cannot filter empty messages from kafka

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3715: URL: https://github.com/apache/hudi/pull/3715#issuecomment-927115196 ## CI report: * ea7a6f04b3c60354589c24bdb70eb45883a56e14 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3698: [HUDI-2474] Refreshing timeline for every operation in Hudi

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3698: URL: https://github.com/apache/hudi/pull/3698#issuecomment-924412554 ## CI report: * dce615fd3058a817e839a852dc5de8b06d518658 Azure:

[jira] [Updated] (HUDI-2494) Fix usage of different key generators with metadata enabled

2021-09-27 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2494: -- Parent: HUDI-1292 Issue Type: Sub-task (was: Improvement) > Fix usage of

[jira] [Created] (HUDI-2494) Fix usage of different key generators with metadata enabled

2021-09-27 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-2494: - Summary: Fix usage of different key generators with metadata enabled Key: HUDI-2494 URL: https://issues.apache.org/jira/browse/HUDI-2494 Project: Apache

[jira] [Updated] (HUDI-2493) Verify removing glob pattern works w/ all key generators

2021-09-27 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2493: -- Description: In the last release we added support to remove glob pattern. i.e.  while

[jira] [Updated] (HUDI-2493) Verify removing glob pattern works w/ all key generators

2021-09-27 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2493: -- Description: In the last release we added support to remove glob pattern. i.e.  while

[jira] [Assigned] (HUDI-2493) Verify removing glob pattern works w/ all key generators

2021-09-27 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-2493: - Assignee: Raymond Xu > Verify removing glob pattern works w/ all key generators

[jira] [Created] (HUDI-2493) Verify removing glob pattern works w/ all key generators

2021-09-27 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-2493: - Summary: Verify removing glob pattern works w/ all key generators Key: HUDI-2493 URL: https://issues.apache.org/jira/browse/HUDI-2493 Project: Apache Hudi

[GitHub] [hudi] hudi-bot edited a comment on pull request #3726: [MINOR] Add a RFC template and folder

2021-09-27 Thread GitBox
hudi-bot edited a comment on pull request #3726: URL: https://github.com/apache/hudi/pull/3726#issuecomment-927945227 ## CI report: * f1b58045603fec113b12eba78884f6e9ad98d929 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3726: [MINOR] Add a RFC template and folder

2021-09-27 Thread GitBox
hudi-bot commented on pull request #3726: URL: https://github.com/apache/hudi/pull/3726#issuecomment-927945227 ## CI report: * f1b58045603fec113b12eba78884f6e9ad98d929 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[GitHub] [hudi] vinothchandar commented on pull request #3726: [MINOR] Add a RFC template and folder

2021-09-27 Thread GitBox
vinothchandar commented on pull request #3726: URL: https://github.com/apache/hudi/pull/3726#issuecomment-927943161 templates for #3725 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] vinothchandar opened a new pull request #3726: [MINOR] Add a RFC template and folder

2021-09-27 Thread GitBox
vinothchandar opened a new pull request #3726: URL: https://github.com/apache/hudi/pull/3726 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the

  1   2   >