[jira] [Commented] (HUDI-1771) Propagate CDC format for hoodie
[ https://issues.apache.org/jira/browse/HUDI-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381786#comment-17381786 ] ASF GitHub Bot commented on HUDI-1771: -- codecov-commenter edited a comment on pull request #3285: URL: https://github.com/apache/hudi/pull/3285#issuecomment-881152765 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3285?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3285](https://codecov.io/gh/apache/hudi/pull/3285?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a517fd1) into [master](https://codecov.io/gh/apache/hudi/commit/a62a6cff32e7d7f62ffdf6362fb8876497fd0528?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a62a6cf) will **decrease** coverage by `21.02%`. > The diff coverage is `50.00%`. [](https://codecov.io/gh/apache/hudi/pull/3285?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3285 +/- ## = - Coverage 47.70% 26.68% -21.03% + Complexity 5550 1294 -4256 = Files 936 386 -550 Lines 4162415732-25892 Branches 4188 1355 -2833 = - Hits 19858 4198-15660 + Misses211229 -8771 + Partials 1766 305 -1461 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `20.27% <50.00%> (-14.19%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `4.88% <ø> (-51.10%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `59.23% <ø> (-0.04%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3285?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...in/java/org/apache/hudi/io/HoodieAppendHandle.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2lvL0hvb2RpZUFwcGVuZEhhbmRsZS5qYXZh) | `0.00% <0.00%> (ø)` | | | [...in/java/org/apache/hudi/io/HoodieCreateHandle.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2lvL0hvb2RpZUNyZWF0ZUhhbmRsZS5qYXZh) | `0.00% <0.00%> (ø)` | | | [...ain/java/org/apache/hudi/io/HoodieMergeHandle.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2lvL0hvb2RpZU1lcmdlSGFuZGxlLmphdmE=) | `0.00% <0.00%> (ø)` | | | [...rg/apache/hudi/client/model/HoodieInternalRow.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L21vZGVsL0hvb2RpZUludGVybmFsUm93LmphdmE=) | `79.48% <87.50%> (+1.70%)` | :arrow_up: | | [...java/org/apache/hudi/io/HoodieRowCreateHandle.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvaW8vSG9vZGllUm93Q3JlYXRlSGFuZGxlLmphdmE=) | `92.77% <100.00%> (+0.27%)` | :arrow_up: | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3285: [HUDI-1771] Propagate CDC format for hoodie
codecov-commenter edited a comment on pull request #3285: URL: https://github.com/apache/hudi/pull/3285#issuecomment-881152765 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3285?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3285](https://codecov.io/gh/apache/hudi/pull/3285?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a517fd1) into [master](https://codecov.io/gh/apache/hudi/commit/a62a6cff32e7d7f62ffdf6362fb8876497fd0528?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a62a6cf) will **decrease** coverage by `21.02%`. > The diff coverage is `50.00%`. [](https://codecov.io/gh/apache/hudi/pull/3285?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3285 +/- ## = - Coverage 47.70% 26.68% -21.03% + Complexity 5550 1294 -4256 = Files 936 386 -550 Lines 4162415732-25892 Branches 4188 1355 -2833 = - Hits 19858 4198-15660 + Misses211229 -8771 + Partials 1766 305 -1461 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `20.27% <50.00%> (-14.19%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `4.88% <ø> (-51.10%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `59.23% <ø> (-0.04%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3285?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...in/java/org/apache/hudi/io/HoodieAppendHandle.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2lvL0hvb2RpZUFwcGVuZEhhbmRsZS5qYXZh) | `0.00% <0.00%> (ø)` | | | [...in/java/org/apache/hudi/io/HoodieCreateHandle.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2lvL0hvb2RpZUNyZWF0ZUhhbmRsZS5qYXZh) | `0.00% <0.00%> (ø)` | | | [...ain/java/org/apache/hudi/io/HoodieMergeHandle.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2lvL0hvb2RpZU1lcmdlSGFuZGxlLmphdmE=) | `0.00% <0.00%> (ø)` | | | [...rg/apache/hudi/client/model/HoodieInternalRow.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L21vZGVsL0hvb2RpZUludGVybmFsUm93LmphdmE=) | `79.48% <87.50%> (+1.70%)` | :arrow_up: | | [...java/org/apache/hudi/io/HoodieRowCreateHandle.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvaW8vSG9vZGllUm93Q3JlYXRlSGFuZGxlLmphdmE=) | `92.77% <100.00%> (+0.27%)` | :arrow_up: | |
[jira] [Commented] (HUDI-1884) MergeInto Support Partial Update For COW
[
https://issues.apache.org/jira/browse/HUDI-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381784#comment-17381784
]
ASF GitHub Bot commented on HUDI-1884:
--
hudi-bot edited a comment on pull request #3154:
URL: https://github.com/apache/hudi/pull/3154#issuecomment-868360998
## CI report:
* 63cf96134656dbb41eaab4e03e747776599255b6 UNKNOWN
* 82b4b8bef8d143a14d445ae35fa5ec8fbbd5889f UNKNOWN
* f5f0fd3f1eb1529643b7129349614ed80d106bed Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=956)
Bot commands
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> MergeInto Support Partial Update For COW
> -
>
> Key: HUDI-1884
> URL: https://issues.apache.org/jira/browse/HUDI-1884
> Project: Apache Hudi
> Issue Type: Sub-task
>Reporter: pengzhiwei
>Assignee: pengzhiwei
>Priority: Major
> Labels: pull-request-available
>
> Support partial update for merge-into statement, just like this:
> {code:java}
> merge into h0
> using s0
> on s0.id = h0.id
> when matched then update set price = s0.price + 10
> when not matched then insert *{code}
> Allow to update partial fields.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hudi] hudi-bot edited a comment on pull request #3154: [HUDI-1884] MergeInto Support Partial Update For COW
hudi-bot edited a comment on pull request #3154: URL: https://github.com/apache/hudi/pull/3154#issuecomment-868360998 ## CI report: * 63cf96134656dbb41eaab4e03e747776599255b6 UNKNOWN * 82b4b8bef8d143a14d445ae35fa5ec8fbbd5889f UNKNOWN * f5f0fd3f1eb1529643b7129349614ed80d106bed Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=956) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HUDI-1884) MergeInto Support Partial Update For COW
[
https://issues.apache.org/jira/browse/HUDI-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381783#comment-17381783
]
ASF GitHub Bot commented on HUDI-1884:
--
codecov-commenter edited a comment on pull request #3154:
URL: https://github.com/apache/hudi/pull/3154#issuecomment-868445474
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> MergeInto Support Partial Update For COW
> -
>
> Key: HUDI-1884
> URL: https://issues.apache.org/jira/browse/HUDI-1884
> Project: Apache Hudi
> Issue Type: Sub-task
>Reporter: pengzhiwei
>Assignee: pengzhiwei
>Priority: Major
> Labels: pull-request-available
>
> Support partial update for merge-into statement, just like this:
> {code:java}
> merge into h0
> using s0
> on s0.id = h0.id
> when matched then update set price = s0.price + 10
> when not matched then insert *{code}
> Allow to update partial fields.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3154: [HUDI-1884] MergeInto Support Partial Update For COW
codecov-commenter edited a comment on pull request #3154: URL: https://github.com/apache/hudi/pull/3154#issuecomment-868445474 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HUDI-2170) Always choose the latest record for HoodieRecordPayload
[ https://issues.apache.org/jira/browse/HUDI-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381782#comment-17381782 ] ASF GitHub Bot commented on HUDI-2170: -- codecov-commenter edited a comment on pull request #3267: URL: https://github.com/apache/hudi/pull/3267#issuecomment-878977860 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3267?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3267](https://codecov.io/gh/apache/hudi/pull/3267?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a275c3a) into [master](https://codecov.io/gh/apache/hudi/commit/01ad449ad68d9c77ee1493fcdf833df53df6106a?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (01ad449) will **decrease** coverage by `34.84%`. > The diff coverage is `0.00%`. [](https://codecov.io/gh/apache/hudi/pull/3267?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3267 +/- ## = - Coverage 52.19% 17.35% -34.85% + Complexity 2659 884 -1775 = Files 335 386 +51 Lines 1498115336 +355 Branches 1505 1338 -167 = - Hits 7819 2661 -5158 - Misses 653612508 +5972 + Partials626 167 -459 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `20.91% <0.00%> (∅)` | | | hudicommon | `?` | | | hudihadoopmr | `?` | | | hudisync | `4.88% <ø> (?)` | | | huditimelineservice | `?` | | | hudiutilities | `9.11% <ø> (-60.54%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3267?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [.../org/apache/hudi/async/AsyncClusteringService.java](https://codecov.io/gh/apache/hudi/pull/3267/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2FzeW5jL0FzeW5jQ2x1c3RlcmluZ1NlcnZpY2UuamF2YQ==) | `0.00% <0.00%> (ø)` | | | [...ava/org/apache/hudi/async/AsyncCompactService.java](https://codecov.io/gh/apache/hudi/pull/3267/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2FzeW5jL0FzeW5jQ29tcGFjdFNlcnZpY2UuamF2YQ==) | `0.00% <0.00%> (ø)` | | | [...java/org/apache/hudi/async/HoodieAsyncService.java](https://codecov.io/gh/apache/hudi/pull/3267/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2FzeW5jL0hvb2RpZUFzeW5jU2VydmljZS5qYXZh) | `0.00% <0.00%> (ø)` | | | [...ient/http/HoodieWriteCommitHttpCallbackClient.java](https://codecov.io/gh/apache/hudi/pull/3267/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NhbGxiYWNrL2NsaWVudC9odHRwL0hvb2RpZVdyaXRlQ29tbWl0SHR0cENhbGxiYWNrQ2xpZW50LmphdmE=) | `0.00% <0.00%> (ø)` | | | [...udi/callback/util/HoodieCommitCallbackFactory.java](https://codecov.io/gh/apache/hudi/pull/3267/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NhbGxiYWNrL3V0aWwvSG9vZGllQ29tbWl0Q2FsbGJhY2tGYWN0b3J5LmphdmE=) | `0.00% <0.00%> (ø)` | | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3267: [HUDI-2170] Always choose the latest record for HoodieRecordPayload
codecov-commenter edited a comment on pull request #3267: URL: https://github.com/apache/hudi/pull/3267#issuecomment-878977860 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3267?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3267](https://codecov.io/gh/apache/hudi/pull/3267?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a275c3a) into [master](https://codecov.io/gh/apache/hudi/commit/01ad449ad68d9c77ee1493fcdf833df53df6106a?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (01ad449) will **decrease** coverage by `34.84%`. > The diff coverage is `0.00%`. [](https://codecov.io/gh/apache/hudi/pull/3267?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3267 +/- ## = - Coverage 52.19% 17.35% -34.85% + Complexity 2659 884 -1775 = Files 335 386 +51 Lines 1498115336 +355 Branches 1505 1338 -167 = - Hits 7819 2661 -5158 - Misses 653612508 +5972 + Partials626 167 -459 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `20.91% <0.00%> (∅)` | | | hudicommon | `?` | | | hudihadoopmr | `?` | | | hudisync | `4.88% <ø> (?)` | | | huditimelineservice | `?` | | | hudiutilities | `9.11% <ø> (-60.54%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3267?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [.../org/apache/hudi/async/AsyncClusteringService.java](https://codecov.io/gh/apache/hudi/pull/3267/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2FzeW5jL0FzeW5jQ2x1c3RlcmluZ1NlcnZpY2UuamF2YQ==) | `0.00% <0.00%> (ø)` | | | [...ava/org/apache/hudi/async/AsyncCompactService.java](https://codecov.io/gh/apache/hudi/pull/3267/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2FzeW5jL0FzeW5jQ29tcGFjdFNlcnZpY2UuamF2YQ==) | `0.00% <0.00%> (ø)` | | | [...java/org/apache/hudi/async/HoodieAsyncService.java](https://codecov.io/gh/apache/hudi/pull/3267/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2FzeW5jL0hvb2RpZUFzeW5jU2VydmljZS5qYXZh) | `0.00% <0.00%> (ø)` | | | [...ient/http/HoodieWriteCommitHttpCallbackClient.java](https://codecov.io/gh/apache/hudi/pull/3267/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NhbGxiYWNrL2NsaWVudC9odHRwL0hvb2RpZVdyaXRlQ29tbWl0SHR0cENhbGxiYWNrQ2xpZW50LmphdmE=) | `0.00% <0.00%> (ø)` | | | [...udi/callback/util/HoodieCommitCallbackFactory.java](https://codecov.io/gh/apache/hudi/pull/3267/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NhbGxiYWNrL3V0aWwvSG9vZGllQ29tbWl0Q2FsbGJhY2tGYWN0b3J5LmphdmE=) | `0.00% <0.00%> (ø)` | | |
[jira] [Commented] (HUDI-1771) Propagate CDC format for hoodie
[ https://issues.apache.org/jira/browse/HUDI-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381780#comment-17381780 ] ASF GitHub Bot commented on HUDI-1771: -- swuferhong closed pull request #3285: URL: https://github.com/apache/hudi/pull/3285 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Propagate CDC format for hoodie > --- > > Key: HUDI-1771 > URL: https://issues.apache.org/jira/browse/HUDI-1771 > Project: Apache Hudi > Issue Type: New Feature > Components: Flink Integration >Reporter: Danny Chen >Assignee: Zheng yunhong >Priority: Major > Labels: pull-request-available, sev:normal > Fix For: 0.9.0 > > > Like what we discussed in the dev mailing list: > https://lists.apache.org/thread.html/r31b2d1404e4e043a5f875b78105ba6f9a801e78f265ad91242ad5eb2%40%3Cdev.hudi.apache.org%3E > Keep the change flags make new use cases possible: using HUDI as the unified > storage format for DWD and DWS layer. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HUDI-1771) Propagate CDC format for hoodie
[ https://issues.apache.org/jira/browse/HUDI-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381781#comment-17381781 ] ASF GitHub Bot commented on HUDI-1771: -- swuferhong opened a new pull request #3285: URL: https://github.com/apache/hudi/pull/3285 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the pull request Propagate CDC format for hoodie. ## Brief change log *(for example:)* - *Modify AnnotationLocation checkstyle rule in checkstyle.xml* ## Verify this pull request *(Please pick either of the following options)* This pull request is a trivial rework / code cleanup without any test coverage. *(or)* This pull request is already covered by existing tests, such as *(please describe tests)*. (or) This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end.* - *Added HoodieClientWriteTest to verify the change.* - *Manually verified the change by running a job locally.* ## Committer checklist - [ ] Has a corresponding JIRA in PR title & commit - [ ] Commit message is descriptive of the change - [ ] CI is green - [ ] Necessary doc changes done or have another open PR - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Propagate CDC format for hoodie > --- > > Key: HUDI-1771 > URL: https://issues.apache.org/jira/browse/HUDI-1771 > Project: Apache Hudi > Issue Type: New Feature > Components: Flink Integration >Reporter: Danny Chen >Assignee: Zheng yunhong >Priority: Major > Labels: pull-request-available, sev:normal > Fix For: 0.9.0 > > > Like what we discussed in the dev mailing list: > https://lists.apache.org/thread.html/r31b2d1404e4e043a5f875b78105ba6f9a801e78f265ad91242ad5eb2%40%3Cdev.hudi.apache.org%3E > Keep the change flags make new use cases possible: using HUDI as the unified > storage format for DWD and DWS layer. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hudi] swuferhong closed pull request #3285: [HUDI-1771] Propagate CDC format for hoodie
swuferhong closed pull request #3285: URL: https://github.com/apache/hudi/pull/3285 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HUDI-2176) Virutal keys support for COW all operations
[ https://issues.apache.org/jira/browse/HUDI-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381778#comment-17381778 ] ASF GitHub Bot commented on HUDI-2176: -- hudi-bot edited a comment on pull request #3282: URL: https://github.com/apache/hudi/pull/3282#issuecomment-880706639 ## CI report: * 2cf38df103e1435c4239824f4908520727e38f69 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=958) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Virutal keys support for COW all operations > --- > > Key: HUDI-2176 > URL: https://issues.apache.org/jira/browse/HUDI-2176 > Project: Apache Hudi > Issue Type: Improvement > Components: Writer Core >Reporter: sivabalan narayanan >Priority: Major > Labels: pull-request-available > Fix For: 0.9.0 > > > Virutal keys support for COW all operations > (merge handle) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hudi] hudi-bot edited a comment on pull request #3282: [HUDI-2176, 2178, 2179] Virtual keys support to COW table
hudi-bot edited a comment on pull request #3282: URL: https://github.com/apache/hudi/pull/3282#issuecomment-880706639 ## CI report: * 2cf38df103e1435c4239824f4908520727e38f69 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=958) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HUDI-2170) Always choose the latest record for HoodieRecordPayload
[
https://issues.apache.org/jira/browse/HUDI-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381776#comment-17381776
]
ASF GitHub Bot commented on HUDI-2170:
--
hudi-bot edited a comment on pull request #3267:
URL: https://github.com/apache/hudi/pull/3267#issuecomment-878894177
## CI report:
* a275c3af0519430c6ff0dc5902dd84dc5d1a0882 Azure:
[FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=955)
Bot commands
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Always choose the latest record for HoodieRecordPayload
> ---
>
> Key: HUDI-2170
> URL: https://issues.apache.org/jira/browse/HUDI-2170
> Project: Apache Hudi
> Issue Type: Improvement
> Components: Common Core
>Reporter: Danny Chen
>Assignee: Danny Chen
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.9.0
>
>
> Now in {{OverwriteWithLatestAvroPayload.preCombine}}, we still choose the old
> record when the new record has the same preCombine field with the old one,
> actually it is more natural to keep the new incoming record instead. The
> {{DefaultHoodieRecordPayload.combineAndGetUpdateValue}} method already did
> that.
> See issue: https://github.com/apache/hudi/issues/3266.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HUDI-2176) Virutal keys support for COW all operations
[ https://issues.apache.org/jira/browse/HUDI-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381777#comment-17381777 ] ASF GitHub Bot commented on HUDI-2176: -- hudi-bot edited a comment on pull request #3282: URL: https://github.com/apache/hudi/pull/3282#issuecomment-880706639 ## CI report: * 258dd4a9659b1edf6417ab645a3e93221cfcd2d6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=943) * 2cf38df103e1435c4239824f4908520727e38f69 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=958) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Virutal keys support for COW all operations > --- > > Key: HUDI-2176 > URL: https://issues.apache.org/jira/browse/HUDI-2176 > Project: Apache Hudi > Issue Type: Improvement > Components: Writer Core >Reporter: sivabalan narayanan >Priority: Major > Labels: pull-request-available > Fix For: 0.9.0 > > > Virutal keys support for COW all operations > (merge handle) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hudi] hudi-bot edited a comment on pull request #3282: [HUDI-2176, 2178, 2179] Virtual keys support to COW table
hudi-bot edited a comment on pull request #3282: URL: https://github.com/apache/hudi/pull/3282#issuecomment-880706639 ## CI report: * 258dd4a9659b1edf6417ab645a3e93221cfcd2d6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=943) * 2cf38df103e1435c4239824f4908520727e38f69 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=958) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3267: [HUDI-2170] Always choose the latest record for HoodieRecordPayload
hudi-bot edited a comment on pull request #3267: URL: https://github.com/apache/hudi/pull/3267#issuecomment-878894177 ## CI report: * a275c3af0519430c6ff0dc5902dd84dc5d1a0882 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=955) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HUDI-2176) Virutal keys support for COW all operations
[ https://issues.apache.org/jira/browse/HUDI-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381775#comment-17381775 ] ASF GitHub Bot commented on HUDI-2176: -- hudi-bot edited a comment on pull request #3282: URL: https://github.com/apache/hudi/pull/3282#issuecomment-880706639 ## CI report: * 258dd4a9659b1edf6417ab645a3e93221cfcd2d6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=943) * 2cf38df103e1435c4239824f4908520727e38f69 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Virutal keys support for COW all operations > --- > > Key: HUDI-2176 > URL: https://issues.apache.org/jira/browse/HUDI-2176 > Project: Apache Hudi > Issue Type: Improvement > Components: Writer Core >Reporter: sivabalan narayanan >Priority: Major > Labels: pull-request-available > Fix For: 0.9.0 > > > Virutal keys support for COW all operations > (merge handle) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hudi] hudi-bot edited a comment on pull request #3282: [HUDI-2176, 2178, 2179] Virtual keys support to COW table
hudi-bot edited a comment on pull request #3282: URL: https://github.com/apache/hudi/pull/3282#issuecomment-880706639 ## CI report: * 258dd4a9659b1edf6417ab645a3e93221cfcd2d6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=943) * 2cf38df103e1435c4239824f4908520727e38f69 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HUDI-1771) Propagate CDC format for hoodie
[ https://issues.apache.org/jira/browse/HUDI-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381773#comment-17381773 ] ASF GitHub Bot commented on HUDI-1771: -- swuferhong closed pull request #3285: URL: https://github.com/apache/hudi/pull/3285 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Propagate CDC format for hoodie > --- > > Key: HUDI-1771 > URL: https://issues.apache.org/jira/browse/HUDI-1771 > Project: Apache Hudi > Issue Type: New Feature > Components: Flink Integration >Reporter: Danny Chen >Assignee: Zheng yunhong >Priority: Major > Labels: pull-request-available, sev:normal > Fix For: 0.9.0 > > > Like what we discussed in the dev mailing list: > https://lists.apache.org/thread.html/r31b2d1404e4e043a5f875b78105ba6f9a801e78f265ad91242ad5eb2%40%3Cdev.hudi.apache.org%3E > Keep the change flags make new use cases possible: using HUDI as the unified > storage format for DWD and DWS layer. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HUDI-1771) Propagate CDC format for hoodie
[ https://issues.apache.org/jira/browse/HUDI-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381774#comment-17381774 ] ASF GitHub Bot commented on HUDI-1771: -- swuferhong opened a new pull request #3285: URL: https://github.com/apache/hudi/pull/3285 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the pull request Propagate CDC format for hoodie. ## Brief change log *(for example:)* - *Modify AnnotationLocation checkstyle rule in checkstyle.xml* ## Verify this pull request *(Please pick either of the following options)* This pull request is a trivial rework / code cleanup without any test coverage. *(or)* This pull request is already covered by existing tests, such as *(please describe tests)*. (or) This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end.* - *Added HoodieClientWriteTest to verify the change.* - *Manually verified the change by running a job locally.* ## Committer checklist - [ ] Has a corresponding JIRA in PR title & commit - [ ] Commit message is descriptive of the change - [ ] CI is green - [ ] Necessary doc changes done or have another open PR - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Propagate CDC format for hoodie > --- > > Key: HUDI-1771 > URL: https://issues.apache.org/jira/browse/HUDI-1771 > Project: Apache Hudi > Issue Type: New Feature > Components: Flink Integration >Reporter: Danny Chen >Assignee: Zheng yunhong >Priority: Major > Labels: pull-request-available, sev:normal > Fix For: 0.9.0 > > > Like what we discussed in the dev mailing list: > https://lists.apache.org/thread.html/r31b2d1404e4e043a5f875b78105ba6f9a801e78f265ad91242ad5eb2%40%3Cdev.hudi.apache.org%3E > Keep the change flags make new use cases possible: using HUDI as the unified > storage format for DWD and DWS layer. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hudi] swuferhong closed pull request #3285: [HUDI-1771] Propagate CDC format for hoodie
swuferhong closed pull request #3285: URL: https://github.com/apache/hudi/pull/3285 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HUDI-2176) Virutal keys support for COW all operations
[
https://issues.apache.org/jira/browse/HUDI-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381770#comment-17381770
]
ASF GitHub Bot commented on HUDI-2176:
--
nsivabalan commented on a change in pull request #3282:
URL: https://github.com/apache/hudi/pull/3282#discussion_r670964858
##
File path: hudi-cli/src/main/scala/org/apache/hudi/cli/SparkHelpers.scala
##
@@ -51,7 +51,8 @@ object SparkHelpers {
// Add current classLoad for config, if not will throw classNotFound of
'HoodieWrapperFileSystem'.
parquetConfig.getHadoopConf().setClassLoader(Thread.currentThread.getContextClassLoader)
-val writer = new HoodieParquetWriter[HoodieJsonPayload,
IndexedRecord](instantTime, destinationFile, parquetConfig, schema, new
SparkTaskContextSupplier())
+val writer = new HoodieParquetWriter[HoodieJsonPayload,
IndexedRecord](instantTime, destinationFile, parquetConfig, schema, new
SparkTaskContextSupplier(),
+ true)
Review comment:
yet to fix cli for virtual keys
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Virutal keys support for COW all operations
> ---
>
> Key: HUDI-2176
> URL: https://issues.apache.org/jira/browse/HUDI-2176
> Project: Apache Hudi
> Issue Type: Improvement
> Components: Writer Core
>Reporter: sivabalan narayanan
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.9.0
>
>
> Virutal keys support for COW all operations
> (merge handle)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hudi] nsivabalan commented on a change in pull request #3282: [HUDI-2176, 2178, 2179] Virtual keys support to COW table
nsivabalan commented on a change in pull request #3282:
URL: https://github.com/apache/hudi/pull/3282#discussion_r670964858
##
File path: hudi-cli/src/main/scala/org/apache/hudi/cli/SparkHelpers.scala
##
@@ -51,7 +51,8 @@ object SparkHelpers {
// Add current classLoad for config, if not will throw classNotFound of
'HoodieWrapperFileSystem'.
parquetConfig.getHadoopConf().setClassLoader(Thread.currentThread.getContextClassLoader)
-val writer = new HoodieParquetWriter[HoodieJsonPayload,
IndexedRecord](instantTime, destinationFile, parquetConfig, schema, new
SparkTaskContextSupplier())
+val writer = new HoodieParquetWriter[HoodieJsonPayload,
IndexedRecord](instantTime, destinationFile, parquetConfig, schema, new
SparkTaskContextSupplier(),
+ true)
Review comment:
yet to fix cli for virtual keys
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HUDI-2176) Virutal keys support for COW all operations
[
https://issues.apache.org/jira/browse/HUDI-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381769#comment-17381769
]
ASF GitHub Bot commented on HUDI-2176:
--
nsivabalan commented on a change in pull request #3282:
URL: https://github.com/apache/hudi/pull/3282#discussion_r670966147
##
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieMergeHandle.java
##
@@ -278,7 +287,7 @@ protected boolean writeRecord(HoodieRecord hoodieRecord,
Option> latestBaseFiles) {
List>,
List, List>> hoodieKeyLocationFetchHandles =
-context.map(latestBaseFiles, partitionPathBaseFile -> new
HoodieKeyLocationFetchHandle<>(config, hoodieTable, partitionPathBaseFile),
parallelism);
+context.map(latestBaseFiles, partitionPathBaseFile -> new
HoodieKeyLocationFetchHandle<>(config, hoodieTable, partitionPathBaseFile,
null), parallelism);
Review comment:
as I have mentioned in the desc, I am not fixing flink and java in this
patch.
##
File path:
hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/client/TestUpdateSchemaEvolution.java
##
@@ -121,7 +121,7 @@ private void
assertSchemaEvolutionOnUpdateResult(WriteStatus insertResult, Hoodi
jsc.parallelize(Arrays.asList(1)).map(x -> {
Executable executable = () -> {
HoodieMergeHandle mergeHandle = new
HoodieMergeHandle(updateTable.getConfig(), "101", updateTable,
-updateRecords.iterator(), updateRecords.get(0).getPartitionPath(),
insertResult.getFileId(), supplier);
+updateRecords.iterator(), updateRecords.get(0).getPartitionPath(),
insertResult.getFileId(), supplier, null);
Review comment:
Have not fixed this test for virtual keys
##
File path:
hudi-common/src/main/java/org/apache/hudi/common/util/ParquetUtils.java
##
@@ -142,6 +143,43 @@
return hoodieKeys;
}
+ /**
+ * Fetch {@link HoodieKey}s from the given parquet file.
+ *
+ * @param filePath The parquet file path.
+ * @param configuration configuration to build fs object
+ * @return {@link List} of {@link HoodieKey}s fetched from the parquet file
+ */
+ @Override
+ public List fetchRecordKeyPartitionPath(Configuration
configuration, Path filePath, BaseKeyGenerator keyGenerator) {
Review comment:
not sure if we can add another argument to existing api and
generate/fetch recordKeys and partition path based on that. Felt this is neat.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Virutal keys support for COW all operations
> ---
>
> Key: HUDI-2176
> URL: https://issues.apache.org/jira/browse/HUDI-2176
> Project: Apache Hudi
> Issue Type: Improvement
> Components: Writer Core
>Reporter: sivabalan narayanan
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.9.0
>
>
> Virutal keys support for COW all operations
> (merge handle)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hudi] nsivabalan commented on a change in pull request #3282: [HUDI-2176, 2178, 2179] Virtual keys support to COW table
nsivabalan commented on a change in pull request #3282:
URL: https://github.com/apache/hudi/pull/3282#discussion_r670966147
##
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieMergeHandle.java
##
@@ -278,7 +287,7 @@ protected boolean writeRecord(HoodieRecord hoodieRecord,
Option> latestBaseFiles) {
List>,
List, List>> hoodieKeyLocationFetchHandles =
-context.map(latestBaseFiles, partitionPathBaseFile -> new
HoodieKeyLocationFetchHandle<>(config, hoodieTable, partitionPathBaseFile),
parallelism);
+context.map(latestBaseFiles, partitionPathBaseFile -> new
HoodieKeyLocationFetchHandle<>(config, hoodieTable, partitionPathBaseFile,
null), parallelism);
Review comment:
as I have mentioned in the desc, I am not fixing flink and java in this
patch.
##
File path:
hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/client/TestUpdateSchemaEvolution.java
##
@@ -121,7 +121,7 @@ private void
assertSchemaEvolutionOnUpdateResult(WriteStatus insertResult, Hoodi
jsc.parallelize(Arrays.asList(1)).map(x -> {
Executable executable = () -> {
HoodieMergeHandle mergeHandle = new
HoodieMergeHandle(updateTable.getConfig(), "101", updateTable,
-updateRecords.iterator(), updateRecords.get(0).getPartitionPath(),
insertResult.getFileId(), supplier);
+updateRecords.iterator(), updateRecords.get(0).getPartitionPath(),
insertResult.getFileId(), supplier, null);
Review comment:
Have not fixed this test for virtual keys
##
File path:
hudi-common/src/main/java/org/apache/hudi/common/util/ParquetUtils.java
##
@@ -142,6 +143,43 @@
return hoodieKeys;
}
+ /**
+ * Fetch {@link HoodieKey}s from the given parquet file.
+ *
+ * @param filePath The parquet file path.
+ * @param configuration configuration to build fs object
+ * @return {@link List} of {@link HoodieKey}s fetched from the parquet file
+ */
+ @Override
+ public List fetchRecordKeyPartitionPath(Configuration
configuration, Path filePath, BaseKeyGenerator keyGenerator) {
Review comment:
not sure if we can add another argument to existing api and
generate/fetch recordKeys and partition path based on that. Felt this is neat.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #2296: [HUDI-1425] Performance loss with the additional hoodieRecords.isEmpty() in HoodieSparkSqlWriter#write
hudi-bot edited a comment on pull request #2296: URL: https://github.com/apache/hudi/pull/2296#issuecomment-874367360 ## CI report: * d4e3d89e0c02f1b9a704a61627f7aafec14e67e6 UNKNOWN * dc36883644cf88971ecf13dc3642d60e60a1e6ec UNKNOWN * cd6767b606cfa19b064e4455f7a396e84e13d839 UNKNOWN * 7ebc9ff5395d8790339586ae3ff7e56b05a6a569 UNKNOWN * 4c86b7708e4204d585180ae8e9187c416ba6d052 UNKNOWN * fa8ba5e890fd394820a0ff91e0395b9be62d7ba8 UNKNOWN * 6ac9d507dc541a7e17741c3e1fcd8a6bb28aecc6 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=954) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HUDI-1425) Performance loss with the additional hoodieRecords.isEmpty() in HoodieSparkSqlWriter#write
[ https://issues.apache.org/jira/browse/HUDI-1425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381764#comment-17381764 ] ASF GitHub Bot commented on HUDI-1425: -- hudi-bot edited a comment on pull request #2296: URL: https://github.com/apache/hudi/pull/2296#issuecomment-874367360 ## CI report: * d4e3d89e0c02f1b9a704a61627f7aafec14e67e6 UNKNOWN * dc36883644cf88971ecf13dc3642d60e60a1e6ec UNKNOWN * cd6767b606cfa19b064e4455f7a396e84e13d839 UNKNOWN * 7ebc9ff5395d8790339586ae3ff7e56b05a6a569 UNKNOWN * 4c86b7708e4204d585180ae8e9187c416ba6d052 UNKNOWN * fa8ba5e890fd394820a0ff91e0395b9be62d7ba8 UNKNOWN * 6ac9d507dc541a7e17741c3e1fcd8a6bb28aecc6 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=954) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Performance loss with the additional hoodieRecords.isEmpty() in > HoodieSparkSqlWriter#write > -- > > Key: HUDI-1425 > URL: https://issues.apache.org/jira/browse/HUDI-1425 > Project: Apache Hudi > Issue Type: Improvement > Components: Spark Integration >Affects Versions: 0.9.0 >Reporter: pengzhiwei >Assignee: pengzhiwei >Priority: Blocker > Labels: pull-request-available > Fix For: 0.9.0 > > Attachments: 截屏2020-11-30 下午9.47.55.png > > > Currently in HoodieSparkSqlWriter#write, there is a _isEmpty()_ test for > _hoodieRecords._ This may be a heavy operator in the case when the > _hoodieRecords_ contains complex RDD operate. > !截屏2020-11-30 下午9.47.55.png|width=1255,height=161! > IMO this test does nothing to do with the performance improve，but rather > affects performance. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HUDI-2176) Virutal keys support for COW all operations
[
https://issues.apache.org/jira/browse/HUDI-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381761#comment-17381761
]
ASF GitHub Bot commented on HUDI-2176:
--
nsivabalan commented on a change in pull request #3282:
URL: https://github.com/apache/hudi/pull/3282#discussion_r670964858
##
File path: hudi-cli/src/main/scala/org/apache/hudi/cli/SparkHelpers.scala
##
@@ -51,7 +51,8 @@ object SparkHelpers {
// Add current classLoad for config, if not will throw classNotFound of
'HoodieWrapperFileSystem'.
parquetConfig.getHadoopConf().setClassLoader(Thread.currentThread.getContextClassLoader)
-val writer = new HoodieParquetWriter[HoodieJsonPayload,
IndexedRecord](instantTime, destinationFile, parquetConfig, schema, new
SparkTaskContextSupplier())
+val writer = new HoodieParquetWriter[HoodieJsonPayload,
IndexedRecord](instantTime, destinationFile, parquetConfig, schema, new
SparkTaskContextSupplier(),
+ true)
Review comment:
yet to fix cli's for virtual keys
##
File path: hudi-common/src/main/java/org/apache/hudi/common/util/OrcUtils.java
##
@@ -105,6 +107,11 @@
return hoodieKeys;
}
+ @Override
+ public List fetchRecordKeyPartitionPath(Configuration
configuration, Path filePath, BaseKeyGenerator keyGenerator) {
+return null;
Review comment:
Yet to add virtual keys support to Orc base files
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Virutal keys support for COW all operations
> ---
>
> Key: HUDI-2176
> URL: https://issues.apache.org/jira/browse/HUDI-2176
> Project: Apache Hudi
> Issue Type: Improvement
> Components: Writer Core
>Reporter: sivabalan narayanan
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.9.0
>
>
> Virutal keys support for COW all operations
> (merge handle)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hudi] nsivabalan commented on a change in pull request #3282: [HUDI-2176, 2178, 2179] Virtual keys support to COW table
nsivabalan commented on a change in pull request #3282:
URL: https://github.com/apache/hudi/pull/3282#discussion_r670964858
##
File path: hudi-cli/src/main/scala/org/apache/hudi/cli/SparkHelpers.scala
##
@@ -51,7 +51,8 @@ object SparkHelpers {
// Add current classLoad for config, if not will throw classNotFound of
'HoodieWrapperFileSystem'.
parquetConfig.getHadoopConf().setClassLoader(Thread.currentThread.getContextClassLoader)
-val writer = new HoodieParquetWriter[HoodieJsonPayload,
IndexedRecord](instantTime, destinationFile, parquetConfig, schema, new
SparkTaskContextSupplier())
+val writer = new HoodieParquetWriter[HoodieJsonPayload,
IndexedRecord](instantTime, destinationFile, parquetConfig, schema, new
SparkTaskContextSupplier(),
+ true)
Review comment:
yet to fix cli's for virtual keys
##
File path: hudi-common/src/main/java/org/apache/hudi/common/util/OrcUtils.java
##
@@ -105,6 +107,11 @@
return hoodieKeys;
}
+ @Override
+ public List fetchRecordKeyPartitionPath(Configuration
configuration, Path filePath, BaseKeyGenerator keyGenerator) {
+return null;
Review comment:
Yet to add virtual keys support to Orc base files
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HUDI-2170) Always choose the latest record for HoodieRecordPayload
[ https://issues.apache.org/jira/browse/HUDI-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381758#comment-17381758 ] ASF GitHub Bot commented on HUDI-2170: -- codecov-commenter edited a comment on pull request #3267: URL: https://github.com/apache/hudi/pull/3267#issuecomment-878977860 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3267?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3267](https://codecov.io/gh/apache/hudi/pull/3267?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a275c3a) into [master](https://codecov.io/gh/apache/hudi/commit/01ad449ad68d9c77ee1493fcdf833df53df6106a?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (01ad449) will **decrease** coverage by `49.36%`. > The diff coverage is `0.00%`. [](https://codecov.io/gh/apache/hudi/pull/3267?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master #3267 +/- ## - Coverage 52.19% 2.83% -49.37% + Complexity 2659 85 -2574 Files 335 284 -51 Lines 14981 11828 -3153 Branches 1505 981 -524 - Hits 7819 335 -7484 - Misses 6536 11467 +4931 + Partials626 26 -600 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `0.00% <0.00%> (∅)` | | | hudicommon | `?` | | | hudihadoopmr | `?` | | | hudisync | `4.88% <ø> (?)` | | | huditimelineservice | `?` | | | hudiutilities | `9.11% <ø> (-60.54%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3267?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [.../org/apache/hudi/async/AsyncClusteringService.java](https://codecov.io/gh/apache/hudi/pull/3267/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2FzeW5jL0FzeW5jQ2x1c3RlcmluZ1NlcnZpY2UuamF2YQ==) | `0.00% <0.00%> (ø)` | | | [...ava/org/apache/hudi/async/AsyncCompactService.java](https://codecov.io/gh/apache/hudi/pull/3267/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2FzeW5jL0FzeW5jQ29tcGFjdFNlcnZpY2UuamF2YQ==) | `0.00% <0.00%> (ø)` | | | [...java/org/apache/hudi/async/HoodieAsyncService.java](https://codecov.io/gh/apache/hudi/pull/3267/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2FzeW5jL0hvb2RpZUFzeW5jU2VydmljZS5qYXZh) | `0.00% <0.00%> (ø)` | | | [...ient/http/HoodieWriteCommitHttpCallbackClient.java](https://codecov.io/gh/apache/hudi/pull/3267/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NhbGxiYWNrL2NsaWVudC9odHRwL0hvb2RpZVdyaXRlQ29tbWl0SHR0cENhbGxiYWNrQ2xpZW50LmphdmE=) | `0.00% <0.00%> (ø)` | | | [...udi/callback/util/HoodieCommitCallbackFactory.java](https://codecov.io/gh/apache/hudi/pull/3267/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NhbGxiYWNrL3V0aWwvSG9vZGllQ29tbWl0Q2FsbGJhY2tGYWN0b3J5LmphdmE=) | `0.00% <0.00%> (ø)` | | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3267: [HUDI-2170] Always choose the latest record for HoodieRecordPayload
codecov-commenter edited a comment on pull request #3267: URL: https://github.com/apache/hudi/pull/3267#issuecomment-878977860 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3267?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3267](https://codecov.io/gh/apache/hudi/pull/3267?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a275c3a) into [master](https://codecov.io/gh/apache/hudi/commit/01ad449ad68d9c77ee1493fcdf833df53df6106a?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (01ad449) will **decrease** coverage by `49.36%`. > The diff coverage is `0.00%`. [](https://codecov.io/gh/apache/hudi/pull/3267?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master #3267 +/- ## - Coverage 52.19% 2.83% -49.37% + Complexity 2659 85 -2574 Files 335 284 -51 Lines 14981 11828 -3153 Branches 1505 981 -524 - Hits 7819 335 -7484 - Misses 6536 11467 +4931 + Partials626 26 -600 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `0.00% <0.00%> (∅)` | | | hudicommon | `?` | | | hudihadoopmr | `?` | | | hudisync | `4.88% <ø> (?)` | | | huditimelineservice | `?` | | | hudiutilities | `9.11% <ø> (-60.54%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3267?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [.../org/apache/hudi/async/AsyncClusteringService.java](https://codecov.io/gh/apache/hudi/pull/3267/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2FzeW5jL0FzeW5jQ2x1c3RlcmluZ1NlcnZpY2UuamF2YQ==) | `0.00% <0.00%> (ø)` | | | [...ava/org/apache/hudi/async/AsyncCompactService.java](https://codecov.io/gh/apache/hudi/pull/3267/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2FzeW5jL0FzeW5jQ29tcGFjdFNlcnZpY2UuamF2YQ==) | `0.00% <0.00%> (ø)` | | | [...java/org/apache/hudi/async/HoodieAsyncService.java](https://codecov.io/gh/apache/hudi/pull/3267/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2FzeW5jL0hvb2RpZUFzeW5jU2VydmljZS5qYXZh) | `0.00% <0.00%> (ø)` | | | [...ient/http/HoodieWriteCommitHttpCallbackClient.java](https://codecov.io/gh/apache/hudi/pull/3267/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NhbGxiYWNrL2NsaWVudC9odHRwL0hvb2RpZVdyaXRlQ29tbWl0SHR0cENhbGxiYWNrQ2xpZW50LmphdmE=) | `0.00% <0.00%> (ø)` | | | [...udi/callback/util/HoodieCommitCallbackFactory.java](https://codecov.io/gh/apache/hudi/pull/3267/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NhbGxiYWNrL3V0aWwvSG9vZGllQ29tbWl0Q2FsbGJhY2tGYWN0b3J5LmphdmE=) | `0.00% <0.00%> (ø)` | | | [...g/apache/hudi/client/AbstractClusteringClient.java](https://codecov.io/gh/apache/hudi/pull/3267/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC9BYnN0cmFjdENsdXN0ZXJpbmdDbGllbnQuamF2YQ==)
[jira] [Commented] (HUDI-1884) MergeInto Support Partial Update For COW
[
https://issues.apache.org/jira/browse/HUDI-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381756#comment-17381756
]
ASF GitHub Bot commented on HUDI-1884:
--
hudi-bot edited a comment on pull request #3154:
URL: https://github.com/apache/hudi/pull/3154#issuecomment-868360998
## CI report:
* 63cf96134656dbb41eaab4e03e747776599255b6 UNKNOWN
* 82b4b8bef8d143a14d445ae35fa5ec8fbbd5889f UNKNOWN
* f5511851a56f5a23263bfb05690217718a329254 Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=940)
* f5f0fd3f1eb1529643b7129349614ed80d106bed Azure:
[PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=956)
Bot commands
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> MergeInto Support Partial Update For COW
> -
>
> Key: HUDI-1884
> URL: https://issues.apache.org/jira/browse/HUDI-1884
> Project: Apache Hudi
> Issue Type: Sub-task
>Reporter: pengzhiwei
>Assignee: pengzhiwei
>Priority: Major
> Labels: pull-request-available
>
> Support partial update for merge-into statement, just like this:
> {code:java}
> merge into h0
> using s0
> on s0.id = h0.id
> when matched then update set price = s0.price + 10
> when not matched then insert *{code}
> Allow to update partial fields.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hudi] hudi-bot edited a comment on pull request #3154: [HUDI-1884] MergeInto Support Partial Update For COW
hudi-bot edited a comment on pull request #3154: URL: https://github.com/apache/hudi/pull/3154#issuecomment-868360998 ## CI report: * 63cf96134656dbb41eaab4e03e747776599255b6 UNKNOWN * 82b4b8bef8d143a14d445ae35fa5ec8fbbd5889f UNKNOWN * f5511851a56f5a23263bfb05690217718a329254 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=940) * f5f0fd3f1eb1529643b7129349614ed80d106bed Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=956) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HUDI-1884) MergeInto Support Partial Update For COW
[ https://issues.apache.org/jira/browse/HUDI-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381753#comment-17381753 ] ASF GitHub Bot commented on HUDI-1884: -- codecov-commenter edited a comment on pull request #3154: URL: https://github.com/apache/hudi/pull/3154#issuecomment-868445474 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3154?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3154](https://codecov.io/gh/apache/hudi/pull/3154?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (f5f0fd3) into [master](https://codecov.io/gh/apache/hudi/commit/a62a6cff32e7d7f62ffdf6362fb8876497fd0528?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a62a6cf) will **decrease** coverage by `44.87%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/3154?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master #3154 +/- ## - Coverage 47.70% 2.83% -44.88% + Complexity 5550 85 -5465 Files 936 284 -652 Lines 41624 11835-29789 Branches 4188 981 -3207 - Hits 19858 335-19523 + Misses2 11474 -8526 + Partials 1766 26 -1740 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `0.00% <ø> (-34.47%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `4.88% <ø> (-51.10%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `9.11% <ø> (-50.15%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3154?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...va/org/apache/hudi/utilities/IdentitySplitter.java](https://codecov.io/gh/apache/hudi/pull/3154/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0lkZW50aXR5U3BsaXR0ZXIuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...va/org/apache/hudi/utilities/schema/SchemaSet.java](https://codecov.io/gh/apache/hudi/pull/3154/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NjaGVtYS9TY2hlbWFTZXQuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/utilities/sources/RowSource.java](https://codecov.io/gh/apache/hudi/pull/3154/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvUm93U291cmNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/utilities/sources/AvroSource.java](https://codecov.io/gh/apache/hudi/pull/3154/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvQXZyb1NvdXJjZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/utilities/sources/JsonSource.java](https://codecov.io/gh/apache/hudi/pull/3154/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvSnNvblNvdXJjZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3154: [HUDI-1884] MergeInto Support Partial Update For COW
codecov-commenter edited a comment on pull request #3154: URL: https://github.com/apache/hudi/pull/3154#issuecomment-868445474 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3154?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3154](https://codecov.io/gh/apache/hudi/pull/3154?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (f5f0fd3) into [master](https://codecov.io/gh/apache/hudi/commit/a62a6cff32e7d7f62ffdf6362fb8876497fd0528?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a62a6cf) will **decrease** coverage by `44.87%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/3154?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master #3154 +/- ## - Coverage 47.70% 2.83% -44.88% + Complexity 5550 85 -5465 Files 936 284 -652 Lines 41624 11835-29789 Branches 4188 981 -3207 - Hits 19858 335-19523 + Misses2 11474 -8526 + Partials 1766 26 -1740 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `0.00% <ø> (-34.47%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `4.88% <ø> (-51.10%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `9.11% <ø> (-50.15%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3154?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...va/org/apache/hudi/utilities/IdentitySplitter.java](https://codecov.io/gh/apache/hudi/pull/3154/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0lkZW50aXR5U3BsaXR0ZXIuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...va/org/apache/hudi/utilities/schema/SchemaSet.java](https://codecov.io/gh/apache/hudi/pull/3154/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NjaGVtYS9TY2hlbWFTZXQuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/utilities/sources/RowSource.java](https://codecov.io/gh/apache/hudi/pull/3154/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvUm93U291cmNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/utilities/sources/AvroSource.java](https://codecov.io/gh/apache/hudi/pull/3154/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvQXZyb1NvdXJjZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/utilities/sources/JsonSource.java](https://codecov.io/gh/apache/hudi/pull/3154/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvSnNvblNvdXJjZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[jira] [Resolved] (HUDI-1633) Make callback return HoodieWriteStat
[ https://issues.apache.org/jira/browse/HUDI-1633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang resolved HUDI-1633. Resolution: Resolved Resolved via master branch : 3b264e80d9ccc58ebd0dff3b00f50ad9d481e03c > Make callback return HoodieWriteStat > > > Key: HUDI-1633 > URL: https://issues.apache.org/jira/browse/HUDI-1633 > Project: Apache Hudi > Issue Type: Improvement >Reporter: liujinhui >Assignee: liujinhui >Priority: Minor > Labels: pull-request-available, sev:normal > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HUDI-1633) Make callback return HoodieWriteStat
[ https://issues.apache.org/jira/browse/HUDI-1633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381751#comment-17381751 ] ASF GitHub Bot commented on HUDI-1633: -- wangxianghu merged pull request #2445: URL: https://github.com/apache/hudi/pull/2445 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Make callback return HoodieWriteStat > > > Key: HUDI-1633 > URL: https://issues.apache.org/jira/browse/HUDI-1633 > Project: Apache Hudi > Issue Type: Improvement >Reporter: liujinhui >Assignee: liujinhui >Priority: Minor > Labels: pull-request-available, sev:normal > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[hudi] branch master updated (38cd74b -> 3b264e8)
This is an automated email from the ASF dual-hosted git repository. wangxianghu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 38cd74b [MINOR] Allow users to choose ORC as base file format in Spark SQL (#3279) add 3b264e8 [HUDI-1633] Make callback return HoodieWriteStat (#2445) No new revisions were added by this update. Summary of changes: .../hudi/callback/HoodieWriteCommitCallback.java | 5 .../http/HoodieWriteCommitHttpCallbackClient.java | 2 +- .../common/HoodieWriteCommitCallbackMessage.java | 33 +++--- .../impl/HoodieWriteCommitHttpCallback.java| 2 -- .../hudi/client/AbstractHoodieWriteClient.java | 2 +- .../kafka/HoodieWriteCommitKafkaCallback.java | 2 +- 6 files changed, 25 insertions(+), 21 deletions(-)
[GitHub] [hudi] wangxianghu merged pull request #2445: [HUDI-1633] Make callback return HoodieWriteStat
wangxianghu merged pull request #2445: URL: https://github.com/apache/hudi/pull/2445 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HUDI-2170) Always choose the latest record for HoodieRecordPayload
[
https://issues.apache.org/jira/browse/HUDI-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381750#comment-17381750
]
ASF GitHub Bot commented on HUDI-2170:
--
hudi-bot edited a comment on pull request #3267:
URL: https://github.com/apache/hudi/pull/3267#issuecomment-878894177
## CI report:
* 9c2e6e7b0efc3e235a9c736a095b4e0e003f8b1c Azure:
[FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=938)
* a275c3af0519430c6ff0dc5902dd84dc5d1a0882 Azure:
[PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=955)
Bot commands
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Always choose the latest record for HoodieRecordPayload
> ---
>
> Key: HUDI-2170
> URL: https://issues.apache.org/jira/browse/HUDI-2170
> Project: Apache Hudi
> Issue Type: Improvement
> Components: Common Core
>Reporter: Danny Chen
>Assignee: Danny Chen
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.9.0
>
>
> Now in {{OverwriteWithLatestAvroPayload.preCombine}}, we still choose the old
> record when the new record has the same preCombine field with the old one,
> actually it is more natural to keep the new incoming record instead. The
> {{DefaultHoodieRecordPayload.combineAndGetUpdateValue}} method already did
> that.
> See issue: https://github.com/apache/hudi/issues/3266.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hudi] hudi-bot edited a comment on pull request #3267: [HUDI-2170] Always choose the latest record for HoodieRecordPayload
hudi-bot edited a comment on pull request #3267: URL: https://github.com/apache/hudi/pull/3267#issuecomment-878894177 ## CI report: * 9c2e6e7b0efc3e235a9c736a095b4e0e003f8b1c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=938) * a275c3af0519430c6ff0dc5902dd84dc5d1a0882 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=955) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HUDI-2170) Always choose the latest record for HoodieRecordPayload
[
https://issues.apache.org/jira/browse/HUDI-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381748#comment-17381748
]
ASF GitHub Bot commented on HUDI-2170:
--
hudi-bot edited a comment on pull request #3267:
URL: https://github.com/apache/hudi/pull/3267#issuecomment-878894177
## CI report:
* 9c2e6e7b0efc3e235a9c736a095b4e0e003f8b1c Azure:
[FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=938)
* a275c3af0519430c6ff0dc5902dd84dc5d1a0882 UNKNOWN
Bot commands
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Always choose the latest record for HoodieRecordPayload
> ---
>
> Key: HUDI-2170
> URL: https://issues.apache.org/jira/browse/HUDI-2170
> Project: Apache Hudi
> Issue Type: Improvement
> Components: Common Core
>Reporter: Danny Chen
>Assignee: Danny Chen
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.9.0
>
>
> Now in {{OverwriteWithLatestAvroPayload.preCombine}}, we still choose the old
> record when the new record has the same preCombine field with the old one,
> actually it is more natural to keep the new incoming record instead. The
> {{DefaultHoodieRecordPayload.combineAndGetUpdateValue}} method already did
> that.
> See issue: https://github.com/apache/hudi/issues/3266.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hudi] hudi-bot edited a comment on pull request #3267: [HUDI-2170] Always choose the latest record for HoodieRecordPayload
hudi-bot edited a comment on pull request #3267: URL: https://github.com/apache/hudi/pull/3267#issuecomment-878894177 ## CI report: * 9c2e6e7b0efc3e235a9c736a095b4e0e003f8b1c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=938) * a275c3af0519430c6ff0dc5902dd84dc5d1a0882 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HUDI-1860) Add INSERT_OVERWRITE support to DeltaStreamer
[ https://issues.apache.org/jira/browse/HUDI-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381747#comment-17381747 ] ASF GitHub Bot commented on HUDI-1860: -- Samrat002 commented on pull request #3184: URL: https://github.com/apache/hudi/pull/3184#issuecomment-881168606 @vinothchandar please review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add INSERT_OVERWRITE support to DeltaStreamer > - > > Key: HUDI-1860 > URL: https://issues.apache.org/jira/browse/HUDI-1860 > Project: Apache Hudi > Issue Type: Sub-task >Reporter: Sagar Sumit >Assignee: Samrat Deb >Priority: Major > Labels: pull-request-available > Original Estimate: 72h > Remaining Estimate: 72h > > As discussed in [this > RFC|https://cwiki.apache.org/confluence/display/HUDI/RFC+-+14+%3A+JDBC+incremental+puller], > having full fetch mode use the inser_overwrite to write to sync would be > better as it can handle schema changes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hudi] Samrat002 commented on pull request #3184: [HUDI-1860] Add INSERT_OVERWRITE and INSERT_OVERWRITE_TABLE support to DeltaStreamer
Samrat002 commented on pull request #3184: URL: https://github.com/apache/hudi/pull/3184#issuecomment-881168606 @vinothchandar please review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (HUDI-1771) Propagate CDC format for hoodie
[ https://issues.apache.org/jira/browse/HUDI-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng yunhong reassigned HUDI-1771: --- Assignee: Zheng yunhong (was: Danny Chen) > Propagate CDC format for hoodie > --- > > Key: HUDI-1771 > URL: https://issues.apache.org/jira/browse/HUDI-1771 > Project: Apache Hudi > Issue Type: New Feature > Components: Flink Integration >Reporter: Danny Chen >Assignee: Zheng yunhong >Priority: Major > Labels: pull-request-available, sev:normal > Fix For: 0.9.0 > > > Like what we discussed in the dev mailing list: > https://lists.apache.org/thread.html/r31b2d1404e4e043a5f875b78105ba6f9a801e78f265ad91242ad5eb2%40%3Cdev.hudi.apache.org%3E > Keep the change flags make new use cases possible: using HUDI as the unified > storage format for DWD and DWS layer. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HUDI-1771) Propagate CDC format for hoodie
[ https://issues.apache.org/jira/browse/HUDI-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381746#comment-17381746 ] ASF GitHub Bot commented on HUDI-1771: -- codecov-commenter edited a comment on pull request #3285: URL: https://github.com/apache/hudi/pull/3285#issuecomment-881152765 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3285?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3285](https://codecov.io/gh/apache/hudi/pull/3285?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a517fd1) into [master](https://codecov.io/gh/apache/hudi/commit/a62a6cff32e7d7f62ffdf6362fb8876497fd0528?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a62a6cf) will **decrease** coverage by `20.36%`. > The diff coverage is `50.00%`. [](https://codecov.io/gh/apache/hudi/pull/3285?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3285 +/- ## = - Coverage 47.70% 27.33% -20.37% + Complexity 5550 1294 -4256 = Files 936 386 -550 Lines 4162415355-26269 Branches 4188 1341 -2847 = - Hits 19858 4198-15660 + Misses210852 -9148 + Partials 1766 305 -1461 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `20.96% <50.00%> (-13.51%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `4.88% <ø> (-51.10%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `59.23% <ø> (-0.04%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3285?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...in/java/org/apache/hudi/io/HoodieAppendHandle.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2lvL0hvb2RpZUFwcGVuZEhhbmRsZS5qYXZh) | `0.00% <0.00%> (ø)` | | | [...in/java/org/apache/hudi/io/HoodieCreateHandle.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2lvL0hvb2RpZUNyZWF0ZUhhbmRsZS5qYXZh) | `0.00% <0.00%> (ø)` | | | [...ain/java/org/apache/hudi/io/HoodieMergeHandle.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2lvL0hvb2RpZU1lcmdlSGFuZGxlLmphdmE=) | `0.00% <0.00%> (ø)` | | | [...rg/apache/hudi/client/model/HoodieInternalRow.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L21vZGVsL0hvb2RpZUludGVybmFsUm93LmphdmE=) | `79.48% <87.50%> (+1.70%)` | :arrow_up: | | [...java/org/apache/hudi/io/HoodieRowCreateHandle.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvaW8vSG9vZGllUm93Q3JlYXRlSGFuZGxlLmphdmE=) | `92.77% <100.00%> (+0.27%)` | :arrow_up: | |
[hudi] branch master updated (a62a6cf -> 38cd74b)
This is an automated email from the ASF dual-hosted git repository. leesf pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from a62a6cf [MINOR] Refactor hive sync tool to reduce duplicate code (#3276) add 38cd74b [MINOR] Allow users to choose ORC as base file format in Spark SQL (#3279) No new revisions were added by this update. Summary of changes: .../org/apache/hudi/config/HoodieWriteConfig.java | 7 +++ .../org/apache/hudi/HoodieSparkSqlWriter.scala | 2 ++ .../hudi/functional/HoodieSparkSqlWriterSuite.scala | 21 +++-- 3 files changed, 24 insertions(+), 6 deletions(-)
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3285: [HUDI-1771] Propagate CDC format for hoodie
codecov-commenter edited a comment on pull request #3285: URL: https://github.com/apache/hudi/pull/3285#issuecomment-881152765 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3285?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3285](https://codecov.io/gh/apache/hudi/pull/3285?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a517fd1) into [master](https://codecov.io/gh/apache/hudi/commit/a62a6cff32e7d7f62ffdf6362fb8876497fd0528?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a62a6cf) will **decrease** coverage by `20.36%`. > The diff coverage is `50.00%`. [](https://codecov.io/gh/apache/hudi/pull/3285?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3285 +/- ## = - Coverage 47.70% 27.33% -20.37% + Complexity 5550 1294 -4256 = Files 936 386 -550 Lines 4162415355-26269 Branches 4188 1341 -2847 = - Hits 19858 4198-15660 + Misses210852 -9148 + Partials 1766 305 -1461 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `20.96% <50.00%> (-13.51%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `4.88% <ø> (-51.10%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `59.23% <ø> (-0.04%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3285?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...in/java/org/apache/hudi/io/HoodieAppendHandle.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2lvL0hvb2RpZUFwcGVuZEhhbmRsZS5qYXZh) | `0.00% <0.00%> (ø)` | | | [...in/java/org/apache/hudi/io/HoodieCreateHandle.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2lvL0hvb2RpZUNyZWF0ZUhhbmRsZS5qYXZh) | `0.00% <0.00%> (ø)` | | | [...ain/java/org/apache/hudi/io/HoodieMergeHandle.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2lvL0hvb2RpZU1lcmdlSGFuZGxlLmphdmE=) | `0.00% <0.00%> (ø)` | | | [...rg/apache/hudi/client/model/HoodieInternalRow.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L21vZGVsL0hvb2RpZUludGVybmFsUm93LmphdmE=) | `79.48% <87.50%> (+1.70%)` | :arrow_up: | | [...java/org/apache/hudi/io/HoodieRowCreateHandle.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvaW8vSG9vZGllUm93Q3JlYXRlSGFuZGxlLmphdmE=) | `92.77% <100.00%> (+0.27%)` | :arrow_up: | |
[GitHub] [hudi] leesf merged pull request #3279: [MINOR] Allow users to choose ORC as base file format in Spark SQL
leesf merged pull request #3279: URL: https://github.com/apache/hudi/pull/3279 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HUDI-1860) Add INSERT_OVERWRITE support to DeltaStreamer
[ https://issues.apache.org/jira/browse/HUDI-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381744#comment-17381744 ] ASF GitHub Bot commented on HUDI-1860: -- codecov-commenter edited a comment on pull request #3184: URL: https://github.com/apache/hudi/pull/3184#issuecomment-870526141 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3184?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3184](https://codecov.io/gh/apache/hudi/pull/3184?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (55947cf) into [master](https://codecov.io/gh/apache/hudi/commit/d024439764ceeca6366cb33689b729a1c69a6272?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (d024439) will **increase** coverage by `3.69%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/hudi/pull/3184?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3184 +/- ## + Coverage 44.10% 47.80% +3.69% - Complexity 5157 5570 +413 Files 936 936 Lines 4162941632 +3 Branches 4189 4188 -1 + Hits 1836219903+1541 + Misses2163819960-1678 - Partials 1629 1769 +140 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `39.97% <ø> (ø)` | | | hudiclient | `34.46% <ø> (ø)` | | | hudicommon | `48.67% <ø> (-0.02%)` | :arrow_down: | | hudiflink | `59.68% <ø> (ø)` | | | hudihadoopmr | `52.02% <ø> (ø)` | | | hudisparkdatasource | `67.35% <ø> (+0.13%)` | :arrow_up: | | hudisync | `55.97% <ø> (+0.24%)` | :arrow_up: | | huditimelineservice | `64.07% <ø> (ø)` | | | hudiutilities | `59.37% <100.00%> (+50.25%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3184?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...apache/hudi/utilities/deltastreamer/DeltaSync.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL2RlbHRhc3RyZWFtZXIvRGVsdGFTeW5jLmphdmE=) | `72.29% <100.00%> (+72.29%)` | :arrow_up: | | [...e/hudi/common/table/log/HoodieLogFormatWriter.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL2xvZy9Ib29kaWVMb2dGb3JtYXRXcml0ZXIuamF2YQ==) | `78.12% <0.00%> (-1.57%)` | :arrow_down: | | [...c/main/java/org/apache/hudi/hive/HiveSyncTool.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvSGl2ZVN5bmNUb29sLmphdmE=) | `78.12% <0.00%> (+0.27%)` | :arrow_up: | | [...in/scala/org/apache/hudi/HoodieStreamingSink.scala](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9odWRpL0hvb2RpZVN0cmVhbWluZ1Npbmsuc2NhbGE=) | `32.00% <0.00%> (+4.00%)` | :arrow_up: | | [.../apache/hudi/utilities/HoodieSnapshotExporter.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVNuYXBzaG90RXhwb3J0ZXIuamF2YQ==) |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3184: [HUDI-1860] Add INSERT_OVERWRITE and INSERT_OVERWRITE_TABLE support to DeltaStreamer
codecov-commenter edited a comment on pull request #3184: URL: https://github.com/apache/hudi/pull/3184#issuecomment-870526141 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3184?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3184](https://codecov.io/gh/apache/hudi/pull/3184?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (55947cf) into [master](https://codecov.io/gh/apache/hudi/commit/d024439764ceeca6366cb33689b729a1c69a6272?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (d024439) will **increase** coverage by `3.69%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/hudi/pull/3184?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3184 +/- ## + Coverage 44.10% 47.80% +3.69% - Complexity 5157 5570 +413 Files 936 936 Lines 4162941632 +3 Branches 4189 4188 -1 + Hits 1836219903+1541 + Misses2163819960-1678 - Partials 1629 1769 +140 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `39.97% <ø> (ø)` | | | hudiclient | `34.46% <ø> (ø)` | | | hudicommon | `48.67% <ø> (-0.02%)` | :arrow_down: | | hudiflink | `59.68% <ø> (ø)` | | | hudihadoopmr | `52.02% <ø> (ø)` | | | hudisparkdatasource | `67.35% <ø> (+0.13%)` | :arrow_up: | | hudisync | `55.97% <ø> (+0.24%)` | :arrow_up: | | huditimelineservice | `64.07% <ø> (ø)` | | | hudiutilities | `59.37% <100.00%> (+50.25%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3184?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...apache/hudi/utilities/deltastreamer/DeltaSync.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL2RlbHRhc3RyZWFtZXIvRGVsdGFTeW5jLmphdmE=) | `72.29% <100.00%> (+72.29%)` | :arrow_up: | | [...e/hudi/common/table/log/HoodieLogFormatWriter.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL2xvZy9Ib29kaWVMb2dGb3JtYXRXcml0ZXIuamF2YQ==) | `78.12% <0.00%> (-1.57%)` | :arrow_down: | | [...c/main/java/org/apache/hudi/hive/HiveSyncTool.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvSGl2ZVN5bmNUb29sLmphdmE=) | `78.12% <0.00%> (+0.27%)` | :arrow_up: | | [...in/scala/org/apache/hudi/HoodieStreamingSink.scala](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9odWRpL0hvb2RpZVN0cmVhbWluZ1Npbmsuc2NhbGE=) | `32.00% <0.00%> (+4.00%)` | :arrow_up: | | [.../apache/hudi/utilities/HoodieSnapshotExporter.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVNuYXBzaG90RXhwb3J0ZXIuamF2YQ==) | `88.79% <0.00%> (+5.17%)` | :arrow_up: | |
[jira] [Commented] (HUDI-2145) Create new bucket when NewFileAssignState filled
[ https://issues.apache.org/jira/browse/HUDI-2145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381743#comment-17381743 ] ASF GitHub Bot commented on HUDI-2145: -- codecov-commenter edited a comment on pull request #3258: URL: https://github.com/apache/hudi/pull/3258#issuecomment-878089095 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3258?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3258](https://codecov.io/gh/apache/hudi/pull/3258?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (d2184c0) into [master](https://codecov.io/gh/apache/hudi/commit/5804ad8e32ae05758ebc5e47f5d4fb4db371ab52?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (5804ad8) will **increase** coverage by `0.08%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/hudi/pull/3258?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3258 +/- ## + Coverage 47.72% 47.81% +0.08% - Complexity 5528 5569 +41 Files 934 936 +2 Lines 4145741626 +169 Branches 4166 4188 +22 + Hits 1978619902 +116 - Misses1991419957 +43 - Partials 1757 1767 +10 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `39.97% <ø> (ø)` | | | hudiclient | `34.46% <ø> (+0.01%)` | :arrow_up: | | hudicommon | `48.69% <ø> (+0.13%)` | :arrow_up: | | hudiflink | `59.73% <100.00%> (-0.31%)` | :arrow_down: | | hudihadoopmr | `52.02% <ø> (+0.73%)` | :arrow_up: | | hudisparkdatasource | `67.41% <ø> (-0.26%)` | :arrow_down: | | hudisync | `55.97% <ø> (+1.46%)` | :arrow_up: | | huditimelineservice | `64.07% <ø> (ø)` | | | hudiutilities | `59.26% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3258?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...g/apache/hudi/sink/partitioner/BucketAssigner.java](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS9zaW5rL3BhcnRpdGlvbmVyL0J1Y2tldEFzc2lnbmVyLmphdmE=) | `84.69% <100.00%> (+1.36%)` | :arrow_up: | | [.../java/org/apache/hudi/client/HoodieReadClient.java](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L0hvb2RpZVJlYWRDbGllbnQuamF2YQ==) | `94.64% <0.00%> (-5.36%)` | :arrow_down: | | [...in/scala/org/apache/hudi/HoodieStreamingSink.scala](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9odWRpL0hvb2RpZVN0cmVhbWluZ1Npbmsuc2NhbGE=) | `33.60% <0.00%> (-4.80%)` | :arrow_down: | | [...c/main/java/org/apache/hudi/util/StreamerUtil.java](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS91dGlsL1N0cmVhbWVyVXRpbC5qYXZh) | `64.00% <0.00%> (-3.80%)` | :arrow_down: | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3258: [HUDI-2145] Create new bucket when NewFileAssignState filled
codecov-commenter edited a comment on pull request #3258: URL: https://github.com/apache/hudi/pull/3258#issuecomment-878089095 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3258?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3258](https://codecov.io/gh/apache/hudi/pull/3258?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (d2184c0) into [master](https://codecov.io/gh/apache/hudi/commit/5804ad8e32ae05758ebc5e47f5d4fb4db371ab52?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (5804ad8) will **increase** coverage by `0.08%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/hudi/pull/3258?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3258 +/- ## + Coverage 47.72% 47.81% +0.08% - Complexity 5528 5569 +41 Files 934 936 +2 Lines 4145741626 +169 Branches 4166 4188 +22 + Hits 1978619902 +116 - Misses1991419957 +43 - Partials 1757 1767 +10 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `39.97% <ø> (ø)` | | | hudiclient | `34.46% <ø> (+0.01%)` | :arrow_up: | | hudicommon | `48.69% <ø> (+0.13%)` | :arrow_up: | | hudiflink | `59.73% <100.00%> (-0.31%)` | :arrow_down: | | hudihadoopmr | `52.02% <ø> (+0.73%)` | :arrow_up: | | hudisparkdatasource | `67.41% <ø> (-0.26%)` | :arrow_down: | | hudisync | `55.97% <ø> (+1.46%)` | :arrow_up: | | huditimelineservice | `64.07% <ø> (ø)` | | | hudiutilities | `59.26% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3258?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...g/apache/hudi/sink/partitioner/BucketAssigner.java](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS9zaW5rL3BhcnRpdGlvbmVyL0J1Y2tldEFzc2lnbmVyLmphdmE=) | `84.69% <100.00%> (+1.36%)` | :arrow_up: | | [.../java/org/apache/hudi/client/HoodieReadClient.java](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L0hvb2RpZVJlYWRDbGllbnQuamF2YQ==) | `94.64% <0.00%> (-5.36%)` | :arrow_down: | | [...in/scala/org/apache/hudi/HoodieStreamingSink.scala](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9odWRpL0hvb2RpZVN0cmVhbWluZ1Npbmsuc2NhbGE=) | `33.60% <0.00%> (-4.80%)` | :arrow_down: | | [...c/main/java/org/apache/hudi/util/StreamerUtil.java](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS91dGlsL1N0cmVhbWVyVXRpbC5qYXZh) | `64.00% <0.00%> (-3.80%)` | :arrow_down: | | [.../hudi/common/util/collection/LazyFileIterable.java](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3V0aWwvY29sbGVjdGlvbi9MYXp5RmlsZUl0ZXJhYmxlLmphdmE=) | `71.73% <0.00%> (-2.68%)` | :arrow_down: | |
[jira] [Commented] (HUDI-1633) Make callback return HoodieWriteStat
[ https://issues.apache.org/jira/browse/HUDI-1633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381740#comment-17381740 ] ASF GitHub Bot commented on HUDI-1633: -- codecov-commenter edited a comment on pull request #2445: URL: https://github.com/apache/hudi/pull/2445#issuecomment-881149916 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2445?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#2445](https://codecov.io/gh/apache/hudi/pull/2445?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (1068b7f) into [master](https://codecov.io/gh/apache/hudi/commit/e3d3677b7e7899705b624925666317f0c074f7c7?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (e3d3677) will **increase** coverage by `4.53%`. > The diff coverage is `0.00%`. [](https://codecov.io/gh/apache/hudi/pull/2445?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#2445 +/- ## + Coverage 50.73% 55.26% +4.53% - Complexity 3064 4364+1300 Files 419 548 +129 Lines 1879727555+8758 Branches 1922 3093+1171 + Hits 953615229+5693 - Misses 848710863+2376 - Partials774 1463 +689 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `39.97% <ø> (+2.71%)` | :arrow_up: | | hudiclient | `∅ <ø> (∅)` | | | hudicommon | `49.31% <ø> (-2.70%)` | :arrow_down: | | hudiflink | `59.68% <ø> (+49.48%)` | :arrow_up: | | hudihadoopmr | `52.08% <ø> (+19.01%)` | :arrow_up: | | hudisparkdatasource | `67.21% <ø> (+1.31%)` | :arrow_up: | | hudisync | `55.97% <ø> (+7.36%)` | :arrow_up: | | huditimelineservice | `64.07% <ø> (-2.77%)` | :arrow_down: | | hudiutilities | `72.84% <0.00%> (+3.36%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/2445?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...callback/kafka/HoodieWriteCommitKafkaCallback.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL2NhbGxiYWNrL2thZmthL0hvb2RpZVdyaXRlQ29tbWl0S2Fma2FDYWxsYmFjay5qYXZh) | `0.00% <0.00%> (ø)` | | | [...e/hudi/common/util/collection/RocksDBBasedMap.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3V0aWwvY29sbGVjdGlvbi9Sb2Nrc0RCQmFzZWRNYXAuamF2YQ==) | `0.00% <0.00%> (-39.29%)` | :arrow_down: | | [.../java/org/apache/hudi/common/util/CommitUtils.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3V0aWwvQ29tbWl0VXRpbHMuamF2YQ==) | `40.47% <0.00%> (-31.53%)` | :arrow_down: | | [...g/apache/hudi/utilities/schema/SchemaProvider.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NjaGVtYS9TY2hlbWFQcm92aWRlci5qYXZh) | `71.42% <0.00%> (-28.58%)` | :arrow_down: | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #2445: [HUDI-1633] Make callback return HoodieWriteStat
codecov-commenter edited a comment on pull request #2445: URL: https://github.com/apache/hudi/pull/2445#issuecomment-881149916 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2445?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#2445](https://codecov.io/gh/apache/hudi/pull/2445?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (1068b7f) into [master](https://codecov.io/gh/apache/hudi/commit/e3d3677b7e7899705b624925666317f0c074f7c7?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (e3d3677) will **increase** coverage by `4.53%`. > The diff coverage is `0.00%`. [](https://codecov.io/gh/apache/hudi/pull/2445?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#2445 +/- ## + Coverage 50.73% 55.26% +4.53% - Complexity 3064 4364+1300 Files 419 548 +129 Lines 1879727555+8758 Branches 1922 3093+1171 + Hits 953615229+5693 - Misses 848710863+2376 - Partials774 1463 +689 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `39.97% <ø> (+2.71%)` | :arrow_up: | | hudiclient | `∅ <ø> (∅)` | | | hudicommon | `49.31% <ø> (-2.70%)` | :arrow_down: | | hudiflink | `59.68% <ø> (+49.48%)` | :arrow_up: | | hudihadoopmr | `52.08% <ø> (+19.01%)` | :arrow_up: | | hudisparkdatasource | `67.21% <ø> (+1.31%)` | :arrow_up: | | hudisync | `55.97% <ø> (+7.36%)` | :arrow_up: | | huditimelineservice | `64.07% <ø> (-2.77%)` | :arrow_down: | | hudiutilities | `72.84% <0.00%> (+3.36%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/2445?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...callback/kafka/HoodieWriteCommitKafkaCallback.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL2NhbGxiYWNrL2thZmthL0hvb2RpZVdyaXRlQ29tbWl0S2Fma2FDYWxsYmFjay5qYXZh) | `0.00% <0.00%> (ø)` | | | [...e/hudi/common/util/collection/RocksDBBasedMap.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3V0aWwvY29sbGVjdGlvbi9Sb2Nrc0RCQmFzZWRNYXAuamF2YQ==) | `0.00% <0.00%> (-39.29%)` | :arrow_down: | | [.../java/org/apache/hudi/common/util/CommitUtils.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3V0aWwvQ29tbWl0VXRpbHMuamF2YQ==) | `40.47% <0.00%> (-31.53%)` | :arrow_down: | | [...g/apache/hudi/utilities/schema/SchemaProvider.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NjaGVtYS9TY2hlbWFQcm92aWRlci5qYXZh) | `71.42% <0.00%> (-28.58%)` | :arrow_down: | | [...java/org/apache/hudi/common/util/CleanerUtils.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3V0aWwvQ2xlYW5lclV0aWxzLmphdmE=) | `47.72% <0.00%> (-15.91%)` | :arrow_down: | |
[jira] [Commented] (HUDI-2184) Support setting hive sync partition extractor class based on flink configuration
[ https://issues.apache.org/jira/browse/HUDI-2184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381736#comment-17381736 ] ASF GitHub Bot commented on HUDI-2184: -- codecov-commenter edited a comment on pull request #3284: URL: https://github.com/apache/hudi/pull/3284#issuecomment-881148186 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3284](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (66d9b95) into [master](https://codecov.io/gh/apache/hudi/commit/a62a6cff32e7d7f62ffdf6362fb8876497fd0528?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a62a6cf) will **decrease** coverage by `3.93%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3284 +/- ## - Coverage 47.70% 43.77% -3.94% + Complexity 5550 4258-1292 Files 936 768 -168 Lines 4162431976-9648 Branches 4188 2864-1324 - Hits 1985813996-5862 + Misses216947-3053 + Partials 1766 1033 -733 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `39.97% <ø> (ø)` | | | hudiclient | `34.46% <ø> (ø)` | | | hudicommon | `48.69% <ø> (+0.31%)` | :arrow_up: | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `55.97% <ø> (ø)` | | | huditimelineservice | `64.07% <ø> (ø)` | | | hudiutilities | `59.26% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...ecution/datasources/Spark3ParsePartitionUtil.scala](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsvc3FsL2V4ZWN1dGlvbi9kYXRhc291cmNlcy9TcGFyazNQYXJzZVBhcnRpdGlvblV0aWwuc2NhbGE=) | | | | [...in/scala/org/apache/hudi/IncrementalRelation.scala](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9odWRpL0luY3JlbWVudGFsUmVsYXRpb24uc2NhbGE=) | | | | [...e/hudi/table/format/mor/MergeOnReadInputSplit.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS90YWJsZS9mb3JtYXQvbW9yL01lcmdlT25SZWFkSW5wdXRTcGxpdC5qYXZh) | | | | [.../hudi/hadoop/realtime/HoodieRealtimeFileSplit.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1oYWRvb3AtbXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvaGFkb29wL3JlYWx0aW1lL0hvb2RpZVJlYWx0aW1lRmlsZVNwbGl0LmphdmE=) | | | | [...e/hudi/hadoop/FileStatusWithBootstrapBaseFile.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1oYWRvb3AtbXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvaGFkb29wL0ZpbGVTdGF0dXNXaXRoQm9vdHN0cmFwQmFzZUZpbGUuamF2YQ==) | | | |
[jira] [Commented] (HUDI-2184) Support setting hive sync partition extractor class based on flink configuration
[ https://issues.apache.org/jira/browse/HUDI-2184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381737#comment-17381737 ] ASF GitHub Bot commented on HUDI-2184: -- codecov-commenter edited a comment on pull request #3284: URL: https://github.com/apache/hudi/pull/3284#issuecomment-881148186 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3284](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (66d9b95) into [master](https://codecov.io/gh/apache/hudi/commit/a62a6cff32e7d7f62ffdf6362fb8876497fd0528?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a62a6cf) will **increase** coverage by `0.09%`. > The diff coverage is `75.00%`. [](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3284 +/- ## + Coverage 47.70% 47.80% +0.09% - Complexity 5550 5568 +18 Files 936 936 Lines 4162441628 +4 Branches 4188 4188 + Hits 1985819900 +42 + Misses219959 -41 - Partials 1766 1769 +3 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `39.97% <ø> (ø)` | | | hudiclient | `34.46% <ø> (ø)` | | | hudicommon | `48.69% <ø> (+0.31%)` | :arrow_up: | | hudiflink | `59.70% <75.00%> (+0.01%)` | :arrow_up: | | hudihadoopmr | `52.02% <ø> (ø)` | | | hudisparkdatasource | `67.35% <ø> (+0.08%)` | :arrow_up: | | hudisync | `55.97% <ø> (ø)` | | | huditimelineservice | `64.07% <ø> (ø)` | | | hudiutilities | `59.26% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...java/org/apache/hudi/table/HoodieTableFactory.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS90YWJsZS9Ib29kaWVUYWJsZUZhY3RvcnkuamF2YQ==) | `84.14% <75.00%> (-0.47%)` | :arrow_down: | | [...in/scala/org/apache/hudi/HoodieStreamingSink.scala](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9odWRpL0hvb2RpZVN0cmVhbWluZ1Npbmsuc2NhbGE=) | `32.00% <0.00%> (+2.39%)` | :arrow_up: | | [...org/apache/hudi/common/config/TypedProperties.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL2NvbmZpZy9UeXBlZFByb3BlcnRpZXMuamF2YQ==) | `87.09% <0.00%> (+12.90%)` | :arrow_up: | | [...hudi/common/config/DFSPropertiesConfiguration.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL2NvbmZpZy9ERlNQcm9wZXJ0aWVzQ29uZmlndXJhdGlvbi5qYXZh) | `78.04% <0.00%> (+78.04%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=continue_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation). > **Legend** - [Click here to learn
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3284: [HUDI-2184] Support setting hive sync partition extractor class based on flink co…
codecov-commenter edited a comment on pull request #3284: URL: https://github.com/apache/hudi/pull/3284#issuecomment-881148186 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3284](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (66d9b95) into [master](https://codecov.io/gh/apache/hudi/commit/a62a6cff32e7d7f62ffdf6362fb8876497fd0528?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a62a6cf) will **increase** coverage by `0.09%`. > The diff coverage is `75.00%`. [](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3284 +/- ## + Coverage 47.70% 47.80% +0.09% - Complexity 5550 5568 +18 Files 936 936 Lines 4162441628 +4 Branches 4188 4188 + Hits 1985819900 +42 + Misses219959 -41 - Partials 1766 1769 +3 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `39.97% <ø> (ø)` | | | hudiclient | `34.46% <ø> (ø)` | | | hudicommon | `48.69% <ø> (+0.31%)` | :arrow_up: | | hudiflink | `59.70% <75.00%> (+0.01%)` | :arrow_up: | | hudihadoopmr | `52.02% <ø> (ø)` | | | hudisparkdatasource | `67.35% <ø> (+0.08%)` | :arrow_up: | | hudisync | `55.97% <ø> (ø)` | | | huditimelineservice | `64.07% <ø> (ø)` | | | hudiutilities | `59.26% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...java/org/apache/hudi/table/HoodieTableFactory.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS90YWJsZS9Ib29kaWVUYWJsZUZhY3RvcnkuamF2YQ==) | `84.14% <75.00%> (-0.47%)` | :arrow_down: | | [...in/scala/org/apache/hudi/HoodieStreamingSink.scala](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9odWRpL0hvb2RpZVN0cmVhbWluZ1Npbmsuc2NhbGE=) | `32.00% <0.00%> (+2.39%)` | :arrow_up: | | [...org/apache/hudi/common/config/TypedProperties.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL2NvbmZpZy9UeXBlZFByb3BlcnRpZXMuamF2YQ==) | `87.09% <0.00%> (+12.90%)` | :arrow_up: | | [...hudi/common/config/DFSPropertiesConfiguration.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL2NvbmZpZy9ERlNQcm9wZXJ0aWVzQ29uZmlndXJhdGlvbi5qYXZh) | `78.04% <0.00%> (+78.04%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=continue_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3284: [HUDI-2184] Support setting hive sync partition extractor class based on flink co…
codecov-commenter edited a comment on pull request #3284: URL: https://github.com/apache/hudi/pull/3284#issuecomment-881148186 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3284](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (66d9b95) into [master](https://codecov.io/gh/apache/hudi/commit/a62a6cff32e7d7f62ffdf6362fb8876497fd0528?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a62a6cf) will **decrease** coverage by `3.93%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3284 +/- ## - Coverage 47.70% 43.77% -3.94% + Complexity 5550 4258-1292 Files 936 768 -168 Lines 4162431976-9648 Branches 4188 2864-1324 - Hits 1985813996-5862 + Misses216947-3053 + Partials 1766 1033 -733 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `39.97% <ø> (ø)` | | | hudiclient | `34.46% <ø> (ø)` | | | hudicommon | `48.69% <ø> (+0.31%)` | :arrow_up: | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `55.97% <ø> (ø)` | | | huditimelineservice | `64.07% <ø> (ø)` | | | hudiutilities | `59.26% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...ecution/datasources/Spark3ParsePartitionUtil.scala](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsvc3FsL2V4ZWN1dGlvbi9kYXRhc291cmNlcy9TcGFyazNQYXJzZVBhcnRpdGlvblV0aWwuc2NhbGE=) | | | | [...in/scala/org/apache/hudi/IncrementalRelation.scala](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9odWRpL0luY3JlbWVudGFsUmVsYXRpb24uc2NhbGE=) | | | | [...e/hudi/table/format/mor/MergeOnReadInputSplit.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS90YWJsZS9mb3JtYXQvbW9yL01lcmdlT25SZWFkSW5wdXRTcGxpdC5qYXZh) | | | | [.../hudi/hadoop/realtime/HoodieRealtimeFileSplit.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1oYWRvb3AtbXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvaGFkb29wL3JlYWx0aW1lL0hvb2RpZVJlYWx0aW1lRmlsZVNwbGl0LmphdmE=) | | | | [...e/hudi/hadoop/FileStatusWithBootstrapBaseFile.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1oYWRvb3AtbXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvaGFkb29wL0ZpbGVTdGF0dXNXaXRoQm9vdHN0cmFwQmFzZUZpbGUuamF2YQ==) | | | | [...n/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9odWRpL0hvb2RpZU1lcmdlT25SZWFkUkRELnNjYWxh) | | | |
[jira] [Commented] (HUDI-2145) Create new bucket when NewFileAssignState filled
[ https://issues.apache.org/jira/browse/HUDI-2145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381735#comment-17381735 ] ASF GitHub Bot commented on HUDI-2145: -- codecov-commenter edited a comment on pull request #3258: URL: https://github.com/apache/hudi/pull/3258#issuecomment-878089095 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3258?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3258](https://codecov.io/gh/apache/hudi/pull/3258?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (d2184c0) into [master](https://codecov.io/gh/apache/hudi/commit/5804ad8e32ae05758ebc5e47f5d4fb4db371ab52?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (5804ad8) will **increase** coverage by `0.06%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/hudi/pull/3258?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3258 +/- ## + Coverage 47.72% 47.79% +0.06% - Complexity 5528 5567 +39 Files 934 936 +2 Lines 4145741626 +169 Branches 4166 4188 +22 + Hits 1978619895 +109 - Misses1991419967 +53 - Partials 1757 1764 +7 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `39.97% <ø> (ø)` | | | hudiclient | `34.46% <ø> (+0.01%)` | :arrow_up: | | hudicommon | `48.69% <ø> (+0.13%)` | :arrow_up: | | hudiflink | `59.73% <100.00%> (-0.31%)` | :arrow_down: | | hudihadoopmr | `52.02% <ø> (+0.73%)` | :arrow_up: | | hudisparkdatasource | `67.21% <ø> (-0.45%)` | :arrow_down: | | hudisync | `55.97% <ø> (+1.46%)` | :arrow_up: | | huditimelineservice | `64.07% <ø> (ø)` | | | hudiutilities | `59.26% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3258?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...g/apache/hudi/sink/partitioner/BucketAssigner.java](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS9zaW5rL3BhcnRpdGlvbmVyL0J1Y2tldEFzc2lnbmVyLmphdmE=) | `84.69% <100.00%> (+1.36%)` | :arrow_up: | | [...in/scala/org/apache/hudi/HoodieStreamingSink.scala](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9odWRpL0hvb2RpZVN0cmVhbWluZ1Npbmsuc2NhbGE=) | `28.00% <0.00%> (-10.40%)` | :arrow_down: | | [.../java/org/apache/hudi/client/HoodieReadClient.java](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L0hvb2RpZVJlYWRDbGllbnQuamF2YQ==) | `94.64% <0.00%> (-5.36%)` | :arrow_down: | | [...c/main/java/org/apache/hudi/util/StreamerUtil.java](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS91dGlsL1N0cmVhbWVyVXRpbC5qYXZh) | `64.00% <0.00%> (-3.80%)` | :arrow_down: | |
[jira] [Commented] (HUDI-1860) Add INSERT_OVERWRITE support to DeltaStreamer
[ https://issues.apache.org/jira/browse/HUDI-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381734#comment-17381734 ] ASF GitHub Bot commented on HUDI-1860: -- codecov-commenter edited a comment on pull request #3184: URL: https://github.com/apache/hudi/pull/3184#issuecomment-870526141 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3184?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3184](https://codecov.io/gh/apache/hudi/pull/3184?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (55947cf) into [master](https://codecov.io/gh/apache/hudi/commit/d024439764ceeca6366cb33689b729a1c69a6272?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (d024439) will **decrease** coverage by `16.75%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/hudi/pull/3184?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3184 +/- ## = - Coverage 44.10% 27.35% -16.76% + Complexity 5157 1294 -3863 = Files 936 386 -550 Lines 4162915346-26283 Branches 4189 1338 -2851 = - Hits 18362 4198-14164 + Misses2163810844-10794 + Partials 1629 304 -1325 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `20.91% <ø> (-13.56%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `4.88% <ø> (-50.86%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `59.37% <100.00%> (+50.25%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3184?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...apache/hudi/utilities/deltastreamer/DeltaSync.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL2RlbHRhc3RyZWFtZXIvRGVsdGFTeW5jLmphdmE=) | `72.29% <100.00%> (+72.29%)` | :arrow_up: | | [...main/java/org/apache/hudi/metrics/HoodieGauge.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvSG9vZGllR2F1Z2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../java/org/apache/hudi/metrics/MetricsReporter.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/metrics/MetricsReporterType.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyVHlwZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3258: [HUDI-2145] Create new bucket when NewFileAssignState filled
codecov-commenter edited a comment on pull request #3258: URL: https://github.com/apache/hudi/pull/3258#issuecomment-878089095 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3258?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3258](https://codecov.io/gh/apache/hudi/pull/3258?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (d2184c0) into [master](https://codecov.io/gh/apache/hudi/commit/5804ad8e32ae05758ebc5e47f5d4fb4db371ab52?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (5804ad8) will **increase** coverage by `0.06%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/hudi/pull/3258?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3258 +/- ## + Coverage 47.72% 47.79% +0.06% - Complexity 5528 5567 +39 Files 934 936 +2 Lines 4145741626 +169 Branches 4166 4188 +22 + Hits 1978619895 +109 - Misses1991419967 +53 - Partials 1757 1764 +7 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `39.97% <ø> (ø)` | | | hudiclient | `34.46% <ø> (+0.01%)` | :arrow_up: | | hudicommon | `48.69% <ø> (+0.13%)` | :arrow_up: | | hudiflink | `59.73% <100.00%> (-0.31%)` | :arrow_down: | | hudihadoopmr | `52.02% <ø> (+0.73%)` | :arrow_up: | | hudisparkdatasource | `67.21% <ø> (-0.45%)` | :arrow_down: | | hudisync | `55.97% <ø> (+1.46%)` | :arrow_up: | | huditimelineservice | `64.07% <ø> (ø)` | | | hudiutilities | `59.26% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3258?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...g/apache/hudi/sink/partitioner/BucketAssigner.java](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS9zaW5rL3BhcnRpdGlvbmVyL0J1Y2tldEFzc2lnbmVyLmphdmE=) | `84.69% <100.00%> (+1.36%)` | :arrow_up: | | [...in/scala/org/apache/hudi/HoodieStreamingSink.scala](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9odWRpL0hvb2RpZVN0cmVhbWluZ1Npbmsuc2NhbGE=) | `28.00% <0.00%> (-10.40%)` | :arrow_down: | | [.../java/org/apache/hudi/client/HoodieReadClient.java](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L0hvb2RpZVJlYWRDbGllbnQuamF2YQ==) | `94.64% <0.00%> (-5.36%)` | :arrow_down: | | [...c/main/java/org/apache/hudi/util/StreamerUtil.java](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS91dGlsL1N0cmVhbWVyVXRpbC5qYXZh) | `64.00% <0.00%> (-3.80%)` | :arrow_down: | | [.../hudi/common/util/collection/LazyFileIterable.java](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3V0aWwvY29sbGVjdGlvbi9MYXp5RmlsZUl0ZXJhYmxlLmphdmE=) | `71.73% <0.00%> (-2.68%)` | :arrow_down: | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3184: [HUDI-1860] Add INSERT_OVERWRITE and INSERT_OVERWRITE_TABLE support to DeltaStreamer
codecov-commenter edited a comment on pull request #3184: URL: https://github.com/apache/hudi/pull/3184#issuecomment-870526141 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3184?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3184](https://codecov.io/gh/apache/hudi/pull/3184?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (55947cf) into [master](https://codecov.io/gh/apache/hudi/commit/d024439764ceeca6366cb33689b729a1c69a6272?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (d024439) will **decrease** coverage by `16.75%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/hudi/pull/3184?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3184 +/- ## = - Coverage 44.10% 27.35% -16.76% + Complexity 5157 1294 -3863 = Files 936 386 -550 Lines 4162915346-26283 Branches 4189 1338 -2851 = - Hits 18362 4198-14164 + Misses2163810844-10794 + Partials 1629 304 -1325 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `20.91% <ø> (-13.56%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `4.88% <ø> (-50.86%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `59.37% <100.00%> (+50.25%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3184?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...apache/hudi/utilities/deltastreamer/DeltaSync.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL2RlbHRhc3RyZWFtZXIvRGVsdGFTeW5jLmphdmE=) | `72.29% <100.00%> (+72.29%)` | :arrow_up: | | [...main/java/org/apache/hudi/metrics/HoodieGauge.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvSG9vZGllR2F1Z2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../java/org/apache/hudi/metrics/MetricsReporter.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/metrics/MetricsReporterType.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyVHlwZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[jira] [Commented] (HUDI-1633) Make callback return HoodieWriteStat
[ https://issues.apache.org/jira/browse/HUDI-1633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381733#comment-17381733 ] ASF GitHub Bot commented on HUDI-1633: -- codecov-commenter edited a comment on pull request #2445: URL: https://github.com/apache/hudi/pull/2445#issuecomment-881149916 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2445?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#2445](https://codecov.io/gh/apache/hudi/pull/2445?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (1068b7f) into [master](https://codecov.io/gh/apache/hudi/commit/e3d3677b7e7899705b624925666317f0c074f7c7?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (e3d3677) will **decrease** coverage by `0.61%`. > The diff coverage is `0.00%`. [](https://codecov.io/gh/apache/hudi/pull/2445?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#2445 +/- ## - Coverage 50.73% 50.11% -0.62% + Complexity 3064 470-2594 Files 419 79 -340 Lines 18797 3492 -15305 Branches 1922 431-1491 - Hits 9536 1750-7786 + Misses 8487 1585-6902 + Partials774 157 -617 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `∅ <ø> (∅)` | | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `4.88% <ø> (-43.74%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `72.84% <0.00%> (+3.36%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/2445?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...callback/kafka/HoodieWriteCommitKafkaCallback.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL2NhbGxiYWNrL2thZmthL0hvb2RpZVdyaXRlQ29tbWl0S2Fma2FDYWxsYmFjay5qYXZh) | `0.00% <0.00%> (ø)` | | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...main/java/org/apache/hudi/hive/HiveSyncConfig.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvSGl2ZVN5bmNDb25maWcuamF2YQ==) | `0.00% <0.00%> (-97.62%)` | :arrow_down: | | [...in/java/org/apache/hudi/hive/SchemaDifference.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvU2NoZW1hRGlmZmVyZW5jZS5qYXZh) | `0.00% <0.00%> (-84.85%)` | :arrow_down: | | [...in/java/org/apache/hudi/hive/HoodieHiveClient.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvSG9vZGllSGl2ZUNsaWVudC5qYXZh) | `0.00% <0.00%> (-72.02%)` | :arrow_down: | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #2445: [HUDI-1633] Make callback return HoodieWriteStat
codecov-commenter edited a comment on pull request #2445: URL: https://github.com/apache/hudi/pull/2445#issuecomment-881149916 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2445?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#2445](https://codecov.io/gh/apache/hudi/pull/2445?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (1068b7f) into [master](https://codecov.io/gh/apache/hudi/commit/e3d3677b7e7899705b624925666317f0c074f7c7?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (e3d3677) will **decrease** coverage by `0.61%`. > The diff coverage is `0.00%`. [](https://codecov.io/gh/apache/hudi/pull/2445?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#2445 +/- ## - Coverage 50.73% 50.11% -0.62% + Complexity 3064 470-2594 Files 419 79 -340 Lines 18797 3492 -15305 Branches 1922 431-1491 - Hits 9536 1750-7786 + Misses 8487 1585-6902 + Partials774 157 -617 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `∅ <ø> (∅)` | | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `4.88% <ø> (-43.74%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `72.84% <0.00%> (+3.36%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/2445?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...callback/kafka/HoodieWriteCommitKafkaCallback.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL2NhbGxiYWNrL2thZmthL0hvb2RpZVdyaXRlQ29tbWl0S2Fma2FDYWxsYmFjay5qYXZh) | `0.00% <0.00%> (ø)` | | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...main/java/org/apache/hudi/hive/HiveSyncConfig.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvSGl2ZVN5bmNDb25maWcuamF2YQ==) | `0.00% <0.00%> (-97.62%)` | :arrow_down: | | [...in/java/org/apache/hudi/hive/SchemaDifference.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvU2NoZW1hRGlmZmVyZW5jZS5qYXZh) | `0.00% <0.00%> (-84.85%)` | :arrow_down: | | [...in/java/org/apache/hudi/hive/HoodieHiveClient.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvSG9vZGllSGl2ZUNsaWVudC5qYXZh) | `0.00% <0.00%> (-72.02%)` | :arrow_down: | |
[jira] [Updated] (HUDI-2183) HiveSyncTool java.lang.NoClassDefFoundError: org/json/JSONException
[
https://issues.apache.org/jira/browse/HUDI-2183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
min.jia updated HUDI-2183:
--
Fix Version/s: (was: 0.8.0)
> HiveSyncTool java.lang.NoClassDefFoundError: org/json/JSONException
>
>
> Key: HUDI-2183
> URL: https://issues.apache.org/jira/browse/HUDI-2183
> Project: Apache Hudi
> Issue Type: Bug
> Components: Spark Integration
>Reporter: min.jia
>Priority: Minor
> Labels: HiveSyncTool
>
> Hive synchronization meta，spark write hudi
> spark verison 2.4.6
> hudi：0.8.0
> hoodie.datasource.hive_sync.use_jdbc = false
>
> {code:java}
> //代码占位符
> {code}
> User class threw exception: java.lang.NoClassDefFoundError:
> org/json/JSONException at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeCreateTable(SemanticAnalyzer.java:10847)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genResolvedParseTree(SemanticAnalyzer.java:10047)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10128)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10013)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:227)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:424) at
> org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308) at
> org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1122) at
> org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1170) at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059) at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049) at
> org.apache.hudi.hive.HoodieHiveClient.updateHiveSQLs(HoodieHiveClient.java:401)
> at
> org.apache.hudi.hive.HoodieHiveClient.updateHiveSQLUsingHiveDriver(HoodieHiveClient.java:384)
> at
> org.apache.hudi.hive.HoodieHiveClient.updateHiveSQL(HoodieHiveClient.java:374)
> at
> org.apache.hudi.hive.HoodieHiveClient.createTable(HoodieHiveClient.java:263)
> at org.apache.hudi.hive.HiveSyncTool.syncSchema(HiveSyncTool.java:199) at
> org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:154) at
> org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:108) at
> org.apache.hudi.HoodieSparkSqlWriter$.org$apache$hudi$HoodieSparkSqlWriter$$syncHive(HoodieSparkSqlWriter.scala:391)
> at
> org.apache.hudi.HoodieSparkSqlWriter$$anonfun$metaSync$2.apply(HoodieSparkSqlWriter.scala:440)
> at
> org.apache.hudi.HoodieSparkSqlWriter$$anonfun$metaSync$2.apply(HoodieSparkSqlWriter.scala:436)
> at scala.collection.mutable.HashSet.foreach(HashSet.scala:78) at
> org.apache.hudi.HoodieSparkSqlWriter$.metaSync(HoodieSparkSqlWriter.scala:436)
> at
> org.apache.hudi.HoodieSparkSqlWriter$.commitAndPerformPostOperations(HoodieSparkSqlWriter.scala:497)
> at
> org.apache.hudi.HoodieSparkSqlWriter$.write(HoodieSparkSqlWriter.scala:222)
> at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:145) at
> org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:45)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:86)
> at
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)
> at
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)
> at
> org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at
> org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152) at
> org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127) at
> org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:80)
> at
> org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:80)
> at
> org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:676)
> at
> org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:676)
> at
> org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:78)
> at
> org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:125)
> at
> org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:73)
> at
> org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:676) at
>
[jira] [Commented] (HUDI-2170) Always choose the latest record for HoodieRecordPayload
[
https://issues.apache.org/jira/browse/HUDI-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381731#comment-17381731
]
ASF GitHub Bot commented on HUDI-2170:
--
danny0405 commented on a change in pull request #3267:
URL: https://github.com/apache/hudi/pull/3267#discussion_r670935008
##
File path:
hudi-common/src/main/java/org/apache/hudi/common/table/log/AbstractHoodieLogRecordScanner.java
##
@@ -113,8 +120,11 @@ protected AbstractHoodieLogRecordScanner(FileSystem fs,
String basePath, List Always choose the latest record for HoodieRecordPayload
> ---
>
> Key: HUDI-2170
> URL: https://issues.apache.org/jira/browse/HUDI-2170
> Project: Apache Hudi
> Issue Type: Improvement
> Components: Common Core
>Reporter: Danny Chen
>Assignee: Danny Chen
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.9.0
>
>
> Now in {{OverwriteWithLatestAvroPayload.preCombine}}, we still choose the old
> record when the new record has the same preCombine field with the old one,
> actually it is more natural to keep the new incoming record instead. The
> {{DefaultHoodieRecordPayload.combineAndGetUpdateValue}} method already did
> that.
> See issue: https://github.com/apache/hudi/issues/3266.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HUDI-2170) Always choose the latest record for HoodieRecordPayload
[
https://issues.apache.org/jira/browse/HUDI-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381730#comment-17381730
]
ASF GitHub Bot commented on HUDI-2170:
--
danny0405 commented on a change in pull request #3267:
URL: https://github.com/apache/hudi/pull/3267#discussion_r670949047
##
File path:
hudi-common/src/main/java/org/apache/hudi/common/model/OverwriteWithLatestAvroPayload.java
##
@@ -90,6 +93,14 @@ protected boolean isDeleteRecord(GenericRecord
genericRecord) {
return (deleteMarker instanceof Boolean && (boolean) deleteMarker);
}
+ /**
+ * Returns the ordering value of given record {@code record}.
+ */
+ protected static Object getOrderingVal(GenericRecord record, Properties
properties) {
+return getNestedFieldVal(record,
+
properties.getProperty(HoodiePayloadProps.PAYLOAD_ORDERING_FIELD_PROP_KEY),
true);
Review comment:
Actually there is no need to care about whether the record comes from
the persisted file or not, just fetch it and return null if the old schema does
not contains the `preCombine` field.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Always choose the latest record for HoodieRecordPayload
> ---
>
> Key: HUDI-2170
> URL: https://issues.apache.org/jira/browse/HUDI-2170
> Project: Apache Hudi
> Issue Type: Improvement
> Components: Common Core
>Reporter: Danny Chen
>Assignee: Danny Chen
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.9.0
>
>
> Now in {{OverwriteWithLatestAvroPayload.preCombine}}, we still choose the old
> record when the new record has the same preCombine field with the old one,
> actually it is more natural to keep the new incoming record instead. The
> {{DefaultHoodieRecordPayload.combineAndGetUpdateValue}} method already did
> that.
> See issue: https://github.com/apache/hudi/issues/3266.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hudi] danny0405 commented on a change in pull request #3267: [HUDI-2170] Always choose the latest record for HoodieRecordPayload
danny0405 commented on a change in pull request #3267: URL: https://github.com/apache/hudi/pull/3267#discussion_r670935008 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/AbstractHoodieLogRecordScanner.java ## @@ -113,8 +120,11 @@ protected AbstractHoodieLogRecordScanner(FileSystem fs, String basePath, List
[GitHub] [hudi] danny0405 commented on a change in pull request #3267: [HUDI-2170] Always choose the latest record for HoodieRecordPayload
danny0405 commented on a change in pull request #3267:
URL: https://github.com/apache/hudi/pull/3267#discussion_r670949047
##
File path:
hudi-common/src/main/java/org/apache/hudi/common/model/OverwriteWithLatestAvroPayload.java
##
@@ -90,6 +93,14 @@ protected boolean isDeleteRecord(GenericRecord
genericRecord) {
return (deleteMarker instanceof Boolean && (boolean) deleteMarker);
}
+ /**
+ * Returns the ordering value of given record {@code record}.
+ */
+ protected static Object getOrderingVal(GenericRecord record, Properties
properties) {
+return getNestedFieldVal(record,
+
properties.getProperty(HoodiePayloadProps.PAYLOAD_ORDERING_FIELD_PROP_KEY),
true);
Review comment:
Actually there is no need to care about whether the record comes from
the persisted file or not, just fetch it and return null if the old schema does
not contains the `preCombine` field.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HUDI-1425) Performance loss with the additional hoodieRecords.isEmpty() in HoodieSparkSqlWriter#write
[ https://issues.apache.org/jira/browse/HUDI-1425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381728#comment-17381728 ] ASF GitHub Bot commented on HUDI-1425: -- hudi-bot edited a comment on pull request #2296: URL: https://github.com/apache/hudi/pull/2296#issuecomment-874367360 ## CI report: * d4e3d89e0c02f1b9a704a61627f7aafec14e67e6 UNKNOWN * dc36883644cf88971ecf13dc3642d60e60a1e6ec UNKNOWN * cd6767b606cfa19b064e4455f7a396e84e13d839 UNKNOWN * 7ebc9ff5395d8790339586ae3ff7e56b05a6a569 UNKNOWN * 4c86b7708e4204d585180ae8e9187c416ba6d052 UNKNOWN * 244d86b04feb328d55597223bdf33f50a17c708f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=941) * fa8ba5e890fd394820a0ff91e0395b9be62d7ba8 UNKNOWN * 6ac9d507dc541a7e17741c3e1fcd8a6bb28aecc6 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=954) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Performance loss with the additional hoodieRecords.isEmpty() in > HoodieSparkSqlWriter#write > -- > > Key: HUDI-1425 > URL: https://issues.apache.org/jira/browse/HUDI-1425 > Project: Apache Hudi > Issue Type: Improvement > Components: Spark Integration >Affects Versions: 0.9.0 >Reporter: pengzhiwei >Assignee: pengzhiwei >Priority: Blocker > Labels: pull-request-available > Fix For: 0.9.0 > > Attachments: 截屏2020-11-30 下午9.47.55.png > > > Currently in HoodieSparkSqlWriter#write, there is a _isEmpty()_ test for > _hoodieRecords._ This may be a heavy operator in the case when the > _hoodieRecords_ contains complex RDD operate. > !截屏2020-11-30 下午9.47.55.png|width=1255,height=161! > IMO this test does nothing to do with the performance improve，but rather > affects performance. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HUDI-2184) Support setting hive sync partition extractor class based on flink configuration
[ https://issues.apache.org/jira/browse/HUDI-2184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381729#comment-17381729 ] ASF GitHub Bot commented on HUDI-2184: -- codecov-commenter edited a comment on pull request #3284: URL: https://github.com/apache/hudi/pull/3284#issuecomment-881148186 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3284](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (66d9b95) into [master](https://codecov.io/gh/apache/hudi/commit/a62a6cff32e7d7f62ffdf6362fb8876497fd0528?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a62a6cf) will **decrease** coverage by `20.39%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3284 +/- ## = - Coverage 47.70% 27.31% -20.40% + Complexity 5550 1292 -4258 = Files 936 386 -550 Lines 4162415338-26286 Branches 4188 1338 -2850 = - Hits 19858 4190-15668 + Misses210844 -9156 + Partials 1766 304 -1462 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `20.91% <ø> (-13.56%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `4.88% <ø> (-51.10%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `59.26% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...main/java/org/apache/hudi/metrics/HoodieGauge.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvSG9vZGllR2F1Z2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../java/org/apache/hudi/metrics/MetricsReporter.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/metrics/MetricsReporterType.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyVHlwZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...rg/apache/hudi/client/bootstrap/BootstrapMode.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC9ib290c3RyYXAvQm9vdHN0cmFwTW9kZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3284: [HUDI-2184] Support setting hive sync partition extractor class based on flink co…
codecov-commenter edited a comment on pull request #3284: URL: https://github.com/apache/hudi/pull/3284#issuecomment-881148186 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3284](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (66d9b95) into [master](https://codecov.io/gh/apache/hudi/commit/a62a6cff32e7d7f62ffdf6362fb8876497fd0528?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a62a6cf) will **decrease** coverage by `20.39%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3284 +/- ## = - Coverage 47.70% 27.31% -20.40% + Complexity 5550 1292 -4258 = Files 936 386 -550 Lines 4162415338-26286 Branches 4188 1338 -2850 = - Hits 19858 4190-15668 + Misses210844 -9156 + Partials 1766 304 -1462 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `20.91% <ø> (-13.56%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `4.88% <ø> (-51.10%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `59.26% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...main/java/org/apache/hudi/metrics/HoodieGauge.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvSG9vZGllR2F1Z2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../java/org/apache/hudi/metrics/MetricsReporter.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/metrics/MetricsReporterType.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyVHlwZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...rg/apache/hudi/client/bootstrap/BootstrapMode.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC9ib290c3RyYXAvQm9vdHN0cmFwTW9kZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[GitHub] [hudi] hudi-bot edited a comment on pull request #2296: [HUDI-1425] Performance loss with the additional hoodieRecords.isEmpty() in HoodieSparkSqlWriter#write
hudi-bot edited a comment on pull request #2296: URL: https://github.com/apache/hudi/pull/2296#issuecomment-874367360 ## CI report: * d4e3d89e0c02f1b9a704a61627f7aafec14e67e6 UNKNOWN * dc36883644cf88971ecf13dc3642d60e60a1e6ec UNKNOWN * cd6767b606cfa19b064e4455f7a396e84e13d839 UNKNOWN * 7ebc9ff5395d8790339586ae3ff7e56b05a6a569 UNKNOWN * 4c86b7708e4204d585180ae8e9187c416ba6d052 UNKNOWN * 244d86b04feb328d55597223bdf33f50a17c708f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=941) * fa8ba5e890fd394820a0ff91e0395b9be62d7ba8 UNKNOWN * 6ac9d507dc541a7e17741c3e1fcd8a6bb28aecc6 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=954) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HUDI-2170) Always choose the latest record for HoodieRecordPayload
[
https://issues.apache.org/jira/browse/HUDI-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381726#comment-17381726
]
ASF GitHub Bot commented on HUDI-2170:
--
danny0405 commented on a change in pull request #3267:
URL: https://github.com/apache/hudi/pull/3267#discussion_r670948572
##
File path:
hudi-common/src/main/java/org/apache/hudi/common/model/OverwriteWithLatestAvroPayload.java
##
@@ -49,7 +49,7 @@ public OverwriteWithLatestAvroPayload(Option
record) {
@Override
public OverwriteWithLatestAvroPayload
preCombine(OverwriteWithLatestAvroPayload another) {
// pick the payload with greatest ordering value
-if (another.orderingVal.compareTo(orderingVal) > 0) {
+if (another.orderingVal.compareTo(orderingVal) >= 0) {
Review comment:
Finally i found that the `another` should always be the old value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Always choose the latest record for HoodieRecordPayload
> ---
>
> Key: HUDI-2170
> URL: https://issues.apache.org/jira/browse/HUDI-2170
> Project: Apache Hudi
> Issue Type: Improvement
> Components: Common Core
>Reporter: Danny Chen
>Assignee: Danny Chen
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.9.0
>
>
> Now in {{OverwriteWithLatestAvroPayload.preCombine}}, we still choose the old
> record when the new record has the same preCombine field with the old one,
> actually it is more natural to keep the new incoming record instead. The
> {{DefaultHoodieRecordPayload.combineAndGetUpdateValue}} method already did
> that.
> See issue: https://github.com/apache/hudi/issues/3266.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HUDI-2170) Always choose the latest record for HoodieRecordPayload
[
https://issues.apache.org/jira/browse/HUDI-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381727#comment-17381727
]
ASF GitHub Bot commented on HUDI-2170:
--
danny0405 commented on a change in pull request #3267:
URL: https://github.com/apache/hudi/pull/3267#discussion_r670948572
##
File path:
hudi-common/src/main/java/org/apache/hudi/common/model/OverwriteWithLatestAvroPayload.java
##
@@ -49,7 +49,7 @@ public OverwriteWithLatestAvroPayload(Option
record) {
@Override
public OverwriteWithLatestAvroPayload
preCombine(OverwriteWithLatestAvroPayload another) {
// pick the payload with greatest ordering value
-if (another.orderingVal.compareTo(orderingVal) > 0) {
+if (another.orderingVal.compareTo(orderingVal) >= 0) {
Review comment:
Finally i found that the `another` should always be the old value. So
the original logic is correct.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Always choose the latest record for HoodieRecordPayload
> ---
>
> Key: HUDI-2170
> URL: https://issues.apache.org/jira/browse/HUDI-2170
> Project: Apache Hudi
> Issue Type: Improvement
> Components: Common Core
>Reporter: Danny Chen
>Assignee: Danny Chen
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.9.0
>
>
> Now in {{OverwriteWithLatestAvroPayload.preCombine}}, we still choose the old
> record when the new record has the same preCombine field with the old one,
> actually it is more natural to keep the new incoming record instead. The
> {{DefaultHoodieRecordPayload.combineAndGetUpdateValue}} method already did
> that.
> See issue: https://github.com/apache/hudi/issues/3266.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hudi] danny0405 commented on a change in pull request #3267: [HUDI-2170] Always choose the latest record for HoodieRecordPayload
danny0405 commented on a change in pull request #3267:
URL: https://github.com/apache/hudi/pull/3267#discussion_r670948572
##
File path:
hudi-common/src/main/java/org/apache/hudi/common/model/OverwriteWithLatestAvroPayload.java
##
@@ -49,7 +49,7 @@ public OverwriteWithLatestAvroPayload(Option
record) {
@Override
public OverwriteWithLatestAvroPayload
preCombine(OverwriteWithLatestAvroPayload another) {
// pick the payload with greatest ordering value
-if (another.orderingVal.compareTo(orderingVal) > 0) {
+if (another.orderingVal.compareTo(orderingVal) >= 0) {
Review comment:
Finally i found that the `another` should always be the old value. So
the original logic is correct.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hudi] danny0405 commented on a change in pull request #3267: [HUDI-2170] Always choose the latest record for HoodieRecordPayload
danny0405 commented on a change in pull request #3267:
URL: https://github.com/apache/hudi/pull/3267#discussion_r670948572
##
File path:
hudi-common/src/main/java/org/apache/hudi/common/model/OverwriteWithLatestAvroPayload.java
##
@@ -49,7 +49,7 @@ public OverwriteWithLatestAvroPayload(Option
record) {
@Override
public OverwriteWithLatestAvroPayload
preCombine(OverwriteWithLatestAvroPayload another) {
// pick the payload with greatest ordering value
-if (another.orderingVal.compareTo(orderingVal) > 0) {
+if (another.orderingVal.compareTo(orderingVal) >= 0) {
Review comment:
Finally i found that the `another` should always be the old value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HUDI-1771) Propagate CDC format for hoodie
[ https://issues.apache.org/jira/browse/HUDI-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381725#comment-17381725 ] ASF GitHub Bot commented on HUDI-1771: -- codecov-commenter edited a comment on pull request #3285: URL: https://github.com/apache/hudi/pull/3285#issuecomment-881152765 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3285?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3285](https://codecov.io/gh/apache/hudi/pull/3285?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a517fd1) into [master](https://codecov.io/gh/apache/hudi/commit/a62a6cff32e7d7f62ffdf6362fb8876497fd0528?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a62a6cf) will **decrease** coverage by `31.97%`. > The diff coverage is `0.00%`. [](https://codecov.io/gh/apache/hudi/pull/3285?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3285 +/- ## = - Coverage 47.70% 15.73% -31.98% + Complexity 5550 492 -5058 = Files 936 284 -652 Lines 4162411838-29786 Branches 4188 984 -3204 = - Hits 19858 1863-17995 + Misses2 9811-10189 + Partials 1766 164 -1602 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `0.00% <0.00%> (-34.47%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `4.88% <ø> (-51.10%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `59.23% <ø> (-0.04%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3285?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...in/java/org/apache/hudi/io/HoodieAppendHandle.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2lvL0hvb2RpZUFwcGVuZEhhbmRsZS5qYXZh) | `0.00% <0.00%> (ø)` | | | [...in/java/org/apache/hudi/io/HoodieCreateHandle.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2lvL0hvb2RpZUNyZWF0ZUhhbmRsZS5qYXZh) | `0.00% <0.00%> (ø)` | | | [...ain/java/org/apache/hudi/io/HoodieMergeHandle.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2lvL0hvb2RpZU1lcmdlSGFuZGxlLmphdmE=) | `0.00% <0.00%> (ø)` | | | [...main/java/org/apache/hudi/metrics/HoodieGauge.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvSG9vZGllR2F1Z2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3285: [HUDI-1771] Propagate CDC format for hoodie
codecov-commenter edited a comment on pull request #3285: URL: https://github.com/apache/hudi/pull/3285#issuecomment-881152765 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3285?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3285](https://codecov.io/gh/apache/hudi/pull/3285?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a517fd1) into [master](https://codecov.io/gh/apache/hudi/commit/a62a6cff32e7d7f62ffdf6362fb8876497fd0528?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a62a6cf) will **decrease** coverage by `31.97%`. > The diff coverage is `0.00%`. [](https://codecov.io/gh/apache/hudi/pull/3285?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3285 +/- ## = - Coverage 47.70% 15.73% -31.98% + Complexity 5550 492 -5058 = Files 936 284 -652 Lines 4162411838-29786 Branches 4188 984 -3204 = - Hits 19858 1863-17995 + Misses2 9811-10189 + Partials 1766 164 -1602 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `0.00% <0.00%> (-34.47%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `4.88% <ø> (-51.10%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `59.23% <ø> (-0.04%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3285?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...in/java/org/apache/hudi/io/HoodieAppendHandle.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2lvL0hvb2RpZUFwcGVuZEhhbmRsZS5qYXZh) | `0.00% <0.00%> (ø)` | | | [...in/java/org/apache/hudi/io/HoodieCreateHandle.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2lvL0hvb2RpZUNyZWF0ZUhhbmRsZS5qYXZh) | `0.00% <0.00%> (ø)` | | | [...ain/java/org/apache/hudi/io/HoodieMergeHandle.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2lvL0hvb2RpZU1lcmdlSGFuZGxlLmphdmE=) | `0.00% <0.00%> (ø)` | | | [...main/java/org/apache/hudi/metrics/HoodieGauge.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvSG9vZGllR2F1Z2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/3285/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[jira] [Commented] (HUDI-1425) Performance loss with the additional hoodieRecords.isEmpty() in HoodieSparkSqlWriter#write
[ https://issues.apache.org/jira/browse/HUDI-1425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381724#comment-17381724 ] ASF GitHub Bot commented on HUDI-1425: -- hudi-bot edited a comment on pull request #2296: URL: https://github.com/apache/hudi/pull/2296#issuecomment-874367360 ## CI report: * d4e3d89e0c02f1b9a704a61627f7aafec14e67e6 UNKNOWN * dc36883644cf88971ecf13dc3642d60e60a1e6ec UNKNOWN * cd6767b606cfa19b064e4455f7a396e84e13d839 UNKNOWN * 7ebc9ff5395d8790339586ae3ff7e56b05a6a569 UNKNOWN * 4c86b7708e4204d585180ae8e9187c416ba6d052 UNKNOWN * 244d86b04feb328d55597223bdf33f50a17c708f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=941) * fa8ba5e890fd394820a0ff91e0395b9be62d7ba8 UNKNOWN * 6ac9d507dc541a7e17741c3e1fcd8a6bb28aecc6 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Performance loss with the additional hoodieRecords.isEmpty() in > HoodieSparkSqlWriter#write > -- > > Key: HUDI-1425 > URL: https://issues.apache.org/jira/browse/HUDI-1425 > Project: Apache Hudi > Issue Type: Improvement > Components: Spark Integration >Affects Versions: 0.9.0 >Reporter: pengzhiwei >Assignee: pengzhiwei >Priority: Blocker > Labels: pull-request-available > Fix For: 0.9.0 > > Attachments: 截屏2020-11-30 下午9.47.55.png > > > Currently in HoodieSparkSqlWriter#write, there is a _isEmpty()_ test for > _hoodieRecords._ This may be a heavy operator in the case when the > _hoodieRecords_ contains complex RDD operate. > !截屏2020-11-30 下午9.47.55.png|width=1255,height=161! > IMO this test does nothing to do with the performance improve，but rather > affects performance. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hudi] hudi-bot edited a comment on pull request #2296: [HUDI-1425] Performance loss with the additional hoodieRecords.isEmpty() in HoodieSparkSqlWriter#write
hudi-bot edited a comment on pull request #2296: URL: https://github.com/apache/hudi/pull/2296#issuecomment-874367360 ## CI report: * d4e3d89e0c02f1b9a704a61627f7aafec14e67e6 UNKNOWN * dc36883644cf88971ecf13dc3642d60e60a1e6ec UNKNOWN * cd6767b606cfa19b064e4455f7a396e84e13d839 UNKNOWN * 7ebc9ff5395d8790339586ae3ff7e56b05a6a569 UNKNOWN * 4c86b7708e4204d585180ae8e9187c416ba6d052 UNKNOWN * 244d86b04feb328d55597223bdf33f50a17c708f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=941) * fa8ba5e890fd394820a0ff91e0395b9be62d7ba8 UNKNOWN * 6ac9d507dc541a7e17741c3e1fcd8a6bb28aecc6 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HUDI-1425) Performance loss with the additional hoodieRecords.isEmpty() in HoodieSparkSqlWriter#write
[ https://issues.apache.org/jira/browse/HUDI-1425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381722#comment-17381722 ] ASF GitHub Bot commented on HUDI-1425: -- hudi-bot edited a comment on pull request #2296: URL: https://github.com/apache/hudi/pull/2296#issuecomment-874367360 ## CI report: * d4e3d89e0c02f1b9a704a61627f7aafec14e67e6 UNKNOWN * dc36883644cf88971ecf13dc3642d60e60a1e6ec UNKNOWN * cd6767b606cfa19b064e4455f7a396e84e13d839 UNKNOWN * 7ebc9ff5395d8790339586ae3ff7e56b05a6a569 UNKNOWN * 4c86b7708e4204d585180ae8e9187c416ba6d052 UNKNOWN * 244d86b04feb328d55597223bdf33f50a17c708f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=941) * fa8ba5e890fd394820a0ff91e0395b9be62d7ba8 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Performance loss with the additional hoodieRecords.isEmpty() in > HoodieSparkSqlWriter#write > -- > > Key: HUDI-1425 > URL: https://issues.apache.org/jira/browse/HUDI-1425 > Project: Apache Hudi > Issue Type: Improvement > Components: Spark Integration >Affects Versions: 0.9.0 >Reporter: pengzhiwei >Assignee: pengzhiwei >Priority: Blocker > Labels: pull-request-available > Fix For: 0.9.0 > > Attachments: 截屏2020-11-30 下午9.47.55.png > > > Currently in HoodieSparkSqlWriter#write, there is a _isEmpty()_ test for > _hoodieRecords._ This may be a heavy operator in the case when the > _hoodieRecords_ contains complex RDD operate. > !截屏2020-11-30 下午9.47.55.png|width=1255,height=161! > IMO this test does nothing to do with the performance improve，but rather > affects performance. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hudi] hudi-bot edited a comment on pull request #2296: [HUDI-1425] Performance loss with the additional hoodieRecords.isEmpty() in HoodieSparkSqlWriter#write
hudi-bot edited a comment on pull request #2296: URL: https://github.com/apache/hudi/pull/2296#issuecomment-874367360 ## CI report: * d4e3d89e0c02f1b9a704a61627f7aafec14e67e6 UNKNOWN * dc36883644cf88971ecf13dc3642d60e60a1e6ec UNKNOWN * cd6767b606cfa19b064e4455f7a396e84e13d839 UNKNOWN * 7ebc9ff5395d8790339586ae3ff7e56b05a6a569 UNKNOWN * 4c86b7708e4204d585180ae8e9187c416ba6d052 UNKNOWN * 244d86b04feb328d55597223bdf33f50a17c708f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=941) * fa8ba5e890fd394820a0ff91e0395b9be62d7ba8 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HUDI-1771) Propagate CDC format for hoodie
[ https://issues.apache.org/jira/browse/HUDI-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381721#comment-17381721 ] ASF GitHub Bot commented on HUDI-1771: -- hudi-bot edited a comment on pull request #3285: URL: https://github.com/apache/hudi/pull/3285#issuecomment-881141261 ## CI report: * a517fd158c2ea902d175eae919636a9866291833 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=953) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Propagate CDC format for hoodie > --- > > Key: HUDI-1771 > URL: https://issues.apache.org/jira/browse/HUDI-1771 > Project: Apache Hudi > Issue Type: New Feature > Components: Flink Integration >Reporter: Danny Chen >Assignee: Danny Chen >Priority: Major > Labels: pull-request-available, sev:normal > Fix For: 0.9.0 > > > Like what we discussed in the dev mailing list: > https://lists.apache.org/thread.html/r31b2d1404e4e043a5f875b78105ba6f9a801e78f265ad91242ad5eb2%40%3Cdev.hudi.apache.org%3E > Keep the change flags make new use cases possible: using HUDI as the unified > storage format for DWD and DWS layer. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hudi] hudi-bot edited a comment on pull request #3285: [HUDI-1771] Propagate CDC format for hoodie
hudi-bot edited a comment on pull request #3285: URL: https://github.com/apache/hudi/pull/3285#issuecomment-881141261 ## CI report: * a517fd158c2ea902d175eae919636a9866291833 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=953) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HUDI-1860) Add INSERT_OVERWRITE support to DeltaStreamer
[ https://issues.apache.org/jira/browse/HUDI-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381718#comment-17381718 ] ASF GitHub Bot commented on HUDI-1860: -- codecov-commenter edited a comment on pull request #3184: URL: https://github.com/apache/hudi/pull/3184#issuecomment-870526141 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3184?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3184](https://codecov.io/gh/apache/hudi/pull/3184?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (55947cf) into [master](https://codecov.io/gh/apache/hudi/commit/d024439764ceeca6366cb33689b729a1c69a6272?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (d024439) will **decrease** coverage by `28.29%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/hudi/pull/3184?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3184 +/- ## = - Coverage 44.10% 15.81% -28.30% + Complexity 5157 495 -4662 = Files 936 284 -652 Lines 4162911838-29791 Branches 4189 981 -3208 = - Hits 18362 1872-16490 + Misses21638 9803-11835 + Partials 1629 163 -1466 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `0.00% <ø> (-34.47%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `4.88% <ø> (-50.86%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `59.37% <100.00%> (+50.25%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3184?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...apache/hudi/utilities/deltastreamer/DeltaSync.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL2RlbHRhc3RyZWFtZXIvRGVsdGFTeW5jLmphdmE=) | `72.29% <100.00%> (+72.29%)` | :arrow_up: | | [...main/java/org/apache/hudi/metrics/HoodieGauge.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvSG9vZGllR2F1Z2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../java/org/apache/hudi/metrics/MetricsReporter.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/metrics/MetricsReporterType.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyVHlwZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[jira] [Commented] (HUDI-2145) Create new bucket when NewFileAssignState filled
[ https://issues.apache.org/jira/browse/HUDI-2145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381719#comment-17381719 ] ASF GitHub Bot commented on HUDI-2145: -- codecov-commenter edited a comment on pull request #3258: URL: https://github.com/apache/hudi/pull/3258#issuecomment-878089095 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3258?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3258](https://codecov.io/gh/apache/hudi/pull/3258?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (d2184c0) into [master](https://codecov.io/gh/apache/hudi/commit/5804ad8e32ae05758ebc5e47f5d4fb4db371ab52?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (5804ad8) will **increase** coverage by `0.06%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/hudi/pull/3258?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3258 +/- ## + Coverage 47.72% 47.79% +0.06% - Complexity 5528 5567 +39 Files 934 936 +2 Lines 4145741626 +169 Branches 4166 4188 +22 + Hits 1978619895 +109 - Misses1991419967 +53 - Partials 1757 1764 +7 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `39.97% <ø> (ø)` | | | hudiclient | `34.46% <ø> (+0.01%)` | :arrow_up: | | hudicommon | `48.69% <ø> (+0.13%)` | :arrow_up: | | hudiflink | `59.73% <100.00%> (-0.31%)` | :arrow_down: | | hudihadoopmr | `52.02% <ø> (+0.73%)` | :arrow_up: | | hudisparkdatasource | `67.21% <ø> (-0.45%)` | :arrow_down: | | hudisync | `55.97% <ø> (+1.46%)` | :arrow_up: | | huditimelineservice | `64.07% <ø> (ø)` | | | hudiutilities | `59.26% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3258?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...g/apache/hudi/sink/partitioner/BucketAssigner.java](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS9zaW5rL3BhcnRpdGlvbmVyL0J1Y2tldEFzc2lnbmVyLmphdmE=) | `84.69% <100.00%> (+1.36%)` | :arrow_up: | | [...in/scala/org/apache/hudi/HoodieStreamingSink.scala](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9odWRpL0hvb2RpZVN0cmVhbWluZ1Npbmsuc2NhbGE=) | `28.00% <0.00%> (-10.40%)` | :arrow_down: | | [.../java/org/apache/hudi/client/HoodieReadClient.java](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L0hvb2RpZVJlYWRDbGllbnQuamF2YQ==) | `94.64% <0.00%> (-5.36%)` | :arrow_down: | | [...c/main/java/org/apache/hudi/util/StreamerUtil.java](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS91dGlsL1N0cmVhbWVyVXRpbC5qYXZh) | `64.00% <0.00%> (-3.80%)` | :arrow_down: | |
[jira] [Commented] (HUDI-2184) Support setting hive sync partition extractor class based on flink configuration
[ https://issues.apache.org/jira/browse/HUDI-2184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381720#comment-17381720 ] ASF GitHub Bot commented on HUDI-2184: -- hudi-bot edited a comment on pull request #3284: URL: https://github.com/apache/hudi/pull/3284#issuecomment-881121030 ## CI report: * 66d9b957a7ecf646b90f5175c8e36d555c227d0e Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=952) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Support setting hive sync partition extractor class based on flink > configuration > > > Key: HUDI-2184 > URL: https://issues.apache.org/jira/browse/HUDI-2184 > Project: Apache Hudi > Issue Type: Improvement > Components: Flink Integration >Reporter: Zheng yunhong >Priority: Major > Labels: pull-request-available > Fix For: 0.9.0 > > > Support setting hive sync partition extractor class based on flink > configuration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hudi] hudi-bot edited a comment on pull request #3284: [HUDI-2184] Support setting hive sync partition extractor class based on flink co…
hudi-bot edited a comment on pull request #3284: URL: https://github.com/apache/hudi/pull/3284#issuecomment-881121030 ## CI report: * 66d9b957a7ecf646b90f5175c8e36d555c227d0e Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=952) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3258: [HUDI-2145] Create new bucket when NewFileAssignState filled
codecov-commenter edited a comment on pull request #3258: URL: https://github.com/apache/hudi/pull/3258#issuecomment-878089095 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3258?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3258](https://codecov.io/gh/apache/hudi/pull/3258?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (d2184c0) into [master](https://codecov.io/gh/apache/hudi/commit/5804ad8e32ae05758ebc5e47f5d4fb4db371ab52?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (5804ad8) will **increase** coverage by `0.06%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/hudi/pull/3258?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3258 +/- ## + Coverage 47.72% 47.79% +0.06% - Complexity 5528 5567 +39 Files 934 936 +2 Lines 4145741626 +169 Branches 4166 4188 +22 + Hits 1978619895 +109 - Misses1991419967 +53 - Partials 1757 1764 +7 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `39.97% <ø> (ø)` | | | hudiclient | `34.46% <ø> (+0.01%)` | :arrow_up: | | hudicommon | `48.69% <ø> (+0.13%)` | :arrow_up: | | hudiflink | `59.73% <100.00%> (-0.31%)` | :arrow_down: | | hudihadoopmr | `52.02% <ø> (+0.73%)` | :arrow_up: | | hudisparkdatasource | `67.21% <ø> (-0.45%)` | :arrow_down: | | hudisync | `55.97% <ø> (+1.46%)` | :arrow_up: | | huditimelineservice | `64.07% <ø> (ø)` | | | hudiutilities | `59.26% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3258?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...g/apache/hudi/sink/partitioner/BucketAssigner.java](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS9zaW5rL3BhcnRpdGlvbmVyL0J1Y2tldEFzc2lnbmVyLmphdmE=) | `84.69% <100.00%> (+1.36%)` | :arrow_up: | | [...in/scala/org/apache/hudi/HoodieStreamingSink.scala](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9odWRpL0hvb2RpZVN0cmVhbWluZ1Npbmsuc2NhbGE=) | `28.00% <0.00%> (-10.40%)` | :arrow_down: | | [.../java/org/apache/hudi/client/HoodieReadClient.java](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L0hvb2RpZVJlYWRDbGllbnQuamF2YQ==) | `94.64% <0.00%> (-5.36%)` | :arrow_down: | | [...c/main/java/org/apache/hudi/util/StreamerUtil.java](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS91dGlsL1N0cmVhbWVyVXRpbC5qYXZh) | `64.00% <0.00%> (-3.80%)` | :arrow_down: | | [.../hudi/common/util/collection/LazyFileIterable.java](https://codecov.io/gh/apache/hudi/pull/3258/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3V0aWwvY29sbGVjdGlvbi9MYXp5RmlsZUl0ZXJhYmxlLmphdmE=) | `71.73% <0.00%> (-2.68%)` | :arrow_down: | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3184: [HUDI-1860] Add INSERT_OVERWRITE and INSERT_OVERWRITE_TABLE support to DeltaStreamer
codecov-commenter edited a comment on pull request #3184: URL: https://github.com/apache/hudi/pull/3184#issuecomment-870526141 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3184?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3184](https://codecov.io/gh/apache/hudi/pull/3184?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (55947cf) into [master](https://codecov.io/gh/apache/hudi/commit/d024439764ceeca6366cb33689b729a1c69a6272?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (d024439) will **decrease** coverage by `28.29%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/hudi/pull/3184?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3184 +/- ## = - Coverage 44.10% 15.81% -28.30% + Complexity 5157 495 -4662 = Files 936 284 -652 Lines 4162911838-29791 Branches 4189 981 -3208 = - Hits 18362 1872-16490 + Misses21638 9803-11835 + Partials 1629 163 -1466 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `0.00% <ø> (-34.47%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `4.88% <ø> (-50.86%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `59.37% <100.00%> (+50.25%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3184?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...apache/hudi/utilities/deltastreamer/DeltaSync.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL2RlbHRhc3RyZWFtZXIvRGVsdGFTeW5jLmphdmE=) | `72.29% <100.00%> (+72.29%)` | :arrow_up: | | [...main/java/org/apache/hudi/metrics/HoodieGauge.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvSG9vZGllR2F1Z2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../java/org/apache/hudi/metrics/MetricsReporter.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/metrics/MetricsReporterType.java](https://codecov.io/gh/apache/hudi/pull/3184/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyVHlwZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[jira] [Commented] (HUDI-1633) Make callback return HoodieWriteStat
[ https://issues.apache.org/jira/browse/HUDI-1633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381716#comment-17381716 ] ASF GitHub Bot commented on HUDI-1633: -- codecov-commenter edited a comment on pull request #2445: URL: https://github.com/apache/hudi/pull/2445#issuecomment-881149916 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2445?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#2445](https://codecov.io/gh/apache/hudi/pull/2445?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (1068b7f) into [master](https://codecov.io/gh/apache/hudi/commit/e3d3677b7e7899705b624925666317f0c074f7c7?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (e3d3677) will **decrease** coverage by `0.61%`. > The diff coverage is `0.00%`. [](https://codecov.io/gh/apache/hudi/pull/2445?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#2445 +/- ## - Coverage 50.73% 50.11% -0.62% + Complexity 3064 470-2594 Files 419 79 -340 Lines 18797 3492 -15305 Branches 1922 431-1491 - Hits 9536 1750-7786 + Misses 8487 1585-6902 + Partials774 157 -617 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `?` | | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `4.88% <ø> (-43.74%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `72.84% <0.00%> (+3.36%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/2445?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...callback/kafka/HoodieWriteCommitKafkaCallback.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL2NhbGxiYWNrL2thZmthL0hvb2RpZVdyaXRlQ29tbWl0S2Fma2FDYWxsYmFjay5qYXZh) | `0.00% <0.00%> (ø)` | | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...main/java/org/apache/hudi/hive/HiveSyncConfig.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvSGl2ZVN5bmNDb25maWcuamF2YQ==) | `0.00% <0.00%> (-97.62%)` | :arrow_down: | | [...in/java/org/apache/hudi/hive/SchemaDifference.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvU2NoZW1hRGlmZmVyZW5jZS5qYXZh) | `0.00% <0.00%> (-84.85%)` | :arrow_down: | | [...in/java/org/apache/hudi/hive/HoodieHiveClient.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvSG9vZGllSGl2ZUNsaWVudC5qYXZh) | `0.00% <0.00%> (-72.02%)` | :arrow_down: | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #2445: [HUDI-1633] Make callback return HoodieWriteStat
codecov-commenter edited a comment on pull request #2445: URL: https://github.com/apache/hudi/pull/2445#issuecomment-881149916 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2445?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#2445](https://codecov.io/gh/apache/hudi/pull/2445?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (1068b7f) into [master](https://codecov.io/gh/apache/hudi/commit/e3d3677b7e7899705b624925666317f0c074f7c7?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (e3d3677) will **decrease** coverage by `0.61%`. > The diff coverage is `0.00%`. [](https://codecov.io/gh/apache/hudi/pull/2445?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#2445 +/- ## - Coverage 50.73% 50.11% -0.62% + Complexity 3064 470-2594 Files 419 79 -340 Lines 18797 3492 -15305 Branches 1922 431-1491 - Hits 9536 1750-7786 + Misses 8487 1585-6902 + Partials774 157 -617 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `?` | | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `4.88% <ø> (-43.74%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `72.84% <0.00%> (+3.36%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/2445?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...callback/kafka/HoodieWriteCommitKafkaCallback.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL2NhbGxiYWNrL2thZmthL0hvb2RpZVdyaXRlQ29tbWl0S2Fma2FDYWxsYmFjay5qYXZh) | `0.00% <0.00%> (ø)` | | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...main/java/org/apache/hudi/hive/HiveSyncConfig.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvSGl2ZVN5bmNDb25maWcuamF2YQ==) | `0.00% <0.00%> (-97.62%)` | :arrow_down: | | [...in/java/org/apache/hudi/hive/SchemaDifference.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvU2NoZW1hRGlmZmVyZW5jZS5qYXZh) | `0.00% <0.00%> (-84.85%)` | :arrow_down: | | [...in/java/org/apache/hudi/hive/HoodieHiveClient.java](https://codecov.io/gh/apache/hudi/pull/2445/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvSG9vZGllSGl2ZUNsaWVudC5qYXZh) | `0.00% <0.00%> (-72.02%)` | :arrow_down: | |
[jira] [Commented] (HUDI-1884) MergeInto Support Partial Update For COW
[
https://issues.apache.org/jira/browse/HUDI-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381714#comment-17381714
]
ASF GitHub Bot commented on HUDI-1884:
--
hudi-bot edited a comment on pull request #3154:
URL: https://github.com/apache/hudi/pull/3154#issuecomment-868360998
## CI report:
* 63cf96134656dbb41eaab4e03e747776599255b6 UNKNOWN
* 82b4b8bef8d143a14d445ae35fa5ec8fbbd5889f UNKNOWN
* f5511851a56f5a23263bfb05690217718a329254 Azure:
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=940)
* f5f0fd3f1eb1529643b7129349614ed80d106bed UNKNOWN
Bot commands
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> MergeInto Support Partial Update For COW
> -
>
> Key: HUDI-1884
> URL: https://issues.apache.org/jira/browse/HUDI-1884
> Project: Apache Hudi
> Issue Type: Sub-task
>Reporter: pengzhiwei
>Assignee: pengzhiwei
>Priority: Major
> Labels: pull-request-available
>
> Support partial update for merge-into statement, just like this:
> {code:java}
> merge into h0
> using s0
> on s0.id = h0.id
> when matched then update set price = s0.price + 10
> when not matched then insert *{code}
> Allow to update partial fields.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hudi] hudi-bot edited a comment on pull request #3154: [HUDI-1884] MergeInto Support Partial Update For COW
hudi-bot edited a comment on pull request #3154: URL: https://github.com/apache/hudi/pull/3154#issuecomment-868360998 ## CI report: * 63cf96134656dbb41eaab4e03e747776599255b6 UNKNOWN * 82b4b8bef8d143a14d445ae35fa5ec8fbbd5889f UNKNOWN * f5511851a56f5a23263bfb05690217718a329254 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=940) * f5f0fd3f1eb1529643b7129349614ed80d106bed UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HUDI-2184) Support setting hive sync partition extractor class based on flink configuration
[ https://issues.apache.org/jira/browse/HUDI-2184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381713#comment-17381713 ] ASF GitHub Bot commented on HUDI-2184: -- codecov-commenter edited a comment on pull request #3284: URL: https://github.com/apache/hudi/pull/3284#issuecomment-881148186 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3284](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (66d9b95) into [master](https://codecov.io/gh/apache/hudi/commit/a62a6cff32e7d7f62ffdf6362fb8876497fd0528?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a62a6cf) will **decrease** coverage by `31.95%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3284 +/- ## = - Coverage 47.70% 15.75% -31.96% + Complexity 5550 493 -5057 = Files 936 284 -652 Lines 4162411830-29794 Branches 4188 981 -3207 = - Hits 19858 1864-17994 + Misses2 9803-10197 + Partials 1766 163 -1603 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `0.00% <ø> (-34.47%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `4.88% <ø> (-51.10%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `59.26% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...main/java/org/apache/hudi/metrics/HoodieGauge.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvSG9vZGllR2F1Z2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../java/org/apache/hudi/metrics/MetricsReporter.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/metrics/MetricsReporterType.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyVHlwZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...rg/apache/hudi/client/bootstrap/BootstrapMode.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC9ib290c3RyYXAvQm9vdHN0cmFwTW9kZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3284: [HUDI-2184] Support setting hive sync partition extractor class based on flink co…
codecov-commenter edited a comment on pull request #3284: URL: https://github.com/apache/hudi/pull/3284#issuecomment-881148186 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3284](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (66d9b95) into [master](https://codecov.io/gh/apache/hudi/commit/a62a6cff32e7d7f62ffdf6362fb8876497fd0528?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a62a6cf) will **decrease** coverage by `31.95%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3284 +/- ## = - Coverage 47.70% 15.75% -31.96% + Complexity 5550 493 -5057 = Files 936 284 -652 Lines 4162411830-29794 Branches 4188 981 -3207 = - Hits 19858 1864-17994 + Misses2 9803-10197 + Partials 1766 163 -1603 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `0.00% <ø> (-34.47%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `4.88% <ø> (-51.10%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `59.26% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3284?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...main/java/org/apache/hudi/metrics/HoodieGauge.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvSG9vZGllR2F1Z2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../java/org/apache/hudi/metrics/MetricsReporter.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/metrics/MetricsReporterType.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyVHlwZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...rg/apache/hudi/client/bootstrap/BootstrapMode.java](https://codecov.io/gh/apache/hudi/pull/3284/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC9ib290c3RyYXAvQm9vdHN0cmFwTW9kZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
