[GitHub] [hudi] pratyakshsharma commented on a diff in pull request #6662: [HUDI-4832] Fix drop partition meta sync

2022-09-15 Thread GitBox
pratyakshsharma commented on code in PR #6662: URL: https://github.com/apache/hudi/pull/6662#discussion_r971690126 ## hudi-sync/hudi-hive-sync/src/test/java/org/apache/hudi/hive/TestHiveSyncTool.java: ## @@ -88,8 +88,8 @@ public class TestHiveSyncTool { private static fina

[GitHub] [hudi] gunjdesai commented on issue #6610: [QUESTION] Faster approach for backfilling older data without stopping realtime pipelines

2022-09-15 Thread GitBox
gunjdesai commented on issue #6610: URL: https://github.com/apache/hudi/issues/6610#issuecomment-124136 @xushiyan yes this is a spark structured streaming job. So we are running the job on K8S Spot instances, there are cases where we face driver eviction, hence we can't use multi-writer

[GitHub] [hudi] danny0405 commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue

2022-09-15 Thread GitBox
danny0405 commented on PR #6634: URL: https://github.com/apache/hudi/pull/6634#issuecomment-1247771867 Can you rebase with the latest master code and force push to re-trigger the tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6600: [RFC-62] Diagnostic Reporter

2022-09-15 Thread GitBox
zhangyue19921010 commented on code in PR #6600: URL: https://github.com/apache/hudi/pull/6600#discussion_r971684664 ## rfc/rfc-62/rfc-62.md: ## @@ -0,0 +1,443 @@ + +# RFC-62: Diagnostic Reporter + + + +## Proposers + +- zhangyue19921...@163.com + +## Approvers + - @codope + - @x

[GitHub] [hudi] hudi-bot commented on pull request #6677: [HUDI-4294][Stacked on 4293] Introduce build action to actually perform index data generation

2022-09-15 Thread GitBox
hudi-bot commented on PR #6677: URL: https://github.com/apache/hudi/pull/6677#issuecomment-1247752999 ## CI report: * 0ce0aee73e1641f071abdfc44d4f5473a425befb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1137

[GitHub] [hudi] hudi-bot commented on pull request #6678: [HUDI-4844] Skip partition value resolving when the field does not ex…

2022-09-15 Thread GitBox
hudi-bot commented on PR #6678: URL: https://github.com/apache/hudi/pull/6678#issuecomment-1247746229 ## CI report: * 8fee8139ce7ecf9271c6c40cd414c60a03d26eba Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1137

[GitHub] [hudi] hudi-bot commented on pull request #6677: [HUDI-4294][Stacked on 4293] Introduce build action to actually perform index data generation

2022-09-15 Thread GitBox
hudi-bot commented on PR #6677: URL: https://github.com/apache/hudi/pull/6677#issuecomment-1247746189 ## CI report: * 0ce0aee73e1641f071abdfc44d4f5473a425befb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1137

[GitHub] [hudi] huberylee commented on pull request #6677: [HUDI-4294][Stacked on 4293] Introduce build action to actually perform index data generation

2022-09-15 Thread GitBox
huberylee commented on PR #6677: URL: https://github.com/apache/hudi/pull/6677#issuecomment-1247742308 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [hudi] hudi-bot commented on pull request #6678: [HUDI-4844] Skip partition value resolving when the field does not ex…

2022-09-15 Thread GitBox
hudi-bot commented on PR #6678: URL: https://github.com/apache/hudi/pull/6678#issuecomment-1247739450 ## CI report: * 8fee8139ce7ecf9271c6c40cd414c60a03d26eba UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] TJX2014 commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue

2022-09-15 Thread GitBox
TJX2014 commented on PR #6634: URL: https://github.com/apache/hudi/pull/6634#issuecomment-1247733905 > TestSavepointsProcedure Hi @danny0405 seems the failed ci record is triggered in two days ago, the failed test is: - Test Create Hoodie Table With Options *** FAILED *** Exp

[GitHub] [hudi] hudi-bot commented on pull request #6677: [HUDI-4294][Stacked on 4293] Introduce build action to actually perform index data generation

2022-09-15 Thread GitBox
hudi-bot commented on PR #6677: URL: https://github.com/apache/hudi/pull/6677#issuecomment-1247731490 ## CI report: * 0ce0aee73e1641f071abdfc44d4f5473a425befb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1137

[jira] [Resolved] (HUDI-4780) hoodie.logfile.max.size It does not take effect, causing the log file to be too large

2022-09-15 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-4780. -- > hoodie.logfile.max.size It does not take effect, causing the log file to be > too large > --

[jira] [Commented] (HUDI-4780) hoodie.logfile.max.size It does not take effect, causing the log file to be too large

2022-09-15 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17605176#comment-17605176 ] Danny Chen commented on HUDI-4780: -- Fixed via master branch: 851c6e12db9fea9e7776dfc3da2d

[jira] [Updated] (HUDI-4780) hoodie.logfile.max.size It does not take effect, causing the log file to be too large

2022-09-15 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-4780: - Status: Open (was: In Progress) > hoodie.logfile.max.size It does not take effect, causing the log file

[GitHub] [hudi] danny0405 commented on pull request #6602: [HUDI-4780] hoodie.logfile.max.size It does not take effect, causing the log file to be too large

2022-09-15 Thread GitBox
danny0405 commented on PR #6602: URL: https://github.com/apache/hudi/pull/6602#issuecomment-1247722560 @yihua Can you help to add a test case here, wanna merge it first to make it into 0.12.1. -- This is an automated message from the Apache Git Service. To respond to the message, please l

[hudi] branch master updated (7ab59b9a0c -> 851c6e12db)

2022-09-15 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 7ab59b9a0c [HUDI-4843] Delete the useless timer in BaseRollbackActionExecutor (#6671) add 851c6e12db [HUDI-4780

[GitHub] [hudi] danny0405 merged pull request #6602: [HUDI-4780] hoodie.logfile.max.size It does not take effect, causing the log file to be too large

2022-09-15 Thread GitBox
danny0405 merged PR #6602: URL: https://github.com/apache/hudi/pull/6602 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache

[jira] [Commented] (HUDI-4843) Useless timer in rollback action

2022-09-15 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17605175#comment-17605175 ] Danny Chen commented on HUDI-4843: -- Fixed via master branch: 7ab59b9a0c40bde0dc4c110ca361

[jira] [Updated] (HUDI-4843) Useless timer in rollback action

2022-09-15 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-4843: - Fix Version/s: 0.13.0 > Useless timer in rollback action > > >

[jira] [Resolved] (HUDI-4843) Useless timer in rollback action

2022-09-15 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-4843. -- > Useless timer in rollback action > > > Key: HUDI-4843 >

[hudi] branch master updated (35d03e9a1b -> 7ab59b9a0c)

2022-09-15 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 35d03e9a1b [HUDI-4837] Stop sleeping where it is not necessary after the success (#6270) add 7ab59b9a0c [HUDI-4

[GitHub] [hudi] danny0405 merged pull request #6671: [HUDI-4843] Delete the useless timer in BaseRollbackActionExecutor

2022-09-15 Thread GitBox
danny0405 merged PR #6671: URL: https://github.com/apache/hudi/pull/6671 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache

[GitHub] [hudi] yesemsanthoshkumar closed issue #4784: [SUPPORT] Partition column not appearing in spark dataframe

2022-09-15 Thread GitBox
yesemsanthoshkumar closed issue #4784: [SUPPORT] Partition column not appearing in spark dataframe URL: https://github.com/apache/hudi/issues/4784 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] yesemsanthoshkumar commented on issue #4784: [SUPPORT] Partition column not appearing in spark dataframe

2022-09-15 Thread GitBox
yesemsanthoshkumar commented on issue #4784: URL: https://github.com/apache/hudi/issues/4784#issuecomment-1247710923 @nsivabalan and @codope I haven't tested it on 0.11. I'll close this issue for now and can be reopened when I or anyone encounter this again. Thanks -- This is an automated

[GitHub] [hudi] yesemsanthoshkumar commented on a diff in pull request #6350: [HUDI-4610] Document list of metrics emitted by Hudi

2022-09-15 Thread GitBox
yesemsanthoshkumar commented on code in PR #6350: URL: https://github.com/apache/hudi/pull/6350#discussion_r971631023 ## website/docs/metrics.md: ## @@ -201,3 +201,32 @@ These `HoodieMetrics` can then be plotted on a standard tool like grafana. Below + +## List of met

[GitHub] [hudi] danny0405 commented on pull request #6634: [HUDI-4813] Fix infer keygen not work in sparksql side issue

2022-09-15 Thread GitBox
danny0405 commented on PR #6634: URL: https://github.com/apache/hudi/pull/6634#issuecomment-1247696507 The CI shows that `TestSavepointsProcedure#Test Call rollback_to_savepoint Procedure` is still failing. -- This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [hudi] KnightChess opened a new issue, #6679: [SUPPORT] Expect job status failed in spark batch model

2022-09-15 Thread GitBox
KnightChess opened a new issue, #6679: URL: https://github.com/apache/hudi/issues/6679 **Describe the problem you faced** ```java @Override public void write(HoodieRecord record, Option avroRecord) { ... fileWriter.writeAvro(record.getRecordKey(),

[jira] [Updated] (HUDI-4844) Skip partition value resolving when the field does not exists for MergeOnReadInputFormat#getReader

2022-09-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4844: - Labels: pull-request-available (was: ) > Skip partition value resolving when the field does not e

[GitHub] [hudi] danny0405 opened a new pull request, #6678: [HUDI-4844] Skip partition value resolving when the field does not ex…

2022-09-15 Thread GitBox
danny0405 opened a new pull request, #6678: URL: https://github.com/apache/hudi/pull/6678 …ists for MergeOnReadInputFormat#getReader ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public A

[GitHub] [hudi] ThinkerLei commented on pull request #6384: [HUDI-4613] Avoid the use of regex expressions when call hoodieFileGroup#addLogFile function

2022-09-15 Thread GitBox
ThinkerLei commented on PR #6384: URL: https://github.com/apache/hudi/pull/6384#issuecomment-1247673437 @codope Thank you so much for your comments. I will work for this soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] hudi-bot commented on pull request #6677: [HUDI-4294][Stacked on 4293] Introduce build action to actually perform index data generation

2022-09-15 Thread GitBox
hudi-bot commented on PR #6677: URL: https://github.com/apache/hudi/pull/6677#issuecomment-1247669313 ## CI report: * 59e2196397ef68d75697a35b1b91e661ef9d3aa4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1137

<    1   2   3   4