[GitHub] [hudi] hudi-bot commented on pull request #7168: [HUDI-5185] Compaction run fails with --hoodieConfigs

2022-11-08 Thread GitBox
hudi-bot commented on PR #7168: URL: https://github.com/apache/hudi/pull/7168#issuecomment-1308348034 ## CI report: * b3db96540b41c7281acfc64349e872e37a0ff0f5 Azure:

[GitHub] [hudi] xiarixiaoyao commented on pull request #7129: [MINOR] Support column type evolution for Hive

2022-11-08 Thread GitBox
xiarixiaoyao commented on PR #7129: URL: https://github.com/apache/hudi/pull/7129#issuecomment-1308344730 run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] hudi-bot commented on pull request #7168: [HUDI-5185] Compaction run fails with --hoodieConfigs

2022-11-08 Thread GitBox
hudi-bot commented on PR #7168: URL: https://github.com/apache/hudi/pull/7168#issuecomment-1308343238 ## CI report: * b3db96540b41c7281acfc64349e872e37a0ff0f5 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7167: [HUDI-5094] Remove partition fields before transform bytes to avro,if enable DROP_PARTITION_COLUMNS.

2022-11-08 Thread GitBox
hudi-bot commented on PR #7167: URL: https://github.com/apache/hudi/pull/7167#issuecomment-1308343193 ## CI report: * eff6224be5ca3b205470ae473ea7ac54ed1012bc Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7167: [HUDI-5094] Remove partition fields before transform bytes to avro,if enable DROP_PARTITION_COLUMNS.

2022-11-08 Thread GitBox
hudi-bot commented on PR #7167: URL: https://github.com/apache/hudi/pull/7167#issuecomment-1308338412 ## CI report: * eff6224be5ca3b205470ae473ea7ac54ed1012bc UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] xicm commented on a diff in pull request #7168: [HUDI-5185] Compaction run fails with --hoodieConfigs

2022-11-08 Thread GitBox
xicm commented on code in PR #7168: URL: https://github.com/apache/hudi/pull/7168#discussion_r1017508563 ## hudi-cli/src/main/java/org/apache/hudi/cli/commands/SparkMain.java: ## @@ -151,7 +151,7 @@ public static void main(String[] args) throws Exception { }

[GitHub] [hudi] danny0405 commented on pull request #5830: [HUDI-3981][RFC-33] Flink engine support for comprehensive schema evolution

2022-11-08 Thread GitBox
danny0405 commented on PR #5830: URL: https://github.com/apache/hudi/pull/5830#issuecomment-1308308066 @trushev Can you rebase the code again and i'm planning to review this code again ~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [hudi] yihua commented on a diff in pull request #6358: [HUDI-4588][HUDI-4472] Revisiting schema handling in the write path to permit auto-evolution with columns being dropped

2022-11-08 Thread GitBox
yihua commented on code in PR #6358: URL: https://github.com/apache/hudi/pull/6358#discussion_r1006166832 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java: ## @@ -412,15 +412,14 @@ public class HoodieWriteConfig extends HoodieConfig

[GitHub] [hudi] xicm commented on a diff in pull request #7168: [HUDI-5185] Compaction run fails with --hoodieConfigs

2022-11-08 Thread GitBox
xicm commented on code in PR #7168: URL: https://github.com/apache/hudi/pull/7168#discussion_r1017505572 ## hudi-cli/src/main/java/org/apache/hudi/cli/commands/CompactionCommand.java: ## @@ -220,13 +220,13 @@ public String compact( @ShellOption(value = {"--parallelism"},

[GitHub] [hudi] danny0405 commented on issue #7162: [SUPPORT] Flink stream api(HoodieFlinkStreamer) write data to hudi create much rollbackfile

2022-11-08 Thread GitBox
danny0405 commented on issue #7162: URL: https://github.com/apache/hudi/issues/7162#issuecomment-1308303368 Did you find any error logs in the JM logging ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[jira] [Updated] (HUDI-5185) Compaction run fails with --hoodieConfigs

2022-11-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5185: - Labels: pull-request-available (was: ) > Compaction run fails with --hoodieConfigs >

[GitHub] [hudi] xicm opened a new pull request, #7168: [HUDI-5185] Compaction run fails with --hoodieConfigs

2022-11-08 Thread GitBox
xicm opened a new pull request, #7168: URL: https://github.com/apache/hudi/pull/7168 ### Change Logs Fix params extraction of "compaction run" in SparkMain. ### Impact compaction run ### Risk level (write none, low medium or high below) low _If medium or

[jira] [Updated] (HUDI-5185) Compaction run fails with --hoodieConfigs

2022-11-08 Thread xi chaomin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xi chaomin updated HUDI-5185: - Description: compaction run --schemaFilePath /tmp/compaction.schema --hoodieConfigs

[jira] [Updated] (HUDI-5185) Compaction run fails with --hoodieConfigs

2022-11-08 Thread xi chaomin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xi chaomin updated HUDI-5185: - Description: compaction run --schemaFilePath /tmp/compaction.schema --hoodieConfigs

[jira] [Created] (HUDI-5185) Compaction run fails with --hoodieConfigs

2022-11-08 Thread xi chaomin (Jira)
xi chaomin created HUDI-5185: Summary: Compaction run fails with --hoodieConfigs Key: HUDI-5185 URL: https://issues.apache.org/jira/browse/HUDI-5185 Project: Apache Hudi Issue Type: Bug

[GitHub] [hudi] eric9204 opened a new pull request, #7167: [HUDI-5094] Remove partition fields before transform bytes to avro,if enable DROP_PARTITION_COLUMNS.

2022-11-08 Thread GitBox
eric9204 opened a new pull request, #7167: URL: https://github.com/apache/hudi/pull/7167 ### Change Logs when enable DROP_PARTITION_COLUMNS, spark streaming failed after first compaction. The compaction action changed schema which in the write config, Hoodie Record Payload can't

[GitHub] [hudi] danny0405 commented on a diff in pull request #7166: [MINOR] Refactor the code for CkpMetadata

2022-11-08 Thread GitBox
danny0405 commented on code in PR #7166: URL: https://github.com/apache/hudi/pull/7166#discussion_r1017473954 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/meta/CkpMetadata.java: ## @@ -61,7 +61,9 @@ public class CkpMetadata implements Serializable {

[GitHub] [hudi] eric9204 closed pull request #7063: [HUDI-5094] Remove partition fields before transform bytes to avro,if enable DROP_PARTITION_COLUMNS

2022-11-08 Thread GitBox
eric9204 closed pull request #7063: [HUDI-5094] Remove partition fields before transform bytes to avro,if enable DROP_PARTITION_COLUMNS URL: https://github.com/apache/hudi/pull/7063 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] hudi-bot commented on pull request #7163: [HUDI-5171] Ensure validateTableConfig also checks for partition path field value switch

2022-11-08 Thread GitBox
hudi-bot commented on PR #7163: URL: https://github.com/apache/hudi/pull/7163#issuecomment-1308270421 ## CI report: * 215fdcbfd06f912d5b0776f5d7e9a7d4238a5135 Azure:

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7166: [MINOR] Refactor the code for CkpMetadata

2022-11-08 Thread GitBox
SteNicholas commented on code in PR #7166: URL: https://github.com/apache/hudi/pull/7166#discussion_r1017451567 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/meta/CkpMetadata.java: ## @@ -61,7 +61,9 @@ public class CkpMetadata implements Serializable {

[GitHub] [hudi] eshu commented on issue #7154: [SUPPORT] Hudi 0.12.2 release (Unknown versionCode:5)

2022-11-08 Thread GitBox
eshu commented on issue #7154: URL: https://github.com/apache/hudi/issues/7154#issuecomment-1308229629 @fengjian428 @nsivabalan ``` 2022-11-09 05:18:57,234 INFO table.HoodieTableMetaClient: Loading HoodieTableMetaClient from ... 2022-11-09 05:18:57,363 INFO

[GitHub] [hudi] hudi-bot commented on pull request #7129: [MINOR] Support column type evolution for Hive

2022-11-08 Thread GitBox
hudi-bot commented on PR #7129: URL: https://github.com/apache/hudi/pull/7129#issuecomment-1308221730 ## CI report: * f6a718f53a29aeb96a4488a848b94656485a0e3d Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7074: [HUDI-5101] Adding spark-structured streaming test support via spark-submit job

2022-11-08 Thread GitBox
hudi-bot commented on PR #7074: URL: https://github.com/apache/hudi/pull/7074#issuecomment-1308221646 ## CI report: * 3c66a4475a5f312ab6241d0f74684197019494c7 Azure:

[hudi] branch master updated: [HUDI-4526] Improve spillableMapBasePath when disk directory is full (#6284)

2022-11-08 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 371296173a [HUDI-4526] Improve

[GitHub] [hudi] xushiyan merged pull request #6284: [HUDI-4526] Improve spillableMapBasePath disk directory is full

2022-11-08 Thread GitBox
xushiyan merged PR #6284: URL: https://github.com/apache/hudi/pull/6284 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] hudi-bot commented on pull request #7074: [HUDI-5101] Adding spark-structured streaming test support via spark-submit job

2022-11-08 Thread GitBox
hudi-bot commented on PR #7074: URL: https://github.com/apache/hudi/pull/7074#issuecomment-1308217707 ## CI report: * 3c66a4475a5f312ab6241d0f74684197019494c7 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7092: [HUDI-5111][HUDI-5112][HUDI-5113] Enchancing integ test support

2022-11-08 Thread GitBox
hudi-bot commented on PR #7092: URL: https://github.com/apache/hudi/pull/7092#issuecomment-1308212813 ## CI report: * cfdf9292f6d5d702b2c71b440de6667b77cb4c40 Azure:

[jira] [Updated] (HUDI-3661) Flink async compaction is not thread safe when use watermark

2022-11-08 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-3661: - Summary: Flink async compaction is not thread safe when use watermark (was: flink async compaction is

[jira] [Commented] (HUDI-3661) flink async compaction is not threadsafe when use watermark

2022-11-08 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17630751#comment-17630751 ] Danny Chen commented on HUDI-3661: -- Thanks, maybe you are right, let's see if there is back pressure when

[jira] [Resolved] (HUDI-5025) Rollback failed with log file not found when rollOver in rollback process

2022-11-08 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-5025. -- > Rollback failed with log file not found when rollOver in rollback process >

[jira] [Updated] (HUDI-5025) Rollback failed with log file not found when rollOver in rollback process

2022-11-08 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-5025: - Fix Version/s: 0.12.2 0.13.0 > Rollback failed with log file not found when rollOver

[jira] [Commented] (HUDI-5025) Rollback failed with log file not found when rollOver in rollback process

2022-11-08 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17630749#comment-17630749 ] Danny Chen commented on HUDI-5025: -- Fixed via master branch: d047fbaed006beaa592f3a58705ce0e998d769e6 >

[hudi] branch master updated: [HUDI-5025] Rollback failed with log file not found when rollOver in rollback process (#6939)

2022-11-08 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new d047fbaed0 [HUDI-5025] Rollback failed with

[GitHub] [hudi] danny0405 merged pull request #6939: [HUDI-5025] Rollback failed with log file not found when rollOver in rollback process

2022-11-08 Thread GitBox
danny0405 merged PR #6939: URL: https://github.com/apache/hudi/pull/6939 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] danny0405 commented on pull request #6100: [HUDI-4392]Flink MOR table compaction plan execution sequence should …

2022-11-08 Thread GitBox
danny0405 commented on PR #6100: URL: https://github.com/apache/hudi/pull/6100#issuecomment-1308195042 Close because it is already resolved. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] danny0405 commented on a diff in pull request #7151: [MINOR] Performance improvement of flink ITs with reused miniCluster

2022-11-08 Thread GitBox
danny0405 commented on code in PR #7151: URL: https://github.com/apache/hudi/pull/7151#discussion_r1017413148 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/utils/AbstractHoodieTestBase.java: ## @@ -0,0 +1,76 @@ +/* + * Licensed to the Apache Software

[GitHub] [hudi] hudi-bot commented on pull request #7092: [HUDI-5111][HUDI-5112][HUDI-5113] Enchancing integ test support

2022-11-08 Thread GitBox
hudi-bot commented on PR #7092: URL: https://github.com/apache/hudi/pull/7092#issuecomment-1308176881 ## CI report: * cfdf9292f6d5d702b2c71b440de6667b77cb4c40 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7166: [minor] Refactor the code for CkpMetadata

2022-11-08 Thread GitBox
hudi-bot commented on PR #7166: URL: https://github.com/apache/hudi/pull/7166#issuecomment-1308173601 ## CI report: * 690e75b4e1e6aed3d77601d2949ad3e37a950457 Azure:

[GitHub] [hudi] nsivabalan commented on a diff in pull request #7092: [HUDI-5111][HUDI-5112][HUDI-5113] Enchancing integ test support

2022-11-08 Thread GitBox
nsivabalan commented on code in PR #7092: URL: https://github.com/apache/hudi/pull/7092#discussion_r1017399907 ## hudi-integ-test/src/main/java/org/apache/hudi/integ/testsuite/dag/nodes/ValidateDatasetNode.java: ## @@ -51,7 +51,7 @@ public Dataset

[GitHub] [hudi] hudi-bot commented on pull request #7166: [minor] Refactor the code for CkpMetadata

2022-11-08 Thread GitBox
hudi-bot commented on PR #7166: URL: https://github.com/apache/hudi/pull/7166#issuecomment-1308167449 ## CI report: * 690e75b4e1e6aed3d77601d2949ad3e37a950457 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] danny0405 opened a new pull request, #7166: [minor] Refactor the code for CkpMetadata

2022-11-08 Thread GitBox
danny0405 opened a new pull request, #7166: URL: https://github.com/apache/hudi/pull/7166 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any

[GitHub] [hudi] hudi-bot commented on pull request #7139: [HUDI-5160] Spark df saveAsTable failed with CTAS

2022-11-08 Thread GitBox
hudi-bot commented on PR #7139: URL: https://github.com/apache/hudi/pull/7139#issuecomment-1308164287 ## CI report: * 97e16f0aafd48e89bb47784a199528b947dd6624 Azure:

[jira] [Commented] (HUDI-5067) Merge the columns stats of multiple log blocks from the same log file

2022-11-08 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17630722#comment-17630722 ] Danny Chen commented on HUDI-5067: -- Fixed via master branch: 6183f684dcd07dab777eaa5341dfa712b955b942 >

[jira] [Resolved] (HUDI-5067) Merge the columns stats of multiple log blocks from the same log file

2022-11-08 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-5067. -- > Merge the columns stats of multiple log blocks from the same log file >

[hudi] branch master updated (1d1181a441 -> 6183f684dc)

2022-11-08 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 1d1181a441 [HUDI-5178] Add Call show_table_properties for spark sql (#7161) add 6183f684dc [HUDI-5067] Merge

[GitHub] [hudi] danny0405 merged pull request #7018: [HUDI-5067] Merge the columns stats of multiple log blocks from the s…

2022-11-08 Thread GitBox
danny0405 merged PR #7018: URL: https://github.com/apache/hudi/pull/7018 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] YannByron commented on pull request #7063: [HUDI-5094] Remove partition fields before transform bytes to avro,if enable DROP_PARTITION_COLUMNS

2022-11-08 Thread GitBox
YannByron commented on PR #7063: URL: https://github.com/apache/hudi/pull/7063#issuecomment-1308149442 @eric9204 please have UT coverage on the changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hudi] YannByron commented on a diff in pull request #7063: [HUDI-5094] Remove partition fields before transform bytes to avro,if enable DROP_PARTITION_COLUMNS

2022-11-08 Thread GitBox
YannByron commented on code in PR #7063: URL: https://github.com/apache/hudi/pull/7063#discussion_r1017368905 ## hudi-common/src/main/java/org/apache/hudi/common/model/HoodieRecordPayload.java: ## @@ -116,7 +121,16 @@ default Option combineAndGetUpdateValue(IndexedRecord

[GitHub] [hudi] YannByron commented on a diff in pull request #7063: [HUDI-5094] Remove partition fields before transform bytes to avro,if enable DROP_PARTITION_COLUMNS

2022-11-08 Thread GitBox
YannByron commented on code in PR #7063: URL: https://github.com/apache/hudi/pull/7063#discussion_r1017364310 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieAppendHandle.java: ## @@ -499,7 +501,7 @@ public List close() { public void write(Map>

[GitHub] [hudi] YannByron commented on pull request #7140: [HUDI-5163] Fixing failure handling with spark datasource write

2022-11-08 Thread GitBox
YannByron commented on PR #7140: URL: https://github.com/apache/hudi/pull/7140#issuecomment-1308132400 > @YannByron : I might need some help to take this patch and make valid fixes for spark-sql classes. Also, we might need to write tests. if you can loop in someone, would be nice.

[GitHub] [hudi] vinothchandar commented on pull request #3312: [HUDI-648][RFC-20] Implement error log/table for Datasource/DeltaStreamer/WriteClient/Compaction writes

2022-11-08 Thread GitBox
vinothchandar commented on PR #3312: URL: https://github.com/apache/hudi/pull/3312#issuecomment-1308123623 @liujinhui1994 any interest in reviving this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[hudi] branch master updated: [HUDI-5178] Add Call show_table_properties for spark sql (#7161)

2022-11-08 Thread mengtao
This is an automated email from the ASF dual-hosted git repository. mengtao pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 1d1181a441 [HUDI-5178] Add Call

[GitHub] [hudi] xiarixiaoyao merged pull request #7161: [HUDI-5178] Add Call show_table_properties for spark sql

2022-11-08 Thread GitBox
xiarixiaoyao merged PR #7161: URL: https://github.com/apache/hudi/pull/7161 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] hudi-bot commented on pull request #7163: [HUDI-5171] Ensure validateTableConfig also checks for partition path field value switch

2022-11-08 Thread GitBox
hudi-bot commented on PR #7163: URL: https://github.com/apache/hudi/pull/7163#issuecomment-1308118886 ## CI report: * f648001b13efa2674c94f6cc40cc3425f63305a5 Azure:

[GitHub] [hudi] jonvex commented on issue #7165: [SUPPORT]problems about Hudi in GCP

2022-11-08 Thread GitBox
jonvex commented on issue #7165: URL: https://github.com/apache/hudi/issues/7165#issuecomment-1308117762 Hi, There are python examples in https://hudi.apache.org/docs/quick-start-guide. There is also an example python file

[jira] [Created] (HUDI-5184) Remove export PYSPARK_SUBMIT_ARGS="--master local[*]" from HoodiePySparkQuickstart.py

2022-11-08 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-5184: - Summary: Remove export PYSPARK_SUBMIT_ARGS="--master local[*]" from HoodiePySparkQuickstart.py Key: HUDI-5184 URL: https://issues.apache.org/jira/browse/HUDI-5184

[GitHub] [hudi] hudi-bot commented on pull request #7163: [HUDI-5171] Ensure validateTableConfig also checks for partition path field value switch

2022-11-08 Thread GitBox
hudi-bot commented on PR #7163: URL: https://github.com/apache/hudi/pull/7163#issuecomment-1308115634 ## CI report: * f648001b13efa2674c94f6cc40cc3425f63305a5 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7129: [MINOR] Support column type evolution for Hive

2022-11-08 Thread GitBox
hudi-bot commented on PR #7129: URL: https://github.com/apache/hudi/pull/7129#issuecomment-1308115541 ## CI report: * Unknown: [CANCELED](TBD) * f6a718f53a29aeb96a4488a848b94656485a0e3d Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7163: [HUDI-5171] Ensure validateTableConfig also checks for partition path field value switch

2022-11-08 Thread GitBox
hudi-bot commented on PR #7163: URL: https://github.com/apache/hudi/pull/7163#issuecomment-1308112130 ## CI report: * f648001b13efa2674c94f6cc40cc3425f63305a5 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7139: [HUDI-5160] Spark df saveAsTable failed with CTAS

2022-11-08 Thread GitBox
hudi-bot commented on PR #7139: URL: https://github.com/apache/hudi/pull/7139#issuecomment-1308111954 ## CI report: * 98b075ea2a7e6c64f78b0b95812f98194eb6e919 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7129: [MINOR] Support column type evolution for Hive

2022-11-08 Thread GitBox
hudi-bot commented on PR #7129: URL: https://github.com/apache/hudi/pull/7129#issuecomment-1308111905 ## CI report: * Unknown: [CANCELED](TBD) * f6a718f53a29aeb96a4488a848b94656485a0e3d UNKNOWN Bot commands @hudi-bot supports the following commands: -

[GitHub] [hudi] hudi-bot commented on pull request #6939: [HUDI-5025] Rollback failed with log file not found when rollOver in rollback process

2022-11-08 Thread GitBox
hudi-bot commented on PR #6939: URL: https://github.com/apache/hudi/pull/6939#issuecomment-1308111634 ## CI report: * 33e1c0d4e837080818ddd7faf5517c7f0c2ca3dd Azure:

[GitHub] [hudi] xushiyan commented on issue #7165: [SUPPORT]problems about Hudi in GCP

2022-11-08 Thread GitBox
xushiyan commented on issue #7165: URL: https://github.com/apache/hudi/issues/7165#issuecomment-1308108979 @jonvex can you handle this please? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] danny0405 commented on pull request #6939: [HUDI-5025] Rollback failed with log file not found when rollOver in rollback process

2022-11-08 Thread GitBox
danny0405 commented on PR #6939: URL: https://github.com/apache/hudi/pull/6939#issuecomment-1308096230 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] fsilent commented on pull request #7129: [MINOR] Support column type evolution for Hive

2022-11-08 Thread GitBox
fsilent commented on PR #7129: URL: https://github.com/apache/hudi/pull/7129#issuecomment-1308066457 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] hudi-bot commented on pull request #7139: [HUDI-5160] Spark df saveAsTable failed with CTAS

2022-11-08 Thread GitBox
hudi-bot commented on PR #7139: URL: https://github.com/apache/hudi/pull/7139#issuecomment-1308065862 ## CI report: * 98b075ea2a7e6c64f78b0b95812f98194eb6e919 Azure:

[GitHub] [hudi] fangzai opened a new issue, #7165: [SUPPORT]problems about Hudi in GCP

2022-11-08 Thread GitBox
fangzai opened a new issue, #7165: URL: https://github.com/apache/hudi/issues/7165 My team want to use Hudi in GCP. We want to create Hudi table and register it in Bigquery table. We find the official guide for it-https://hudi.apache.org/docs/gcp_bigquery.

[GitHub] [hudi] dongkelun commented on pull request #6419: [HUDI-2057] CTAS Generate An External Table When Create Managed Table

2022-11-08 Thread GitBox
dongkelun commented on PR #6419: URL: https://github.com/apache/hudi/pull/6419#issuecomment-1308062982 @yihua @xushiyan Hello, I have added the validation to test the CTAS table type -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[hudi] branch master updated: [HUDI-5154] Improve hudi-spark-client Lambada writing (#7127)

2022-11-08 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new b3292ebf4a [HUDI-5154] Improve hudi-spark-client

[GitHub] [hudi] yihua merged pull request #7127: [HUDI-5154] Improve hudi-spark-client Lambada writing

2022-11-08 Thread GitBox
yihua merged PR #7127: URL: https://github.com/apache/hudi/pull/7127 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] waywtdcc commented on issue #7073: [BUG] Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

2022-11-08 Thread GitBox
waywtdcc commented on issue #7073: URL: https://github.com/apache/hudi/issues/7073#issuecomment-1308024853 Can this be added to the 0.13 plan? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[jira] [Updated] (HUDI-4888) Add validation to block COW table to use consistent hashing bucket index

2022-11-08 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-4888: -- Status: In Progress (was: Open) > Add validation to block COW table to use consistent hashing

[GitHub] [hudi] hudi-bot commented on pull request #7163: [HUDI-5171] Ensure validateTableConfig also checks for partition path field value switch

2022-11-08 Thread GitBox
hudi-bot commented on PR #7163: URL: https://github.com/apache/hudi/pull/7163#issuecomment-1307994077 ## CI report: * d6551e269d14b61d3a8776d227b27f9045b463f8 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7163: [HUDI-5171] Ensure validateTableConfig also checks for partition path field value switch

2022-11-08 Thread GitBox
hudi-bot commented on PR #7163: URL: https://github.com/apache/hudi/pull/7163#issuecomment-1307990174 ## CI report: * d6551e269d14b61d3a8776d227b27f9045b463f8 Azure:

[jira] [Updated] (HUDI-5036) Update contribution guide based on PR validation

2022-11-08 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5036: -- Status: Patch Available (was: In Progress) > Update contribution guide based on PR validation

[jira] [Created] (HUDI-5183) Cloudwatch mertics won't work for CLI

2022-11-08 Thread Shawn Chang (Jira)
Shawn Chang created HUDI-5183: - Summary: Cloudwatch mertics won't work for CLI Key: HUDI-5183 URL: https://issues.apache.org/jira/browse/HUDI-5183 Project: Apache Hudi Issue Type: Bug

[jira] [Closed] (HUDI-5048) add CopyToTempView support

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5048. - Resolution: Fixed > add CopyToTempView support > -- > >

[jira] [Closed] (HUDI-5085) When a flink job has multiple sink tables, the index loading status is abnormal.

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5085. - Resolution: Fixed > When a flink job has multiple sink tables, the index loading status

[jira] [Updated] (HUDI-5048) add CopyToTempView support

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5048: -- Fix Version/s: 0.12.2 > add CopyToTempView support > -- > >

[jira] [Closed] (HUDI-5089) Refactor HoodieCommitMetadata deserialization

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5089. - Resolution: Fixed > Refactor HoodieCommitMetadata deserialization >

[jira] [Closed] (HUDI-5058) The primary key cannot be empty when Flink reads an error from the hudi table

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5058. - Resolution: Fixed > The primary key cannot be empty when Flink reads an error from the

[jira] [Closed] (HUDI-4946) merge into with no preCombineField has dup row in only insert

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-4946. - Resolution: Fixed > merge into with no preCombineField has dup row in only insert >

[jira] [Closed] (HUDI-5072) Extract transform duplicate code

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5072. - Resolution: Fixed > Extract transform duplicate code > >

[jira] [Updated] (HUDI-5090) throw runtime Exception when flink streming job checkpoint abort

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5090: -- Fix Version/s: 0.12.2 > throw runtime Exception when flink streming job checkpoint

[jira] [Closed] (HUDI-5090) throw runtime Exception when flink streming job checkpoint abort

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5090. - Resolution: Fixed > throw runtime Exception when flink streming job checkpoint abort >

[jira] [Updated] (HUDI-4946) merge into with no preCombineField has dup row in only insert

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4946: -- Fix Version/s: 0.12.2 > merge into with no preCombineField has dup row in only insert >

[jira] [Updated] (HUDI-5072) Extract transform duplicate code

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5072: -- Fix Version/s: 0.12.2 > Extract transform duplicate code >

[jira] [Updated] (HUDI-5072) Extract transform duplicate code

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5072: -- Status: Open (was: In Progress) > Extract transform duplicate code >

[jira] [Closed] (HUDI-5000) Support schema evolution for Hive

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5000. - Resolution: Fixed > Support schema evolution for Hive > -

[jira] [Updated] (HUDI-1570) Add Avg record size in commit metadata

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-1570: -- Fix Version/s: 0.12.2 > Add Avg record size in commit metadata >

[jira] [Created] (HUDI-5182) Create Hudi CLI Bundle

2022-11-08 Thread Rahil Chertara (Jira)
Rahil Chertara created HUDI-5182: Summary: Create Hudi CLI Bundle Key: HUDI-5182 URL: https://issues.apache.org/jira/browse/HUDI-5182 Project: Apache Hudi Issue Type: Task

[jira] [Updated] (HUDI-5035) Remove deprecated API usage in SparkPreCommitValidator#validate

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5035: -- Fix Version/s: 0.12.2 > Remove deprecated API usage in SparkPreCommitValidator#validate

[jira] [Updated] (HUDI-5035) Remove deprecated API usage in SparkPreCommitValidator#validate

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5035: -- Status: Open (was: In Progress) > Remove deprecated API usage in

[jira] [Closed] (HUDI-5035) Remove deprecated API usage in SparkPreCommitValidator#validate

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5035. - Resolution: Fixed > Remove deprecated API usage in SparkPreCommitValidator#validate >

[jira] [Closed] (HUDI-5102) source operator(monitor and reader) support user uid

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5102. - Resolution: Fixed > source operator(monitor and reader) support user uid >

[jira] [Closed] (HUDI-4839) rocksdbjni is not compatible with apple silicon

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-4839. - Resolution: Fixed > rocksdbjni is not compatible with apple silicon >

[jira] [Closed] (HUDI-5057) Fix msck repair hudi table

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5057. - Fix Version/s: 0.12.2 Resolution: Fixed > Fix msck repair hudi table >

[jira] [Updated] (HUDI-4839) rocksdbjni is not compatible with apple silicon

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4839: -- Fix Version/s: 0.12.2 > rocksdbjni is not compatible with apple silicon >

[jira] [Closed] (HUDI-4741) Deadlock when restarting failed TM in AbstractStreamWriteFunction

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-4741. - Resolution: Fixed > Deadlock when restarting failed TM in AbstractStreamWriteFunction >

[jira] [Closed] (HUDI-5105) Add Call show_commit_extra_metadata for spark sql

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5105. - Fix Version/s: 0.12.2 Resolution: Fixed > Add Call show_commit_extra_metadata for

  1   2   >