[GitHub] [hudi] hudi-bot commented on pull request #7194: [HUDI-5206] RowColumnReader should not return null value for certain null child columns

2022-11-13 Thread GitBox
hudi-bot commented on PR #7194: URL: https://github.com/apache/hudi/pull/7194#issuecomment-1313234850 ## CI report: * 6253ae4e6ea771a8e9fe1cb0c407913347f28bcf Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1299

[GitHub] [hudi] XuQianJin-Stars commented on issue #7154: [SUPPORT] Hudi 0.12.2 release (Unknown versionCode:5)

2022-11-13 Thread GitBox
XuQianJin-Stars commented on issue #7154: URL: https://github.com/apache/hudi/issues/7154#issuecomment-1313229523 > @fengjian428 thanks for verifying. how about the downgrading 5->4? it also works fine. -- This is an automated message from the Apache Git Service. To respond to the m

[GitHub] [hudi] hudi-bot commented on pull request #7193: [HUDI-5203] Handle null fields in debezium avro payloads

2022-11-13 Thread GitBox
hudi-bot commented on PR #7193: URL: https://github.com/apache/hudi/pull/7193#issuecomment-1313229426 ## CI report: * 2c2d59181be075453bd97f26d6f81c2880a1cff9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1299

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7194: [HUDI-5206] RowColumnReader should not return null value for certain null child columns

2022-11-13 Thread GitBox
SteNicholas commented on code in PR #7194: URL: https://github.com/apache/hudi/pull/7194#discussion_r1021149627 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/table/ITTestHoodieDataSource.java: ## @@ -1303,8 +1303,8 @@ void testParquetComplexTypes(String opera

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7194: [HUDI-5206] RowColumnReader should not return null value for certain null child columns

2022-11-13 Thread GitBox
SteNicholas commented on code in PR #7194: URL: https://github.com/apache/hudi/pull/7194#discussion_r1021110485 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/table/ITTestHoodieDataSource.java: ## @@ -1303,8 +1303,8 @@ void testParquetComplexTypes(String opera

[GitHub] [hudi] trushev commented on pull request #7124: [HUDI-5151] Fix bug with broken flink data skipping caused by ClassNotFoundException of InLineFileSystem

2022-11-13 Thread GitBox
trushev commented on PR #7124: URL: https://github.com/apache/hudi/pull/7124#issuecomment-1313192391 Pls merge this fix -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

[GitHub] [hudi] eric9204 commented on a diff in pull request #7167: [HUDI-5094] Remove partition fields before transform bytes to avro,if enable DROP_PARTITION_COLUMNS.

2022-11-13 Thread GitBox
eric9204 commented on code in PR #7167: URL: https://github.com/apache/hudi/pull/7167#discussion_r1021136971 ## hudi-common/src/main/java/org/apache/hudi/common/model/HoodieRecordPayload.java: ## @@ -116,7 +121,15 @@ default Option combineAndGetUpdateValue(IndexedRecord current

[GitHub] [hudi] hudi-bot commented on pull request #7173: [HUDI-5189] Make HiveAvroSerializer compatible with hive3

2022-11-13 Thread GitBox
hudi-bot commented on PR #7173: URL: https://github.com/apache/hudi/pull/7173#issuecomment-1313173645 ## CI report: * 6b00f7e0849c61a2e841c4236f24824963191656 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1299

[GitHub] [hudi] hudi-bot commented on pull request #7194: [HUDI-5206] RowColumnReader should not return null value for certain null child columns

2022-11-13 Thread GitBox
hudi-bot commented on PR #7194: URL: https://github.com/apache/hudi/pull/7194#issuecomment-1313169943 ## CI report: * 6253ae4e6ea771a8e9fe1cb0c407913347f28bcf Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1299

[GitHub] [hudi] hudi-bot commented on pull request #7167: [HUDI-5094] Remove partition fields before transform bytes to avro,if enable DROP_PARTITION_COLUMNS.

2022-11-13 Thread GitBox
hudi-bot commented on PR #7167: URL: https://github.com/apache/hudi/pull/7167#issuecomment-1313169874 ## CI report: * 6885e2e5aab82b65a7cb1da2a72bdfc089eb42b4 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=129

[GitHub] [hudi] trushev commented on pull request #5830: [HUDI-3981][RFC-33] Flink engine support for comprehensive schema evolution

2022-11-13 Thread GitBox
trushev commented on PR #5830: URL: https://github.com/apache/hudi/pull/5830#issuecomment-1313166445 > [3981.patch.zip](https://github.com/apache/hudi/files/9977268/3981.patch.zip) Thanks for the contribution, have reviewed some of the part, and left a local patch here and some comments ~

[GitHub] [hudi] hudi-bot commented on pull request #7194: [HUDI-5206] RowColumnReader should not return null value for certain null child columns

2022-11-13 Thread GitBox
hudi-bot commented on PR #7194: URL: https://github.com/apache/hudi/pull/7194#issuecomment-1313166153 ## CI report: * 6253ae4e6ea771a8e9fe1cb0c407913347f28bcf UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7167: [HUDI-5094] Remove partition fields before transform bytes to avro,if enable DROP_PARTITION_COLUMNS.

2022-11-13 Thread GitBox
hudi-bot commented on PR #7167: URL: https://github.com/apache/hudi/pull/7167#issuecomment-1313166066 ## CI report: * 4598dc15c7b9f5092f90d3c3c889329e52bf9ae5 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=129

[GitHub] [hudi] TengHuo opened a new pull request, #7195: [DOCS][MINOR] Add bucket and Flink state index options and update the default in indexing.md doc

2022-11-13 Thread GitBox
TengHuo opened a new pull request, #7195: URL: https://github.com/apache/hudi/pull/7195 ### Change Logs Add bucket and Flink state index options and update the default in indexing.md doc ### Impact None ### Risk level (write none, low medium or high below)

[GitHub] [hudi] trushev commented on a diff in pull request #5830: [HUDI-3981][RFC-33] Flink engine support for comprehensive schema evolution

2022-11-13 Thread GitBox
trushev commented on code in PR #5830: URL: https://github.com/apache/hudi/pull/5830#discussion_r1021115903 ## hudi-common/src/main/java/org/apache/hudi/common/table/log/AbstractHoodieLogRecordReader.java: ## @@ -817,6 +805,8 @@ public abstract static class Builder { publ

[GitHub] [hudi] wulei0302 commented on a diff in pull request #6963: [WIP][HUDI-5040] flink multi writer for bucket index

2022-11-13 Thread GitBox
wulei0302 commented on code in PR #6963: URL: https://github.com/apache/hudi/pull/6963#discussion_r1021110814 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieAppendHandle.java: ## @@ -164,20 +166,31 @@ private void init(HoodieRecord record) { Stri

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7194: [HUDI-5206] RowColumnReader should not return null value for certain null child columns

2022-11-13 Thread GitBox
SteNicholas commented on code in PR #7194: URL: https://github.com/apache/hudi/pull/7194#discussion_r1021110485 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/table/ITTestHoodieDataSource.java: ## @@ -1303,8 +1303,8 @@ void testParquetComplexTypes(String opera

[GitHub] [hudi] danny0405 commented on a diff in pull request #7194: [HUDI-5206] RowColumnReader should not return null value for certain null child columns

2022-11-13 Thread GitBox
danny0405 commented on code in PR #7194: URL: https://github.com/apache/hudi/pull/7194#discussion_r1021106842 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/table/ITTestHoodieDataSource.java: ## @@ -1303,8 +1303,8 @@ void testParquetComplexTypes(String operati

[GitHub] [hudi] danny0405 commented on issue #7126: [SUPPORT]Flink 1.16

2022-11-13 Thread GitBox
danny0405 commented on issue #7126: URL: https://github.com/apache/hudi/issues/7126#issuecomment-1313141902 Cool, would support that if the 1.16.1 released. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hudi] zhangyue19921010 commented on pull request #5659: [HUDI-4137] SnowflakeSyncTool MVP implementation to integrate with Snowflake

2022-11-13 Thread GitBox
zhangyue19921010 commented on PR #5659: URL: https://github.com/apache/hudi/pull/5659#issuecomment-1313136378 Hey everyone. How is this amazing work going? We are planing to migrate our all historical data into Lakehouse using Hudi. At the same time, we are also investigating wheth

[GitHub] [hudi] trushev commented on a diff in pull request #5830: [HUDI-3981][RFC-33] Flink engine support for comprehensive schema evolution

2022-11-13 Thread GitBox
trushev commented on code in PR #5830: URL: https://github.com/apache/hudi/pull/5830#discussion_r1021098630 ## hudi-common/src/main/java/org/apache/hudi/internal/schema/convert/AvroInternalSchemaConverter.java: ## @@ -91,6 +91,11 @@ public static InternalSchema convert(Schema sc

[jira] [Closed] (HUDI-4496) ORC fails w/ Spark 3.1

2022-11-13 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-4496. Resolution: Fixed > ORC fails w/ Spark 3.1 > -- > > Key: HUDI-4496 >

[GitHub] [hudi] trushev commented on a diff in pull request #5830: [HUDI-3981][RFC-33] Flink engine support for comprehensive schema evolution

2022-11-13 Thread GitBox
trushev commented on code in PR #5830: URL: https://github.com/apache/hudi/pull/5830#discussion_r1021080563 ## hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFormatReader.java: ## @@ -141,4 +139,8 @@ public HoodieLogBlock prev() throws IOException { ret

[jira] [Updated] (HUDI-5206) RowColumnReader should not return null value for certain null child columns

2022-11-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5206: - Labels: pull-request-available (was: ) > RowColumnReader should not return null value for certain

[GitHub] [hudi] SteNicholas opened a new pull request, #7194: [HUDI-5206] RowColumnReader should not return null value for certain null child columns

2022-11-13 Thread GitBox
SteNicholas opened a new pull request, #7194: URL: https://github.com/apache/hudi/pull/7194 ### Change Logs When reading to vector of certain null child columns of row type column, `RowColumnReader` should not return null value because the value of the row type column may not be null

[jira] [Assigned] (HUDI-5206) RowColumnReader should not return null value for certain null child columns

2022-11-13 Thread Nicholas Jiang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Jiang reassigned HUDI-5206: Assignee: Nicholas Jiang > RowColumnReader should not return null value for certain null ch

[GitHub] [hudi] xushiyan commented on issue #7192: [SUPPORT] UPSERT failed with errors

2022-11-13 Thread GitBox
xushiyan commented on issue #7192: URL: https://github.com/apache/hudi/issues/7192#issuecomment-1313119111 > checkpoints are moved forward, but data was not processed), but data was not processed. It created a small piece of data, and finished without any meaningful information. @es

[jira] [Created] (HUDI-5206) RowColumnReader should not return null value for certain null child columns

2022-11-13 Thread Nicholas Jiang (Jira)
Nicholas Jiang created HUDI-5206: Summary: RowColumnReader should not return null value for certain null child columns Key: HUDI-5206 URL: https://issues.apache.org/jira/browse/HUDI-5206 Project: Apac

[GitHub] [hudi] hudi-bot commented on pull request #7167: [HUDI-5094] Remove partition fields before transform bytes to avro,if enable DROP_PARTITION_COLUMNS.

2022-11-13 Thread GitBox
hudi-bot commented on PR #7167: URL: https://github.com/apache/hudi/pull/7167#issuecomment-1313114614 ## CI report: * 4598dc15c7b9f5092f90d3c3c889329e52bf9ae5 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=129

[GitHub] [hudi] hudi-bot commented on pull request #7167: [HUDI-5094] Remove partition fields before transform bytes to avro,if enable DROP_PARTITION_COLUMNS.

2022-11-13 Thread GitBox
hudi-bot commented on PR #7167: URL: https://github.com/apache/hudi/pull/7167#issuecomment-1313111340 ## CI report: * b6078d75271edd5c9a4af7e13f087a043fa7869e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1299

[GitHub] [hudi] eshu commented on issue #7192: [SUPPORT] UPSERT failed with errors

2022-11-13 Thread GitBox
eshu commented on issue #7192: URL: https://github.com/apache/hudi/issues/7192#issuecomment-1313110216 @xushiyan Just take a look on code: https://github.com/apache/hudi/blob/release-0.11.1/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.sca

[GitHub] [hudi] trushev commented on a diff in pull request #5830: [HUDI-3981][RFC-33] Flink engine support for comprehensive schema evolution

2022-11-13 Thread GitBox
trushev commented on code in PR #5830: URL: https://github.com/apache/hudi/pull/5830#discussion_r1021080563 ## hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFormatReader.java: ## @@ -141,4 +139,8 @@ public HoodieLogBlock prev() throws IOException { ret

[GitHub] [hudi] xushiyan commented on issue #6283: [SUPPORT] No .marker files

2022-11-13 Thread GitBox
xushiyan commented on issue #6283: URL: https://github.com/apache/hudi/issues/6283#issuecomment-1313097158 @eshu how should we reproduce it? can you zip and share a problematic sample dataset so we can debug it? -- This is an automated message from the Apache Git Service. To respond to th

[GitHub] [hudi] xushiyan closed issue #7126: [SUPPORT]Flink 1.16

2022-11-13 Thread GitBox
xushiyan closed issue #7126: [SUPPORT]Flink 1.16 URL: https://github.com/apache/hudi/issues/7126 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits

[GitHub] [hudi] xushiyan commented on issue #7126: [SUPPORT]Flink 1.16

2022-11-13 Thread GitBox
xushiyan commented on issue #7126: URL: https://github.com/apache/hudi/issues/7126#issuecomment-1313092900 filed https://issues.apache.org/jira/browse/HUDI-5205 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[jira] [Assigned] (HUDI-5205) Support flink 1.16

2022-11-13 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-5205: Assignee: Danny Chen > Support flink 1.16 > -- > > Key: HUDI-5205 >

[jira] [Updated] (HUDI-5205) Support flink 1.16

2022-11-13 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-5205: - Fix Version/s: 1.0.0 > Support flink 1.16 > -- > > Key: HUDI-5205 >

[jira] [Created] (HUDI-5205) Support flink 1.16

2022-11-13 Thread Raymond Xu (Jira)
Raymond Xu created HUDI-5205: Summary: Support flink 1.16 Key: HUDI-5205 URL: https://issues.apache.org/jira/browse/HUDI-5205 Project: Apache Hudi Issue Type: New Feature Components: fl

[GitHub] [hudi] xushiyan commented on issue #7191: [SUPPORT] Missing Data with Amazon Athena in Glue Table with Hudi 0.10.1

2022-11-13 Thread GitBox
xushiyan commented on issue #7191: URL: https://github.com/apache/hudi/issues/7191#issuecomment-1313084953 @aniketnanna these are all aws managed services involved. have you filed aws support case? -- This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [hudi] hudi-bot commented on pull request #7193: [HUDI-5203] Handle null fields in debezium avro payloads

2022-11-13 Thread GitBox
hudi-bot commented on PR #7193: URL: https://github.com/apache/hudi/pull/7193#issuecomment-1313061581 ## CI report: * 2c2d59181be075453bd97f26d6f81c2880a1cff9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1299

[GitHub] [hudi] hudi-bot commented on pull request #7173: [HUDI-5189] Make HiveAvroSerializer compatible with hive3

2022-11-13 Thread GitBox
hudi-bot commented on PR #7173: URL: https://github.com/apache/hudi/pull/7173#issuecomment-1313061507 ## CI report: * 198db51b584ad83112d09ada850d97893ef1d055 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1292

[GitHub] [hudi] hudi-bot commented on pull request #7167: [HUDI-5094] Remove partition fields before transform bytes to avro,if enable DROP_PARTITION_COLUMNS.

2022-11-13 Thread GitBox
hudi-bot commented on PR #7167: URL: https://github.com/apache/hudi/pull/7167#issuecomment-1313061461 ## CI report: * b6078d75271edd5c9a4af7e13f087a043fa7869e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1299

[GitHub] [hudi] xushiyan commented on issue #7192: [SUPPORT] UPSERT failed with errors

2022-11-13 Thread GitBox
xushiyan commented on issue #7192: URL: https://github.com/apache/hudi/issues/7192#issuecomment-1313059588 what is the job setup? code snippet to run and hudi/spark configs. and what is the data schema? processing from where to where? -- This is an automated message from the Apache Git Se

[GitHub] [hudi] hudi-bot commented on pull request #7193: [HUDI-5203] Handle null fields in debezium avro payloads

2022-11-13 Thread GitBox
hudi-bot commented on PR #7193: URL: https://github.com/apache/hudi/pull/7193#issuecomment-1313059144 ## CI report: * 2c2d59181be075453bd97f26d6f81c2880a1cff9 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7173: [HUDI-5189] Make HiveAvroSerializer compatible with hive3

2022-11-13 Thread GitBox
hudi-bot commented on PR #7173: URL: https://github.com/apache/hudi/pull/7173#issuecomment-1313059118 ## CI report: * 198db51b584ad83112d09ada850d97893ef1d055 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1292

[GitHub] [hudi] hudi-bot commented on pull request #7167: [HUDI-5094] Remove partition fields before transform bytes to avro,if enable DROP_PARTITION_COLUMNS.

2022-11-13 Thread GitBox
hudi-bot commented on PR #7167: URL: https://github.com/apache/hudi/pull/7167#issuecomment-1313059088 ## CI report: * b6078d75271edd5c9a4af7e13f087a043fa7869e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1299

[GitHub] [hudi] hudi-bot commented on pull request #7167: [HUDI-5094] Remove partition fields before transform bytes to avro,if enable DROP_PARTITION_COLUMNS.

2022-11-13 Thread GitBox
hudi-bot commented on PR #7167: URL: https://github.com/apache/hudi/pull/7167#issuecomment-1313056891 ## CI report: * b6078d75271edd5c9a4af7e13f087a043fa7869e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1299

[GitHub] [hudi] xushiyan merged pull request #7181: [HUDI-5201] add totalRecordsDeleted metric

2022-11-13 Thread GitBox
xushiyan merged PR #7181: URL: https://github.com/apache/hudi/pull/7181 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.

[hudi] branch master updated: [HUDI-5201] Add totalRecordsDeleted metric (#7181)

2022-11-13 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 5ac605c9e7 [HUDI-5201] Add totalRecordsDeleted m

[jira] [Updated] (HUDI-5203) Debezium payload does not handle null-field cases

2022-11-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5203: - Labels: pull-request-available (was: ) > Debezium payload does not handle null-field cases >

[GitHub] [hudi] xushiyan opened a new pull request, #7193: [HUDI-5203] Handle null fields in debezium avro payloads

2022-11-13 Thread GitBox
xushiyan opened a new pull request, #7193: URL: https://github.com/apache/hudi/pull/7193 ### Change Logs Handle cases where event seq (mysql) or LSN (postgres) is null for debezium avro payload. ### Impact NA ### Risk level Low ### Documentation Upda

[GitHub] [hudi] hudi-bot commented on pull request #7167: [HUDI-5094] Remove partition fields before transform bytes to avro,if enable DROP_PARTITION_COLUMNS.

2022-11-13 Thread GitBox
hudi-bot commented on PR #7167: URL: https://github.com/apache/hudi/pull/7167#issuecomment-1312964391 ## CI report: * 82fcd037fe81678d19594378d08a53cf6308c5a3 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1294

[jira] [Created] (HUDI-5204) Unable to write data when Schema evolution is enabled

2022-11-13 Thread Wally Tang (Jira)
Wally Tang created HUDI-5204: Summary: Unable to write data when Schema evolution is enabled Key: HUDI-5204 URL: https://issues.apache.org/jira/browse/HUDI-5204 Project: Apache Hudi Issue Type: B

[GitHub] [hudi] fengjian428 commented on issue #7154: [SUPPORT] Hudi 0.12.2 release (Unknown versionCode:5)

2022-11-13 Thread GitBox
fengjian428 commented on issue #7154: URL: https://github.com/apache/hudi/issues/7154#issuecomment-1312931950 > @fengjian428 thanks for verifying. how about the downgrading 5->4? also works fine on my side ![image](https://user-images.githubusercontent.com/4403474/201558263-ab66c071-ed

[jira] [Resolved] (HUDI-5191) avro compatibility with spark3.2+

2022-11-13 Thread zouxxyy (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zouxxyy resolved HUDI-5191. --- > avro compatibility with spark3.2+ > - > > Key: HUDI-5191 >

[GitHub] [hudi] xiarixiaoyao commented on pull request #3391: [HUDI-83] Fix Timestamp/Date type read by Hive3

2022-11-13 Thread GitBox
xiarixiaoyao commented on PR #3391: URL: https://github.com/apache/hudi/pull/3391#issuecomment-1312919011 > @xiarixiaoyao This PR has been delayed for almost a year or so, and has been put on hold by review after each change of the code. Since I use presto less, my initial thought was to wa

[GitHub] [hudi] cdmikechen commented on pull request #3391: [HUDI-83] Fix Timestamp/Date type read by Hive3

2022-11-13 Thread GitBox
cdmikechen commented on PR #3391: URL: https://github.com/apache/hudi/pull/3391#issuecomment-1312868102 @xiarixiaoyao This PR has been delayed for almost a year or so, and has been put on hold by review after each change of the code. Since I use presto less, my initial thought was to wa

[GitHub] [hudi] eshu opened a new issue, #7192: [SUPPORT] UPSERT failed with errors

2022-11-13 Thread GitBox
eshu opened a new issue, #7192: URL: https://github.com/apache/hudi/issues/7192 On one of my jobs I received the error ``` 2022-11-13 22:22:04,827 ERROR [main] hudi.HoodieSparkSqlWriter$ (HoodieSparkSqlWriter.scala:commitAndPerformPostOperations(689)): UPSERT failed with errors ```

[GitHub] [hudi] hussein-awala commented on issue #7185: [SUPPORT] Error to write in s3 bucket with K8s

2022-11-13 Thread GitBox
hussein-awala commented on issue #7185: URL: https://github.com/apache/hudi/issues/7185#issuecomment-1312839240 I had a similar issue with Spark despite having Hadoop AWS sdk added (equivalent to `flink-s3-fs-hadoop-1.15.0.jar` you are using), and I solved it by downloading [hudi-aws](http

[GitHub] [hudi] hudi-bot commented on pull request #7003: [minor] add more test for rfc46

2022-11-13 Thread GitBox
hudi-bot commented on PR #7003: URL: https://github.com/apache/hudi/pull/7003#issuecomment-1312822381 ## CI report: * f91a0b4dc3417bcdbe77d823dd7fbe1f11d13e0b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1299

[GitHub] [hudi] hudi-bot commented on pull request #7003: [minor] add more test for rfc46

2022-11-13 Thread GitBox
hudi-bot commented on PR #7003: URL: https://github.com/apache/hudi/pull/7003#issuecomment-1312793776 ## CI report: * 277ad5daccc57608517966c687db1445119989ba Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=129

[GitHub] [hudi] hudi-bot commented on pull request #7155: [HUDI-4871] Upgrade to spark 3.3.1 & 3.2.2

2022-11-13 Thread GitBox
hudi-bot commented on PR #7155: URL: https://github.com/apache/hudi/pull/7155#issuecomment-1312793033 ## CI report: * e65ec914c65744fcf449bc140b4948861d5a1deb UNKNOWN * e79c9b8ce3c9294994427b96d9c95f667665c326 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #7003: [minor] add more test for rfc46

2022-11-13 Thread GitBox
hudi-bot commented on PR #7003: URL: https://github.com/apache/hudi/pull/7003#issuecomment-1312792966 ## CI report: * 5d0b1166a3f8bfa4be7de4ec221d5f4d1567ab2e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1295

[GitHub] [hudi] aniketnanna opened a new issue, #7191: [SUPPORT] Missing Data with Amazon Athena in Glue Table with Hudi 0.10.1

2022-11-13 Thread GitBox
aniketnanna opened a new issue, #7191: URL: https://github.com/apache/hudi/issues/7191 ### Highlight of Issues Facing: 1. Missing Data 2. DDL changes in Hudi Tables 3. Upgrade to Newer Version ### Detailed Description of Issues: **1. Missing Data** a. For around 20 t

[GitHub] [hudi] hudi-bot commented on pull request #7181: [HUDI-5201] add totalRecordsDeleted metric

2022-11-13 Thread GitBox
hudi-bot commented on PR #7181: URL: https://github.com/apache/hudi/pull/7181#issuecomment-1312792057 ## CI report: * 0567b98c5823b062b85442f1baa7cadba278fec4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1298

[GitHub] [hudi] hudi-bot commented on pull request #7003: [minor] add more test for rfc46

2022-11-13 Thread GitBox
hudi-bot commented on PR #7003: URL: https://github.com/apache/hudi/pull/7003#issuecomment-1312780675 ## CI report: * 5d0b1166a3f8bfa4be7de4ec221d5f4d1567ab2e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1295

[GitHub] [hudi] hudi-bot commented on pull request #7003: [minor] add more test for rfc46

2022-11-13 Thread GitBox
hudi-bot commented on PR #7003: URL: https://github.com/apache/hudi/pull/7003#issuecomment-1312779830 ## CI report: * 5d0b1166a3f8bfa4be7de4ec221d5f4d1567ab2e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1295

[GitHub] [hudi] hudi-bot commented on pull request #7189: [HUDI-5159]Add support write success file to finished partition in flink streaming append write

2022-11-13 Thread GitBox
hudi-bot commented on PR #7189: URL: https://github.com/apache/hudi/pull/7189#issuecomment-1312779039 ## CI report: * 166de6fb738f7593f9c62672260756f75445a44d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1298

[GitHub] [hudi] hudi-bot commented on pull request #7021: [Minor] fix multi deser avro payload

2022-11-13 Thread GitBox
hudi-bot commented on PR #7021: URL: https://github.com/apache/hudi/pull/7021#issuecomment-1312777869 ## CI report: * 311bacece34cf8ae17c94826c83e7700a58f4d12 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1298

[jira] [Updated] (HUDI-5203) Debezium payload does not handle null-field cases

2022-11-13 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-5203: - Description: https://github.com/apache/hudi/issues/7152 > Debezium payload does not handle null-field case

[jira] [Created] (HUDI-5203) Debezium payload does not handle null-field cases

2022-11-13 Thread Raymond Xu (Jira)
Raymond Xu created HUDI-5203: Summary: Debezium payload does not handle null-field cases Key: HUDI-5203 URL: https://issues.apache.org/jira/browse/HUDI-5203 Project: Apache Hudi Issue Type: Bug

[GitHub] [hudi] hudi-bot commented on pull request #7155: [HUDI-4871] Upgrade to spark 3.3.1 & 3.2.2

2022-11-13 Thread GitBox
hudi-bot commented on PR #7155: URL: https://github.com/apache/hudi/pull/7155#issuecomment-1312763161 ## CI report: * e65ec914c65744fcf449bc140b4948861d5a1deb UNKNOWN * 16dcf36744f82d7c23bb0e0c65d626e4c9c0b489 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #7181: [HUDI-5201] add totalRecordsDeleted metric

2022-11-13 Thread GitBox
hudi-bot commented on PR #7181: URL: https://github.com/apache/hudi/pull/7181#issuecomment-1312762263 ## CI report: * 0567b98c5823b062b85442f1baa7cadba278fec4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1298

[GitHub] [hudi] hudi-bot commented on pull request #7155: [HUDI-4871] Upgrade to spark 3.3.1 & 3.2.2

2022-11-13 Thread GitBox
hudi-bot commented on PR #7155: URL: https://github.com/apache/hudi/pull/7155#issuecomment-1312762240 ## CI report: * e65ec914c65744fcf449bc140b4948861d5a1deb UNKNOWN * 16dcf36744f82d7c23bb0e0c65d626e4c9c0b489 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #7181: [HUDI-5201] add totalRecordsDeleted metric

2022-11-13 Thread GitBox
hudi-bot commented on PR #7181: URL: https://github.com/apache/hudi/pull/7181#issuecomment-1312761276 ## CI report: * 0567b98c5823b062b85442f1baa7cadba278fec4 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] xushiyan commented on issue #7152: [SUPPORT] Failed to combine/merge new record with old value in storage MySQL Debezium

2022-11-13 Thread GitBox
xushiyan commented on issue #7152: URL: https://github.com/apache/hudi/issues/7152#issuecomment-1312759419 @hoanpvzz in your error stacktrace, it says `"_event_seq": null` that's the reason for NPE raising from `org.apache.hudi.common.model.debezium.MySqlDebeziumAvroPayload#extractSeq`

[GitHub] [hudi] Zouxxyy commented on issue #6679: [SUPPORT] Expect job status failed in spark batch model

2022-11-13 Thread GitBox
Zouxxyy commented on issue #6679: URL: https://github.com/apache/hudi/issues/6679#issuecomment-1312755250 @KnightChess Hi, the problem has been fixed, but the test is missing, can you provide a way to reproduce the problem, then I can add it to the UT -- This is an automated message from

[GitHub] [hudi] xushiyan commented on issue #7154: [SUPPORT] Hudi 0.12.2 release (Unknown versionCode:5)

2022-11-13 Thread GitBox
xushiyan commented on issue #7154: URL: https://github.com/apache/hudi/issues/7154#issuecomment-1312751387 @fengjian428 thanks for verifying. how about the downgrading 5->4? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[hudi] branch master updated (49a2110fc8 -> b10a7c32de)

2022-11-13 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 49a2110fc8 [HUDI-4496] Fix Orc support broken for Spark 3.x and more (#6227) add b10a7c32de [HUDI-5200] Clean up

[GitHub] [hudi] xushiyan merged pull request #7190: [HUDI-5200] Clean up resources in hudi common UT

2022-11-13 Thread GitBox
xushiyan merged PR #7190: URL: https://github.com/apache/hudi/pull/7190 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.

[GitHub] [hudi] hudi-bot commented on pull request #7190: [HUDI-5200] Clean up resources in hudi common UT

2022-11-13 Thread GitBox
hudi-bot commented on PR #7190: URL: https://github.com/apache/hudi/pull/7190#issuecomment-1312747901 ## CI report: * 0333ad2bc56ab62c8847447f93a91c4c5df0788d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1298

[GitHub] [hudi] xushiyan commented on issue #7135: [SUPPORT] Issues with Spark3_3Adapter while using spark to write to hudi

2022-11-13 Thread GitBox
xushiyan commented on issue #7135: URL: https://github.com/apache/hudi/issues/7135#issuecomment-1312747135 @devanshguptatrepp i checked your dependencies. You only need to put `hudi-spark3.3-bundle_2.12` on your classpath to supply hudi dependencies. Users are not supposed to use hudi modul

[GitHub] [hudi] Zouxxyy commented on a diff in pull request #7190: [HUDI-5200] Clean up resources in hudi common UT

2022-11-13 Thread GitBox
Zouxxyy commented on code in PR #7190: URL: https://github.com/apache/hudi/pull/7190#discussion_r1020913952 ## hudi-common/src/test/java/org/apache/hudi/common/testutils/HoodieCommonTestHarness.java: ## @@ -91,6 +91,12 @@ protected void initMetaClient() throws IOException {

[GitHub] [hudi] hudi-bot commented on pull request #7189: [HUDI-5159]Add support write success file to finished partition in flink streaming append write

2022-11-13 Thread GitBox
hudi-bot commented on PR #7189: URL: https://github.com/apache/hudi/pull/7189#issuecomment-1312745789 ## CI report: * 870a77c977df5539f051737dbf17046da3b03ab3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1297

[GitHub] [hudi] hudi-bot commented on pull request #7021: [Minor] fix multi deser avro payload

2022-11-13 Thread GitBox
hudi-bot commented on PR #7021: URL: https://github.com/apache/hudi/pull/7021#issuecomment-1312745704 ## CI report: * 543f95483b2b2381d3ea821d4824faf70d875d07 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1298

[GitHub] [hudi] hudi-bot commented on pull request #7189: [HUDI-5159]Add support write success file to finished partition in flink streaming append write

2022-11-13 Thread GitBox
hudi-bot commented on PR #7189: URL: https://github.com/apache/hudi/pull/7189#issuecomment-1312744684 ## CI report: * 870a77c977df5539f051737dbf17046da3b03ab3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1297

[GitHub] [hudi] hudi-bot commented on pull request #7021: [Minor] fix multi deser avro payload

2022-11-13 Thread GitBox
hudi-bot commented on PR #7021: URL: https://github.com/apache/hudi/pull/7021#issuecomment-1312744601 ## CI report: * 543f95483b2b2381d3ea821d4824faf70d875d07 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1298

[GitHub] [hudi] hudi-bot commented on pull request #7181: [HUDI-5201] add totalRecordsDeleted metric

2022-11-13 Thread GitBox
hudi-bot commented on PR #7181: URL: https://github.com/apache/hudi/pull/7181#issuecomment-1312743484 ## CI report: * 0567b98c5823b062b85442f1baa7cadba278fec4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1298

[GitHub] [hudi] xushiyan commented on issue #7158: [SUPPORT] Request for support for Azure Blob Storage Events Source and Events Hoodie Incr Source in DeltaStreamer

2022-11-13 Thread GitBox
xushiyan commented on issue #7158: URL: https://github.com/apache/hudi/issues/7158#issuecomment-1312742953 @rtdt99 i've filed this https://issues.apache.org/jira/browse/HUDI-5202 This kind of feature is mostly community-driven so there is no specific timeline. Anyone interested can pi

[jira] [Created] (HUDI-5202) Implement DeltaStreamer source for Azure Blob storage

2022-11-13 Thread Raymond Xu (Jira)
Raymond Xu created HUDI-5202: Summary: Implement DeltaStreamer source for Azure Blob storage Key: HUDI-5202 URL: https://issues.apache.org/jira/browse/HUDI-5202 Project: Apache Hudi Issue Type: N

[GitHub] [hudi] xushiyan commented on a diff in pull request #7190: [HUDI-5200] Clean up resources in hudi common UT

2022-11-13 Thread GitBox
xushiyan commented on code in PR #7190: URL: https://github.com/apache/hudi/pull/7190#discussion_r1020908718 ## hudi-common/src/test/java/org/apache/hudi/common/testutils/HoodieCommonTestHarness.java: ## @@ -91,6 +91,12 @@ protected void initMetaClient() throws IOException {

[jira] [Updated] (HUDI-5200) Resources are not cleaned up in UT

2022-11-13 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-5200: - Fix Version/s: 0.12.2 > Resources are not cleaned up in UT > -- > >

[jira] [Assigned] (HUDI-5200) Resources are not cleaned up in UT

2022-11-13 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-5200: Assignee: zouxxyy > Resources are not cleaned up in UT > -- > >

[jira] [Updated] (HUDI-5200) Resources are not cleaned up in UT

2022-11-13 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-5200: - Component/s: tests-ci > Resources are not cleaned up in UT > -- > >

[GitHub] [hudi] xushiyan commented on issue #6984: [SUPPORT] hudi metrics with flink so little

2022-11-13 Thread GitBox
xushiyan commented on issue #6984: URL: https://github.com/apache/hudi/issues/6984#issuecomment-1312734398 i found HUDI-2141 for flink and there is already a WIP PR https://github.com/apache/hudi/pull/3235 by @yuzhaojing we can align the efforts there with what we're proposing here.

[jira] [Closed] (HUDI-5201) add totalRecordsDeleted metric

2022-11-13 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-5201. Resolution: Done > add totalRecordsDeleted metric > -- > > Key:

[jira] [Updated] (HUDI-2141) Integration flink metric in flink stream

2022-11-13 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2141: - Component/s: metrics > Integration flink metric in flink stream >

[jira] [Deleted] (HUDI-4596) Add totalRecordsDeleted in hudi metrics

2022-11-13 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu deleted HUDI-4596: - > Add totalRecordsDeleted in hudi metrics > --- > > Key:

[jira] [Updated] (HUDI-5201) add totalRecordsDeleted metric

2022-11-13 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-5201: - Epic Link: HUDI-3526 > add totalRecordsDeleted metric > -- > >

[jira] [Updated] (HUDI-5201) add totalRecordsDeleted metric

2022-11-13 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-5201: - Issue Type: New Feature (was: Task) > add totalRecordsDeleted metric > -- > >

  1   2   >