Re: [PR] Incoming batch schema is not compatible with the table's one #9980 [hudi]

2023-12-17 Thread via GitHub 


danny0405 commented on code in PR #10308:
URL: https://github.com/apache/hudi/pull/10308#discussion_r1429432577


##
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala:
##
@@ -1092,6 +1092,10 @@ class HoodieSparkSqlWriterInternal {
   && mergedParams.getOrElse(DataSourceWriteOptions.TABLE_TYPE.key, 
COPY_ON_WRITE.name) == MERGE_ON_READ.name) {
   mergedParams.put(HoodieTableConfig.DROP_PARTITION_COLUMNS.key, "false")
 }
+// use meta sync database to fill hoodie.table.name if it not sets
+if (!mergedParams.contains(HoodieTableConfig.DATABASE_NAME.key()) && 
mergedParams.contains(HoodieSyncConfig.META_SYNC_DATABASE_NAME.key())) {

Review Comment:
   It looks like you are fixing the meta sync config, maybe just a UT for the 
spark write config.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] Incoming batch schema is not compatible with the table's one #9980 [hudi]

2023-12-14 Thread via GitHub 


njalan commented on PR #10308:
URL: https://github.com/apache/hudi/pull/10308#issuecomment-1856063736

   > Can we write a UT for it.
   
   I  should add UT in which module?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] Incoming batch schema is not compatible with the table's one #9980 [hudi]

2023-12-12 Thread via GitHub 


danny0405 commented on PR #10308:
URL: https://github.com/apache/hudi/pull/10308#issuecomment-1853188317

   Can we write a UT for it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] Incoming batch schema is not compatible with the table's one #9980 [hudi]

2023-12-12 Thread via GitHub 


hudi-bot commented on PR #10308:
URL: https://github.com/apache/hudi/pull/10308#issuecomment-1852496996

   
   ## CI report:
   
   * 14d5465e2e85b66ff4404a5c9b46f19e9c9a0e73 Azure: 
[FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21472)
 
   
   
   Bot commands
 @hudi-bot supports the following commands:
   
- `@hudi-bot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] Incoming batch schema is not compatible with the table's one #9980 [hudi]

2023-12-12 Thread via GitHub 


hudi-bot commented on PR #10308:
URL: https://github.com/apache/hudi/pull/10308#issuecomment-1852079550

   
   ## CI report:
   
   * 737e09fc37912e88f640393b11357cb8b27a29c5 Azure: 
[FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21464)
 
   * 14d5465e2e85b66ff4404a5c9b46f19e9c9a0e73 Azure: 
[PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21472)
 
   
   
   Bot commands
 @hudi-bot supports the following commands:
   
- `@hudi-bot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] Incoming batch schema is not compatible with the table's one #9980 [hudi]

2023-12-12 Thread via GitHub 


hudi-bot commented on PR #10308:
URL: https://github.com/apache/hudi/pull/10308#issuecomment-1852064937

   
   ## CI report:
   
   * 737e09fc37912e88f640393b11357cb8b27a29c5 Azure: 
[FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21464)
 
   * 14d5465e2e85b66ff4404a5c9b46f19e9c9a0e73 UNKNOWN
   
   
   Bot commands
 @hudi-bot supports the following commands:
   
- `@hudi-bot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] Incoming batch schema is not compatible with the table's one #9980 [hudi]

2023-12-12 Thread via GitHub 


njalan commented on code in PR #10308:
URL: https://github.com/apache/hudi/pull/10308#discussion_r1423964727


##
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala:
##
@@ -1092,6 +1092,10 @@ class HoodieSparkSqlWriterInternal {
   && mergedParams.getOrElse(DataSourceWriteOptions.TABLE_TYPE.key, 
COPY_ON_WRITE.name) == MERGE_ON_READ.name) {
   mergedParams.put(HoodieTableConfig.DROP_PARTITION_COLUMNS.key, "false")
 }
+// use meta sync database to fill hoodie.table.name if it not sets
+if (!mergedParams.contains(HoodieTableConfig.DATABASE_NAME.key()) && 
mergedParams.contains(HoodieSyncConfig.META_SYNC_DATABASE_NAME.key())) {

Review Comment:
   @danny0405 Yes, I updated the comments just now.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] Incoming batch schema is not compatible with the table's one #9980 [hudi]

2023-12-12 Thread via GitHub 


danny0405 commented on code in PR #10308:
URL: https://github.com/apache/hudi/pull/10308#discussion_r1423799754


##
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala:
##
@@ -1092,6 +1092,10 @@ class HoodieSparkSqlWriterInternal {
   && mergedParams.getOrElse(DataSourceWriteOptions.TABLE_TYPE.key, 
COPY_ON_WRITE.name) == MERGE_ON_READ.name) {
   mergedParams.put(HoodieTableConfig.DROP_PARTITION_COLUMNS.key, "false")
 }
+// use meta sync database to fill hoodie.table.name if it not sets
+if (!mergedParams.contains(HoodieTableConfig.DATABASE_NAME.key()) && 
mergedParams.contains(HoodieSyncConfig.META_SYNC_DATABASE_NAME.key())) {

Review Comment:
   Are you saying `hoodie.database.name` ?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] Incoming batch schema is not compatible with the table's one #9980 [hudi]

2023-12-12 Thread via GitHub 


hudi-bot commented on PR #10308:
URL: https://github.com/apache/hudi/pull/10308#issuecomment-1851635617

   
   ## CI report:
   
   * 737e09fc37912e88f640393b11357cb8b27a29c5 Azure: 
[FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21464)
 
   
   
   Bot commands
 @hudi-bot supports the following commands:
   
- `@hudi-bot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] Incoming batch schema is not compatible with the table's one #9980 [hudi]

2023-12-11 Thread via GitHub 


hudi-bot commented on PR #10308:
URL: https://github.com/apache/hudi/pull/10308#issuecomment-1851342245

   
   ## CI report:
   
   * 737e09fc37912e88f640393b11357cb8b27a29c5 Azure: 
[PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21464)
 
   
   
   Bot commands
 @hudi-bot supports the following commands:
   
- `@hudi-bot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] Incoming batch schema is not compatible with the table's one #9980 [hudi]

2023-12-11 Thread via GitHub 


hudi-bot commented on PR #10308:
URL: https://github.com/apache/hudi/pull/10308#issuecomment-1851336257

   
   ## CI report:
   
   * 737e09fc37912e88f640393b11357cb8b27a29c5 UNKNOWN
   
   
   Bot commands
 @hudi-bot supports the following commands:
   
- `@hudi-bot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org