Re: [PR] fix: [native_iceberg_compat / native_datafusion] Fall back to Spark for maps containing structs [datafusion-comet]

2025-05-23 Thread via GitHub 


andygrove closed pull request #1764: fix: [native_iceberg_compat / 
native_datafusion] Fall back to Spark for maps containing structs
URL: https://github.com/apache/datafusion-comet/pull/1764


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] fix: [native_iceberg_compat / native_datafusion] Fall back to Spark for maps containing structs [datafusion-comet]

2025-05-21 Thread via GitHub 


andygrove commented on PR #1764:
URL: 
https://github.com/apache/datafusion-comet/pull/1764#issuecomment-2899455164

   Moving this to draft because we may want to merge 
https://github.com/apache/datafusion-comet/pull/1771 instead


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] fix: [native_iceberg_compat / native_datafusion] Fall back to Spark for maps containing structs [datafusion-comet]

2025-05-21 Thread via GitHub 


kazuyukitanimura commented on code in PR #1764:
URL: https://github.com/apache/datafusion-comet/pull/1764#discussion_r2101105764


##
spark/src/test/scala/org/apache/comet/exec/CometNativeReaderSuite.scala:
##
@@ -153,7 +153,8 @@ class CometNativeReaderSuite extends CometTestBase with 
AdaptiveSparkPlanHelper
   "select c0 from tbl")
   }
 
-  test("native reader - read MAP of value STRUCT fields") {
+  // https://github.com/apache/datafusion-comet/issues/1754
+  ignore("native reader - read MAP of value STRUCT fields") {

Review Comment:
   Now we are doing the data type support check for maps of struct. Dow we 
still have to ignore these tests?
   If so, is there a way to ignore only for native datafusion/iceberg-compat?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] fix: [native_iceberg_compat / native_datafusion] Fall back to Spark for maps containing structs [datafusion-comet]

2025-05-21 Thread via GitHub 


andygrove commented on PR #1764:
URL: 
https://github.com/apache/datafusion-comet/pull/1764#issuecomment-2899221853

   One test keeps failing without any actual test failures. This is happening 
on other PRs as well.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] fix: [native_iceberg_compat / native_datafusion] Fall back to Spark for maps containing structs [datafusion-comet]

2025-05-21 Thread via GitHub 


andygrove commented on code in PR #1764:
URL: https://github.com/apache/datafusion-comet/pull/1764#discussion_r2101157036


##
spark/src/test/scala/org/apache/comet/exec/CometNativeReaderSuite.scala:
##
@@ -153,7 +153,8 @@ class CometNativeReaderSuite extends CometTestBase with 
AdaptiveSparkPlanHelper
   "select c0 from tbl")
   }
 
-  test("native reader - read MAP of value STRUCT fields") {
+  // https://github.com/apache/datafusion-comet/issues/1754
+  ignore("native reader - read MAP of value STRUCT fields") {

Review Comment:
   This test suite is specific to native datafusion/iceberg-compat scans. The 
fallback is just a temporary workaround until we have the real fix for #1754, 
then we can re-enable these tests.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] fix: [native_iceberg_compat / native_datafusion] Fall back to Spark for maps containing structs [datafusion-comet]

2025-05-21 Thread via GitHub 


codecov-commenter commented on PR #1764:
URL: 
https://github.com/apache/datafusion-comet/pull/1764#issuecomment-2898695884

   ## 
[Codecov](https://app.codecov.io/gh/apache/datafusion-comet/pull/1764?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
 Report
   Attention: Patch coverage is `50.0%` with `4 lines` in your changes 
missing coverage. Please review.
   > Project coverage is 57.23%. Comparing base 
[(`f09f8af`)](https://app.codecov.io/gh/apache/datafusion-comet/commit/f09f8af64c6599255e116a376f4f008f2fd63b43?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
 to head 
[(`843f9e8`)](https://app.codecov.io/gh/apache/datafusion-comet/commit/843f9e8b212c9826c6a114e436ee349b0ee7e3d9?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   > Report is 204 commits behind head on main.
   
   | [Files with missing 
lines](https://app.codecov.io/gh/apache/datafusion-comet/pull/1764?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
 | Patch % | Lines |
   |---|---|---|
   | 
[.../main/scala/org/apache/comet/DataTypeSupport.scala](https://app.codecov.io/gh/apache/datafusion-comet/pull/1764?src=pr&el=tree&filepath=spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fcomet%2FDataTypeSupport.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9jb21ldC9EYXRhVHlwZVN1cHBvcnQuc2NhbGE=)
 | 50.00% | [1 Missing and 3 partials :warning: 
](https://app.codecov.io/gh/apache/datafusion-comet/pull/1764?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
 |
   
   Additional details and impacted files
   
   
   ```diff
   @@ Coverage Diff  @@
   ##   main#1764  +/-   ##
   
   + Coverage 56.12%   57.23%   +1.11% 
   - Complexity  976 1072  +96 
   
 Files   119  130  +11 
 Lines 1174312586 +843 
 Branches   2251 2359 +108 
   
   + Hits   6591 7204 +613 
   - Misses 4012 4179 +167 
   - Partials   1140 1203  +63 
   ```
   
   
   
   [:umbrella: View full report in Codecov by 
Sentry](https://app.codecov.io/gh/apache/datafusion-comet/pull/1764?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   
   :loudspeaker: Have feedback on the report? [Share it 
here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   
:rocket: New features to boost your workflow: 
   
   - :snowflake: [Test 
Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, 
report on failures, and find test suite problems.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]