[GitHub] [spark] huaxingao commented on pull request #36781: [SPARK-39393][SQL] Parquet data source only supports push-down predicate filters for non-repeated primitive types

huaxingao commented on PR #36781: URL: https://github.com/apache/spark/pull/36781#issuecomment-1150422569 @Borjianamin98 I forgot that I need to add you to the contributors list first. I just did and assigned the jira OK :) -- This is an automated message from the Apache Git Service. To

[GitHub] [spark] huaxingao commented on pull request #36781: [SPARK-39393][SQL] Parquet data source only supports push-down predicate filters for non-repeated primitive types

huaxingao commented on PR #36781: URL: https://github.com/apache/spark/pull/36781#issuecomment-1150390322 @Borjianamin98 Do you have a jira account? I tried to assign the jira to you but can't find you. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] huaxingao commented on pull request #36781: [SPARK-39393][SQL] Parquet data source only supports push-down predicate filters for non-repeated primitive types

huaxingao commented on PR #36781: URL: https://github.com/apache/spark/pull/36781#issuecomment-1150387353 Merged to master/3.3/3.2/3.1. Thanks @Borjianamin98 for your first contribution and welcome to Spark community! Also thanks @LuciferYang @dcoliversun for reviewing! -- This is

[GitHub] [spark] huaxingao commented on pull request #36781: [SPARK-39393][SQL] Parquet data source only supports push-down predicate filters for non-repeated primitive types

huaxingao commented on PR #36781: URL: https://github.com/apache/spark/pull/36781#issuecomment-1149525380 I think over. I think it's better to have a separate PR to fix the explain problem. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] huaxingao commented on pull request #36781: [SPARK-39393][SQL] Parquet data source only supports push-down predicate filters for non-repeated primitive types

huaxingao commented on PR #36781: URL: https://github.com/apache/spark/pull/36781#issuecomment-1149518253 The fix looks good but the explain result bothers me. Here is what I got from the explain result: ``` spark.read.parquet(dir.getCanonicalPath).filter("isnotnull(f)").explain(true)

[GitHub] [spark] huaxingao commented on pull request #36781: [SPARK-39393][SQL] Parquet data source only supports push-down predicate filters for non-repeated primitive types

huaxingao commented on PR #36781: URL: https://github.com/apache/spark/pull/36781#issuecomment-1148022608 @Borjianamin98 Could you please add a test? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go