[
https://issues.apache.org/jira/browse/SPARK-3928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14536114#comment-14536114
]
Yana Kadiyska commented on SPARK-3928:
--------------------------------------
[~tkyaw] Your suggested workaround does work. One question though -- what are
the implications of turning off "spark.sql.parquet.useDataSourceApi"? My
particular concern is with predicate pushdowns into parquet -- am I going to
lose these (it's hard to tell from the UI if pushdown is happening correctly).
Also, can you clarify if you still plan to fix this for 1.4 or "New parquet
implementation does not contain wild card support yet" means that we'd have to
live with spark.sql.parquet.useDataSourceApi until further time?
> Support wildcard matches on Parquet files
> -----------------------------------------
>
> Key: SPARK-3928
> URL: https://issues.apache.org/jira/browse/SPARK-3928
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core, SQL
> Reporter: Nicholas Chammas
> Assignee: Cheng Lian
> Priority: Minor
> Fix For: 1.3.0
>
>
> {{SparkContext.textFile()}} supports patterns like {{part-*}} and
> {{2014-\?\?-\?\?}}.
> It would be nice if {{SparkContext.parquetFile()}} did the same.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
