[jira] [Updated] (SPARK-36983) ignoreCorruptFiles does not work when schema change from int to string when a file having more than 35 records

2021-10-12 Thread mike (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mike updated SPARK-36983: - Description: Precondition: Spark 3.1 run locally on my Macbook Pro(16G Ram,i7, 2015) In folder A having two pa

[jira] [Updated] (SPARK-36983) ignoreCorruptFiles does not work when schema change from int to string when a file having more than 35 records

2021-10-12 Thread mike (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mike updated SPARK-36983: - Description: Precondition: In folder A having two parquet files * File 1: have some columns and one of them is

[jira] [Updated] (SPARK-36983) ignoreCorruptFiles does not work when schema change from int to string when a file having more than 35 records

2021-10-12 Thread mike (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mike updated SPARK-36983: - Summary: ignoreCorruptFiles does not work when schema change from int to string when a file having more than 35

[jira] [Updated] (SPARK-36983) ignoreCorruptFiles does not work when schema change from int to string when a file having more than 35 records

2021-10-12 Thread mike (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mike updated SPARK-36983: - Description: Precondition: In folder A having two parquet files * File 1: have some columns and one of them is