[jira] [Commented] (SPARK-38829) New configuration for controlling timestamp inference of Parquet
[ https://issues.apache.org/jira/browse/SPARK-38829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683156#comment-17683156 ] Apache Spark commented on SPARK-38829: -- User 'gengliangwang' has created a pull request for this issue: https://github.com/apache/spark/pull/39856 > New configuration for controlling timestamp inference of Parquet > > > Key: SPARK-38829 > URL: https://issues.apache.org/jira/browse/SPARK-38829 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.3.0 >Reporter: Gengliang Wang >Assignee: Ivan Sadikov >Priority: Major > Fix For: 3.3.0 > > > A new SQL conf which can fallback to the behavior that reads all the Parquet > Timestamp column as TimestampType. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-38829) New configuration for controlling timestamp inference of Parquet
[ https://issues.apache.org/jira/browse/SPARK-38829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17537931#comment-17537931 ] Apache Spark commented on SPARK-38829: -- User 'LuciferYang' has created a pull request for this issue: https://github.com/apache/spark/pull/36573 > New configuration for controlling timestamp inference of Parquet > > > Key: SPARK-38829 > URL: https://issues.apache.org/jira/browse/SPARK-38829 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.3.0 >Reporter: Gengliang Wang >Assignee: Ivan Sadikov >Priority: Major > Fix For: 3.3.0 > > > A new SQL conf which can fallback to the behavior that reads all the Parquet > Timestamp column as TimestampType. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-38829) New configuration for controlling timestamp inference of Parquet
[ https://issues.apache.org/jira/browse/SPARK-38829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17520910#comment-17520910 ] Apache Spark commented on SPARK-38829: -- User 'sadikovi' has created a pull request for this issue: https://github.com/apache/spark/pull/36158 > New configuration for controlling timestamp inference of Parquet > > > Key: SPARK-38829 > URL: https://issues.apache.org/jira/browse/SPARK-38829 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.3.0 >Reporter: Gengliang Wang >Assignee: Ivan Sadikov >Priority: Major > > A new SQL conf which can fallback to the behavior that reads all the Parquet > Timestamp column as TimestampType. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-38829) New configuration for controlling timestamp inference of Parquet
[ https://issues.apache.org/jira/browse/SPARK-38829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17520909#comment-17520909 ] Apache Spark commented on SPARK-38829: -- User 'sadikovi' has created a pull request for this issue: https://github.com/apache/spark/pull/36158 > New configuration for controlling timestamp inference of Parquet > > > Key: SPARK-38829 > URL: https://issues.apache.org/jira/browse/SPARK-38829 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.3.0 >Reporter: Gengliang Wang >Assignee: Ivan Sadikov >Priority: Major > > A new SQL conf which can fallback to the behavior that reads all the Parquet > Timestamp column as TimestampType. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-38829) New configuration for controlling timestamp inference of Parquet
[ https://issues.apache.org/jira/browse/SPARK-38829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17520900#comment-17520900 ] Gengliang Wang commented on SPARK-38829: [~ivan.sadikov] +1, disabling it in 3.3 is simpler > New configuration for controlling timestamp inference of Parquet > > > Key: SPARK-38829 > URL: https://issues.apache.org/jira/browse/SPARK-38829 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.3.0 >Reporter: Gengliang Wang >Assignee: Ivan Sadikov >Priority: Major > > A new SQL conf which can fallback to the behavior that reads all the Parquet > Timestamp column as TimestampType. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-38829) New configuration for controlling timestamp inference of Parquet
[ https://issues.apache.org/jira/browse/SPARK-38829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17520870#comment-17520870 ] Ivan Sadikov commented on SPARK-38829: -- [~Gengliang.Wang] Do you still want to merge the PR for 3.3 to disable TimestampNTZ there or would you prefer to have a feature flag and backport it to 3.3? I was thinking about disabling in 3.3 to unblock the existing work so there will be less time pressure with the config in master. > New configuration for controlling timestamp inference of Parquet > > > Key: SPARK-38829 > URL: https://issues.apache.org/jira/browse/SPARK-38829 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.3.0 >Reporter: Gengliang Wang >Assignee: Ivan Sadikov >Priority: Major > > A new SQL conf which can fallback to the behavior that reads all the Parquet > Timestamp column as TimestampType. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-38829) New configuration for controlling timestamp inference of Parquet
[ https://issues.apache.org/jira/browse/SPARK-38829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17520405#comment-17520405 ] Gengliang Wang commented on SPARK-38829: [~ivan.sadikov] I think we still need a new configuration on master branch > New configuration for controlling timestamp inference of Parquet > > > Key: SPARK-38829 > URL: https://issues.apache.org/jira/browse/SPARK-38829 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.3.0 >Reporter: Gengliang Wang >Assignee: Ivan Sadikov >Priority: Major > > A new SQL conf which can fallback to the behavior that reads all the Parquet > Timestamp column as TimestampType. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-38829) New configuration for controlling timestamp inference of Parquet
[ https://issues.apache.org/jira/browse/SPARK-38829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17520297#comment-17520297 ] Apache Spark commented on SPARK-38829: -- User 'sadikovi' has created a pull request for this issue: https://github.com/apache/spark/pull/36137 > New configuration for controlling timestamp inference of Parquet > > > Key: SPARK-38829 > URL: https://issues.apache.org/jira/browse/SPARK-38829 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.3.0 >Reporter: Gengliang Wang >Assignee: Ivan Sadikov >Priority: Major > > A new SQL conf which can fallback to the behavior that reads all the Parquet > Timestamp column as TimestampType. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-38829) New configuration for controlling timestamp inference of Parquet
[ https://issues.apache.org/jira/browse/SPARK-38829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17520296#comment-17520296 ] Ivan Sadikov commented on SPARK-38829: -- I opened [https://github.com/apache/spark/pull/36137] to disable TimestampNTZType support in Parquet in 3.3. > New configuration for controlling timestamp inference of Parquet > > > Key: SPARK-38829 > URL: https://issues.apache.org/jira/browse/SPARK-38829 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.3.0 >Reporter: Gengliang Wang >Assignee: Ivan Sadikov >Priority: Major > > A new SQL conf which can fallback to the behavior that reads all the Parquet > Timestamp column as TimestampType. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-38829) New configuration for controlling timestamp inference of Parquet
[ https://issues.apache.org/jira/browse/SPARK-38829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17520295#comment-17520295 ] Apache Spark commented on SPARK-38829: -- User 'sadikovi' has created a pull request for this issue: https://github.com/apache/spark/pull/36137 > New configuration for controlling timestamp inference of Parquet > > > Key: SPARK-38829 > URL: https://issues.apache.org/jira/browse/SPARK-38829 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.3.0 >Reporter: Gengliang Wang >Assignee: Ivan Sadikov >Priority: Major > > A new SQL conf which can fallback to the behavior that reads all the Parquet > Timestamp column as TimestampType. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
