[jira] [Commented] (SPARK-42258) pyspark.sql.functions should not expose typing.cast

2023-02-13 Thread Furcy Pin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688120#comment-17688120 ] Furcy Pin commented on SPARK-42258: --- Sure, I can do that. Do you have any preference between option 1

[jira] [Created] (SPARK-42258) pyspark.sql.functions should not expose typing.cast

2023-01-31 Thread Furcy Pin (Jira)
Furcy Pin created SPARK-42258: - Summary: pyspark.sql.functions should not expose typing.cast Key: SPARK-42258 URL: https://issues.apache.org/jira/browse/SPARK-42258 Project: Spark Issue Type:

[jira] [Updated] (SPARK-39059) When using multiple SparkSessions, DataFrame.resolve uses configuration from the wrong session

2022-09-15 Thread Furcy Pin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Furcy Pin updated SPARK-39059: -- Description: We encountered unexpected error when using SparkSession.newSession and the

[jira] [Updated] (SPARK-39059) When using multiple SparkSessions, DataFrame.resolve uses configuration from the wrong session

2022-09-15 Thread Furcy Pin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Furcy Pin updated SPARK-39059: -- Description: We encountered unexpected error when using SparkSession.newSession and the

[jira] [Updated] (SPARK-39059) When using multiple SparkSessions, DataFrame.resolve uses configuration from the wrong session

2022-04-28 Thread Furcy Pin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Furcy Pin updated SPARK-39059: -- Description: We encountered unexpected error when using SparkSession.newSession and the

[jira] [Created] (SPARK-39059) When using multiple SparkSessions, DataFrame.resolve uses configuration from the wrong session

2022-04-28 Thread Furcy Pin (Jira)
Furcy Pin created SPARK-39059: - Summary: When using multiple SparkSessions, DataFrame.resolve uses configuration from the wrong session Key: SPARK-39059 URL: https://issues.apache.org/jira/browse/SPARK-39059

[jira] [Updated] (SPARK-38912) Clean "classproperty" workaround in pyspark.sql.session once support for Python 3.8 is dropped

2022-04-20 Thread Furcy Pin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Furcy Pin updated SPARK-38912: -- Description: *Context* The proper bugfix of SPARK-38870 would be to use the @classmethod + @property

[jira] [Created] (SPARK-38964) Upgrade Python 3.9 version in CI

2022-04-20 Thread Furcy Pin (Jira)
Furcy Pin created SPARK-38964: - Summary: Upgrade Python 3.9 version in CI Key: SPARK-38964 URL: https://issues.apache.org/jira/browse/SPARK-38964 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-38912) Clean "classproperty" workaround in pyspark.sql.session once support for Python 3.8 is dropped

2022-04-20 Thread Furcy Pin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Furcy Pin updated SPARK-38912: -- Description: *Context* The proper bugfix of SPARK-38870 would be to use the @classmethod + @property

[jira] [Created] (SPARK-38912) Clean "classproperty" workaround in pyspark.sql.session once support for Python 3.8 is dropped

2022-04-15 Thread Furcy Pin (Jira)
Furcy Pin created SPARK-38912: - Summary: Clean "classproperty" workaround in pyspark.sql.session once support for Python 3.8 is dropped Key: SPARK-38912 URL: https://issues.apache.org/jira/browse/SPARK-38912

[jira] [Updated] (SPARK-38870) SparkSession.builder returns a new builder in Scala, but not in Python

2022-04-12 Thread Furcy Pin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Furcy Pin updated SPARK-38870: -- Description: In pyspark, _SparkSession.builder_ always returns the same static builder, while the

[jira] [Updated] (SPARK-38870) SparkSession.builder returns a new builder in Scala, but not in Python

2022-04-12 Thread Furcy Pin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Furcy Pin updated SPARK-38870: -- Description: In pyspark, _SparkSession.builder_ always returns the same static builder, while the

[jira] [Created] (SPARK-38870) SparkSession.builder returns a new builder in Scala, but not in Python

2022-04-12 Thread Furcy Pin (Jira)
Furcy Pin created SPARK-38870: - Summary: SparkSession.builder returns a new builder in Scala, but not in Python Key: SPARK-38870 URL: https://issues.apache.org/jira/browse/SPARK-38870 Project: Spark

[jira] [Created] (SPARK-31657) CSV Writer writes no header for empty DataFrames

2020-05-07 Thread Furcy Pin (Jira)
Furcy Pin created SPARK-31657: - Summary: CSV Writer writes no header for empty DataFrames Key: SPARK-31657 URL: https://issues.apache.org/jira/browse/SPARK-31657 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-13587) Support virtualenv in PySpark

2019-10-04 Thread Furcy Pin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16944564#comment-16944564 ] Furcy Pin commented on SPARK-13587: --- Hello, I don't know where to ask this, but we have been using

[jira] [Commented] (SPARK-10795) FileNotFoundException while deploying pyspark job on cluster

2018-08-24 Thread Furcy Pin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591868#comment-16591868 ] Furcy Pin commented on SPARK-10795: --- Hi, I came across this ticket with the same issue: my yarn job

[jira] [Commented] (SPARK-21791) ORC should support column names with dot

2018-07-16 Thread Furcy Pin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544899#comment-16544899 ] Furcy Pin commented on SPARK-21791: --- Indeed, it works like this. Awesome! thanks!      > ORC

[jira] [Commented] (SPARK-21791) ORC should support column names with dot

2018-07-13 Thread Furcy Pin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16543291#comment-16543291 ] Furcy Pin commented on SPARK-21791: --- Hi, this bug is in fact only half-solved. Here is a minimal

[jira] [Comment Edited] (SPARK-20384) supporting value classes over primitives in DataSets

2018-03-29 Thread Furcy Pin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418691#comment-16418691 ] Furcy Pin edited comment on SPARK-20384 at 3/29/18 10:10 AM: - +1 on this

[jira] [Commented] (SPARK-20384) supporting value classes over primitives in DataSets

2018-03-29 Thread Furcy Pin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418691#comment-16418691 ] Furcy Pin commented on SPARK-20384: --- +1 on this issue. I think the generic use case is that the

[jira] [Commented] (SPARK-17430) Spark task Hangs after OOM while DAG scheduler tries to serialize a task

2018-02-19 Thread Furcy Pin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369367#comment-16369367 ] Furcy Pin commented on SPARK-17430: --- Same problem here in spark 2.2.1 The job hangs forever, the GUI

[jira] [Commented] (SPARK-13802) Fields order in Row(**kwargs) is not consistent with Schema.toInternal method

2017-04-25 Thread Furcy Pin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15983094#comment-15983094 ] Furcy Pin commented on SPARK-13802: --- Hi, I ran into similar issues and found this Jira, so I would like

[jira] [Created] (SPARK-19135) SQL: Type inconsistencies with Structs, Arrays and Nulls

2017-01-09 Thread Furcy Pin (JIRA)
Furcy Pin created SPARK-19135: - Summary: SQL: Type inconsistencies with Structs, Arrays and Nulls Key: SPARK-19135 URL: https://issues.apache.org/jira/browse/SPARK-19135 Project: Spark Issue

[jira] [Commented] (SPARK-18609) [SQL] column mixup with CROSS JOIN

2016-12-02 Thread Furcy Pin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15714899#comment-15714899 ] Furcy Pin commented on SPARK-18609: --- Hi, I found another minimal example that yields a similar, yet

[jira] [Created] (SPARK-18609) [SQL] column mixup with CROSS JOIN

2016-11-28 Thread Furcy Pin (JIRA)
Furcy Pin created SPARK-18609: - Summary: [SQL] column mixup with CROSS JOIN Key: SPARK-18609 URL: https://issues.apache.org/jira/browse/SPARK-18609 Project: Spark Issue Type: Bug Affects

[jira] [Commented] (SPARK-17212) TypeCoercion support widening conversion between DateType and TimestampType

2016-11-27 Thread Furcy Pin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15699575#comment-15699575 ] Furcy Pin commented on SPARK-17212: --- Hi, I have the same issue between DateType and StringType on

[jira] [Commented] (SPARK-17592) SQL: CAST string as INT inconsistent with Hive

2016-09-22 Thread Furcy Pin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15512413#comment-15512413 ] Furcy Pin commented on SPARK-17592: --- SQL server: query fails (like postgres)

[jira] [Created] (SPARK-17592) SQL: CAST string as INT inconsistent with Hive

2016-09-19 Thread Furcy Pin (JIRA)
Furcy Pin created SPARK-17592: - Summary: SQL: CAST string as INT inconsistent with Hive Key: SPARK-17592 URL: https://issues.apache.org/jira/browse/SPARK-17592 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-17296) Spark SQL: cross join + two joins = BUG

2016-08-29 Thread Furcy Pin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446382#comment-15446382 ] Furcy Pin commented on SPARK-17296: --- Yes, this is not critical though, a workaround is to invert the

[jira] [Created] (SPARK-17296) Spark SQL: cross join + two joins = BUG

2016-08-29 Thread Furcy Pin (JIRA)
Furcy Pin created SPARK-17296: - Summary: Spark SQL: cross join + two joins = BUG Key: SPARK-17296 URL: https://issues.apache.org/jira/browse/SPARK-17296 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-16771) Infinite recursion loop in org.apache.spark.sql.catalyst.trees.TreeNode when table name collides.

2016-07-29 Thread Furcy Pin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15399261#comment-15399261 ] Furcy Pin commented on SPARK-16771: --- wow, that was quick, thanks a lot [~dongjoon] ! > Infinite

[jira] [Created] (SPARK-16795) Spark's HiveThriftServer should be able to use multiple sqlContexts

2016-07-29 Thread Furcy Pin (JIRA)
Furcy Pin created SPARK-16795: - Summary: Spark's HiveThriftServer should be able to use multiple sqlContexts Key: SPARK-16795 URL: https://issues.apache.org/jira/browse/SPARK-16795 Project: Spark

[jira] [Commented] (SPARK-11248) Spark hivethriftserver is using the wrong user to while getting HDFS permissions

2016-07-28 Thread Furcy Pin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15397882#comment-15397882 ] Furcy Pin commented on SPARK-11248: --- +1 I'm trying on spark 2.0.0, and I've configured

[jira] [Updated] (SPARK-16771) Infinite recursion loop in org.apache.spark.sql.catalyst.trees.TreeNode when table name collides.

2016-07-28 Thread Furcy Pin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Furcy Pin updated SPARK-16771: -- Description: How to reproduce: In spark-sql on Hive {code} DROP TABLE IF EXISTS t1 ; CREATE TABLE

[jira] [Created] (SPARK-16771) Infinite recursion loop in org.apache.spark.sql.catalyst.trees.TreeNode when table name collides.

2016-07-28 Thread Furcy Pin (JIRA)
Furcy Pin created SPARK-16771: - Summary: Infinite recursion loop in org.apache.spark.sql.catalyst.trees.TreeNode when table name collides. Key: SPARK-16771 URL: https://issues.apache.org/jira/browse/SPARK-16771

[jira] [Commented] (SPARK-9600) DataFrameWriter.saveAsTable always writes data to "/user/hive/warehouse"

2015-12-24 Thread Furcy Pin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15070862#comment-15070862 ] Furcy Pin commented on SPARK-9600: -- You are right that I forgot to wipe the metastore and warehouse

[jira] [Commented] (SPARK-9600) DataFrameWriter.saveAsTable always writes data to "/user/hive/warehouse"

2015-12-22 Thread Furcy Pin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15067972#comment-15067972 ] Furcy Pin commented on SPARK-9600: -- Hi, I would like to reopen this issue as I ran into it as well. Here