[jira] [Commented] (SPARK-46776) Pandas UDF Serialization Error: Expected Array, got

[ https://issues.apache.org/jira/browse/SPARK-46776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17896255#comment-17896255 ] Peter Andrew commented on SPARK-46776: -- This issue also affects `applyInPandas`. Un

[jira] [Commented] (SPARK-50176) Connect operation stuck in `ReadyForExecution` even though query has finished

[ https://issues.apache.org/jira/browse/SPARK-50176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17894685#comment-17894685 ] Peter Andrew commented on SPARK-50176: -- I see, that makes more sense, thank you [~c

[jira] [Commented] (SPARK-50176) Connect operation stuck in `ReadyForExecution` even though query has finished

[ https://issues.apache.org/jira/browse/SPARK-50176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17894519#comment-17894519 ] Peter Andrew commented on SPARK-50176: -- Indeed, that's quite odd. Here are the log

[jira] [Updated] (SPARK-50176) Connect operation stuck in `ReadyForExecution` even though query has finished

[ https://issues.apache.org/jira/browse/SPARK-50176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Andrew updated SPARK-50176: - Attachment: spark-issue-logs.log.gz > Connect operation stuck in `ReadyForExecution` even though

[jira] [Commented] (SPARK-50176) Connect operation stuck in `ReadyForExecution` even though query has finished

[ https://issues.apache.org/jira/browse/SPARK-50176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17894348#comment-17894348 ] Peter Andrew commented on SPARK-50176: -- [~changgyoopark-db] driver log is a bit big

[jira] [Commented] (SPARK-50176) Connect operation stuck in `ReadyForExecution` even though query has finished

[ https://issues.apache.org/jira/browse/SPARK-50176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17894214#comment-17894214 ] Peter Andrew commented on SPARK-50176: -- Note that once the SQL query has completed,

[jira] [Updated] (SPARK-50176) Connect operation stuck in `ReadyForExecution` even though query has finished

[ https://issues.apache.org/jira/browse/SPARK-50176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Andrew updated SPARK-50176: - Description: Using PySpark Connect to run `writeTo(...).overwritePartitions(...)` commands, I f

[jira] [Created] (SPARK-50176) Connect operation stuck in `ReadyForExecution` even though query has finished

Peter Andrew created SPARK-50176: Summary: Connect operation stuck in `ReadyForExecution` even though query has finished Key: SPARK-50176 URL: https://issues.apache.org/jira/browse/SPARK-50176 Project

[jira] [Updated] (SPARK-50118) Spark removes working directory while Python UDF runs

[ https://issues.apache.org/jira/browse/SPARK-50118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Andrew updated SPARK-50118: - Description: With Spark Connect + PySpark, we can stage files using `spark.addArtifacts`. When

[jira] [Updated] (SPARK-50118) Spark removes working directory while Python UDF runs

[ https://issues.apache.org/jira/browse/SPARK-50118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Andrew updated SPARK-50118: - Description: With Spark Connect + PySpark, we can stage files using `spark.addArtifacts`. When

[jira] [Created] (SPARK-50118) Spark removes working directory while Python UDF runs

Peter Andrew created SPARK-50118: Summary: Spark removes working directory while Python UDF runs Key: SPARK-50118 URL: https://issues.apache.org/jira/browse/SPARK-50118 Project: Spark Issue T

[jira] [Commented] (SPARK-37259) JDBC read is always going to wrap the query in a select statement

[ https://issues.apache.org/jira/browse/SPARK-37259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17449067#comment-17449067 ] Andrew commented on SPARK-37259: I've created another PR following [~KevinAppelBofa]  id

[jira] [Updated] (SPARK-36686) Fix SimplifyConditionalsInPredicate to be null-safe

[ https://issues.apache.org/jira/browse/SPARK-36686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew updated SPARK-36686: --- Description: SimplifyConditionalsInPredicate rule is not null-safe and leads to incorrect results (was: Sim

[jira] [Updated] (SPARK-36686) Fix SimplifyConditionalsInPredicate to be null-safe

[ https://issues.apache.org/jira/browse/SPARK-36686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew updated SPARK-36686: --- Description: SimplifyConditionalsInPredicate rule is not null-safe and leads to incorrect results   Repro

[jira] [Updated] (SPARK-36686) Fix SimplifyConditionalsInPredicate to be null-safe

[ https://issues.apache.org/jira/browse/SPARK-36686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew updated SPARK-36686: --- Description: SimplifyConditionalsInPredicate rule is not null-safe and leads to incorrect results   Repro

[jira] [Updated] (SPARK-36686) Fix SimplifyConditionalsInPredicate to be null-safe

[ https://issues.apache.org/jira/browse/SPARK-36686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew updated SPARK-36686: --- Description: SimplifyConditionalsInPredicate rule is not null-safe and leads to incorrect results   Repro

[jira] [Updated] (SPARK-36686) Fix SimplifyConditionalsInPredicate to be null-safe

[ https://issues.apache.org/jira/browse/SPARK-36686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew updated SPARK-36686: --- Description: SimplifyConditionalsInPredicate rule is not null-safe and leads to incorrect results   Repro

[jira] [Updated] (SPARK-36686) Fix SimplifyConditionalsInPredicate to be null-safe

[ https://issues.apache.org/jira/browse/SPARK-36686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew updated SPARK-36686: --- Description: SimplifyConditionalsInPredicate rule is not null-safe and leads to incorrect results   Repro

[jira] [Updated] (SPARK-36686) Fix SimplifyConditionalsInPredicate to be null-safe

[ https://issues.apache.org/jira/browse/SPARK-36686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew updated SPARK-36686: --- Summary: Fix SimplifyConditionalsInPredicate to be null-safe (was: fix SimplifyConditionalsInPredicate to b

[jira] [Created] (SPARK-36686) fix SimplifyConditionalsInPredicate to be null-safe

Andrew created SPARK-36686: -- Summary: fix SimplifyConditionalsInPredicate to be null-safe Key: SPARK-36686 URL: https://issues.apache.org/jira/browse/SPARK-36686 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-20035) Spark 2.0.2 writes empty file if no record is in the dataset

[ https://issues.apache.org/jira/browse/SPARK-20035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew updated SPARK-20035: --- Description: When there is no record in a dataset, the call to write with the spark-csv creates empty file (

[jira] [Created] (SPARK-20035) spark-csv writes empty file if no record is in the dataset

Andrew created SPARK-20035: -- Summary: spark-csv writes empty file if no record is in the dataset Key: SPARK-20035 URL: https://issues.apache.org/jira/browse/SPARK-20035 Project: Spark Issue Type: Bu