[jira] [Commented] (SPARK-31299) Pyspark.ml.clustering illegalArgumentException with dataframe created from rows

2020-04-01 Thread Lukas Thaler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17073422#comment-17073422 ] Lukas Thaler commented on SPARK-31299: -- Oh dear. Now, that's embarrassing. Thank yo

[jira] [Commented] (SPARK-31312) Transforming Hive simple UDF (using JAR) expression may incur CNFE in later evaluation

2020-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17073409#comment-17073409 ] Dongjoon Hyun commented on SPARK-31312: --- Since I saw your opinion, I'll not ping y

[jira] [Commented] (SPARK-31312) Transforming Hive simple UDF (using JAR) expression may incur CNFE in later evaluation

2020-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17073407#comment-17073407 ] Dongjoon Hyun commented on SPARK-31312: --- It's for informimg the users (and the dow

[jira] [Updated] (SPARK-31325) Control a plan explain mode in the events of SQL listeners via SQLConf

2020-04-01 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-31325: - Summary: Control a plan explain mode in the events of SQL listeners via SQLConf (was: C

[jira] [Created] (SPARK-31325) Control a plan explain mode in listeners via SQLConf

2020-04-01 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-31325: Summary: Control a plan explain mode in listeners via SQLConf Key: SPARK-31325 URL: https://issues.apache.org/jira/browse/SPARK-31325 Project: Spark

[jira] [Commented] (SPARK-30839) Add version information for Spark configuration

2020-04-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17073346#comment-17073346 ] Hyukjin Kwon commented on SPARK-30839: -- [~beliefer]do you have more tickets or PRs

[jira] [Resolved] (SPARK-31324) StreamingQuery stop() timeout exception should include the stream ID

2020-04-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31324. -- Fix Version/s: 3.0.0 Resolution: Fixed Fixed in https://github.com/apache/spark/pull/28

[jira] [Commented] (SPARK-31324) StreamingQuery stop() timeout exception should include the stream ID

2020-04-01 Thread Mukul Murthy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17073289#comment-17073289 ] Mukul Murthy commented on SPARK-31324: -- Gotcha, thanks. Was not planning on backpor

[jira] [Resolved] (SPARK-30297) Executor heartbeat expired cause app hung up forever

2020-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30297. --- Resolution: Duplicate > Executor heartbeat expired cause app hung up forever > -

[jira] [Commented] (SPARK-31324) StreamingQuery stop() timeout exception should include the stream ID

2020-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17073273#comment-17073273 ] Dongjoon Hyun commented on SPARK-31324: --- Hi, [~mukulmurthy]. Please use the versio

[jira] [Updated] (SPARK-31324) StreamingQuery stop() timeout exception should include the stream ID

2020-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31324: -- Priority: Trivial (was: Major) > StreamingQuery stop() timeout exception should include the s

[jira] [Updated] (SPARK-31324) StreamingQuery stop() timeout exception should include the stream ID

2020-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31324: -- Affects Version/s: (was: 2.4.4) 3.1.0 > StreamingQuery stop() timeo

[jira] [Commented] (SPARK-31312) Transforming Hive simple UDF (using JAR) expression may incur CNFE in later evaluation

2020-04-01 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17073264#comment-17073264 ] Jungtaek Lim commented on SPARK-31312: -- No, it wasn't triggered by SPARK-26560 and

[jira] [Commented] (SPARK-30951) Potential data loss for legacy applications after switch to proleptic Gregorian calendar

2020-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17073233#comment-17073233 ] Dongjoon Hyun commented on SPARK-30951: --- The decision will depend on how much peop

[jira] [Commented] (SPARK-30951) Potential data loss for legacy applications after switch to proleptic Gregorian calendar

2020-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17073231#comment-17073231 ] Dongjoon Hyun commented on SPARK-30951: --- I'm +1 basically. Please make a Jira issu

[jira] [Created] (SPARK-31324) StreamingQuery stop() timeout exception should include the stream ID

2020-04-01 Thread Mukul Murthy (Jira)
Mukul Murthy created SPARK-31324: Summary: StreamingQuery stop() timeout exception should include the stream ID Key: SPARK-31324 URL: https://issues.apache.org/jira/browse/SPARK-31324 Project: Spark

[jira] [Commented] (SPARK-29574) spark with user provided hadoop doesn't work on kubernetes

2020-04-01 Thread Devin Boyer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17073145#comment-17073145 ] Devin Boyer commented on SPARK-29574: - Will this or can this change be backported to

[jira] [Resolved] (SPARK-31322) rename QueryPlan.collectInPlanAndSubqueries to collectWithSubqueries

2020-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31322. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28092 [https://

[jira] [Assigned] (SPARK-31285) Do not upper case schedule mode

2020-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-31285: - Assignee: ulysses you > Do not upper case schedule mode > -

[jira] [Resolved] (SPARK-31285) Do not upper case schedule mode

2020-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31285. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28049 [https://

[jira] [Comment Edited] (SPARK-31312) Transforming Hive simple UDF (using JAR) expression may incur CNFE in later evaluation

2020-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17073075#comment-17073075 ] Dongjoon Hyun edited comment on SPARK-31312 at 4/1/20, 6:42 PM: --

[jira] [Updated] (SPARK-31312) Transforming Hive simple UDF (using JAR) expression may incur CNFE in later evaluation

2020-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31312: -- Affects Version/s: (was: 3.1.0) > Transforming Hive simple UDF (using JAR) expression may

[jira] [Commented] (SPARK-31312) Transforming Hive simple UDF (using JAR) expression may incur CNFE in later evaluation

2020-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17073075#comment-17073075 ] Dongjoon Hyun commented on SPARK-31312: --- Hi, [~kabhwan]. Could you add old version

[jira] [Updated] (SPARK-31313) Add `m01` node name to support Minikube 1.8.x

2020-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31313: -- Fix Version/s: (was: 3.1.0) 3.0.0 > Add `m01` node name to support Mini

[jira] [Updated] (SPARK-31313) Add `m01` node name to support Minikube 1.8.x

2020-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31313: -- Affects Version/s: (was: 3.1.0) 3.0.0 > Add `m01` node name to supp

[jira] [Updated] (SPARK-31299) Pyspark.ml.clustering illegalArgumentException with dataframe created from rows

2020-04-01 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-31299: - Description: I hope this is the right place and way to report a bug in (at least) the PySpark A

[jira] [Commented] (SPARK-31299) Pyspark.ml.clustering illegalArgumentException with dataframe created from rows

2020-04-01 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17073027#comment-17073027 ] Bryan Cutler commented on SPARK-31299: -- It looks like you are using {{DenseVector}}

[jira] [Resolved] (SPARK-31299) Pyspark.ml.clustering illegalArgumentException with dataframe created from rows

2020-04-01 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-31299. -- Resolution: Not A Problem > Pyspark.ml.clustering illegalArgumentException with dataframe crea

[jira] [Commented] (SPARK-30951) Potential data loss for legacy applications after switch to proleptic Gregorian calendar

2020-04-01 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17073026#comment-17073026 ] Bruce Robbins commented on SPARK-30951: --- Since there is some inherit danger in a m

[jira] [Created] (SPARK-31323) Stage level scheduling: dedup resource profiles on creation

2020-04-01 Thread Thomas Graves (Jira)
Thomas Graves created SPARK-31323: - Summary: Stage level scheduling: dedup resource profiles on creation Key: SPARK-31323 URL: https://issues.apache.org/jira/browse/SPARK-31323 Project: Spark

[jira] [Commented] (SPARK-30443) "Managed memory leak detected" even with no calls to take() or limit()

2020-04-01 Thread Ben Roling (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072857#comment-17072857 ] Ben Roling commented on SPARK-30443: Thanks [~xiaojuwu]. Using explain() on the tes

[jira] [Created] (SPARK-31322) rename QueryPlan.collectInPlanAndSubqueries to collectWithSubqueries

2020-04-01 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-31322: --- Summary: rename QueryPlan.collectInPlanAndSubqueries to collectWithSubqueries Key: SPARK-31322 URL: https://issues.apache.org/jira/browse/SPARK-31322 Project: Spark

[jira] [Commented] (SPARK-28624) make_date is inconsistent when reading from table

2020-04-01 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072694#comment-17072694 ] Yuming Wang commented on SPARK-28624: - [~maxgekk] collect() also returns incorrect r

[jira] [Updated] (SPARK-28624) make_date is inconsistent when reading from table

2020-04-01 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28624: Attachment: collect make_date.png > make_date is inconsistent when reading from table > --

[jira] [Updated] (SPARK-31320) Fix release script for 3.0.0

2020-04-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31320: - Summary: Fix release script for 3.0.0 (was: fix release script for 3.0.0) > Fix release script

[jira] [Resolved] (SPARK-31320) fix release script for 3.0.0

2020-04-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31320. -- Fix Version/s: 3.0.0 Resolution: Fixed Fixed in https://github.com/apache/spark/pull/28

[jira] [Created] (SPARK-31321) Remove SaveMode check in v2 FileWriteBuilder

2020-04-01 Thread Kent Yao (Jira)
Kent Yao created SPARK-31321: Summary: Remove SaveMode check in v2 FileWriteBuilder Key: SPARK-31321 URL: https://issues.apache.org/jira/browse/SPARK-31321 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-31311) Benchmark date-time rebasing in ORC datasource

2020-04-01 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31311. - Resolution: Fixed Issue resolved by pull request 28076 [https://github.com/apache/spark/pull/280