[jira] [Assigned] (SPARK-21350) Fix the error message when the number of arguments is wrong when invoking a UDF

[ https://issues.apache.org/jira/browse/SPARK-21350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21350: Assignee: Apache Spark (was: Xiao Li) > Fix the error message when the number of

[jira] [Commented] (SPARK-21350) Fix the error message when the number of arguments is wrong when invoking a UDF

[ https://issues.apache.org/jira/browse/SPARK-21350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079433#comment-16079433 ] Apache Spark commented on SPARK-21350: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-21350) Fix the error message when the number of arguments is wrong when invoking a UDF

[ https://issues.apache.org/jira/browse/SPARK-21350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21350: Assignee: Xiao Li (was: Apache Spark) > Fix the error message when the number of

[jira] [Created] (SPARK-21350) Fix the error message when the number of arguments is wrong when invoking a UDF

Xiao Li created SPARK-21350: --- Summary: Fix the error message when the number of arguments is wrong when invoking a UDF Key: SPARK-21350 URL: https://issues.apache.org/jira/browse/SPARK-21350 Project: Spark

[jira] [Updated] (SPARK-21345) SparkSessionBuilderSuite should clean up stopped sessions

[ https://issues.apache.org/jira/browse/SPARK-21345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-21345: Fix Version/s: 2.1.2 > SparkSessionBuilderSuite should clean up stopped sessions >

[jira] [Comment Edited] (SPARK-21348) Spark history Server load too slow when eventlog is large

[ https://issues.apache.org/jira/browse/SPARK-21348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079421#comment-16079421 ] duyanghao edited comment on SPARK-21348 at 7/9/17 2:36 AM: --- [~jerryshao] ok, i

[jira] [Commented] (SPARK-21348) Spark history Server load too slow when eventlog is large

[ https://issues.apache.org/jira/browse/SPARK-21348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079421#comment-16079421 ] duyanghao commented on SPARK-21348: --- @Saisai Shao ok, i will follow

[jira] [Comment Edited] (SPARK-21348) Spark history Server load too slow when eventlog is large

[ https://issues.apache.org/jira/browse/SPARK-21348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079421#comment-16079421 ] duyanghao edited comment on SPARK-21348 at 7/9/17 2:35 AM: --- [~jerryshao] ok, i

[jira] [Closed] (SPARK-21348) Spark history Server load too slow when eventlog is large

[ https://issues.apache.org/jira/browse/SPARK-21348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] duyanghao closed SPARK-21348. - Resolution: Duplicate > Spark history Server load too slow when eventlog is large >

[jira] [Comment Edited] (SPARK-21346) Spark does not use SSL for HTTP File Server and Broadcast Server

[ https://issues.apache.org/jira/browse/SPARK-21346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079407#comment-16079407 ] Saisai Shao edited comment on SPARK-21346 at 7/9/17 1:29 AM: - Jars, files can

[jira] [Commented] (SPARK-21346) Spark does not use SSL for HTTP File Server and Broadcast Server

[ https://issues.apache.org/jira/browse/SPARK-21346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079407#comment-16079407 ] Saisai Shao commented on SPARK-21346: - Jars, files can be downloaded from HTTPS if you configured

[jira] [Comment Edited] (SPARK-21346) Spark does not use SSL for HTTP File Server and Broadcast Server

[ https://issues.apache.org/jira/browse/SPARK-21346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079405#comment-16079405 ] John edited comment on SPARK-21346 at 7/9/17 1:27 AM: -- Sorry, I wasn't aware of

[jira] [Commented] (SPARK-21346) Spark does not use SSL for HTTP File Server and Broadcast Server

[ https://issues.apache.org/jira/browse/SPARK-21346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079405#comment-16079405 ] John commented on SPARK-21346: -- Sorry, I wasn't aware of that. Do you mind elaborating more on what kind of

[jira] [Commented] (SPARK-20660) Not able to merge Dataframes with different column orders

[ https://issues.apache.org/jira/browse/SPARK-20660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079402#comment-16079402 ] Takeshi Yamamuro commented on SPARK-20660: -- You mean this function like?

[jira] [Comment Edited] (SPARK-21334) Fix metrics for external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-21334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079389#comment-16079389 ] Saisai Shao edited comment on SPARK-21334 at 7/9/17 12:43 AM: -- Are you using

[jira] [Commented] (SPARK-21334) Fix metrics for external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-21334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079389#comment-16079389 ] Saisai Shao commented on SPARK-21334: - Are you using external shuffle service with YARN? If so I

[jira] [Commented] (SPARK-21346) Spark does not use SSL for HTTP File Server and Broadcast Server

[ https://issues.apache.org/jira/browse/SPARK-21346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079385#comment-16079385 ] Saisai Shao commented on SPARK-21346: - SSL is still used for fetch remote resources from HTTPS. If

[jira] [Commented] (SPARK-21348) Spark history Server load too slow when eventlog is large

[ https://issues.apache.org/jira/browse/SPARK-21348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079381#comment-16079381 ] Saisai Shao commented on SPARK-21348: - SPARK-18085 is trying to address this issue. [~duyanghao] are

[jira] [Assigned] (SPARK-21349) Make TASK_SIZE_TO_WARN_KB configurable

[ https://issues.apache.org/jira/browse/SPARK-21349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21349: Assignee: (was: Apache Spark) > Make TASK_SIZE_TO_WARN_KB configurable >

[jira] [Commented] (SPARK-21349) Make TASK_SIZE_TO_WARN_KB configurable

[ https://issues.apache.org/jira/browse/SPARK-21349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079349#comment-16079349 ] Apache Spark commented on SPARK-21349: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-21349) Make TASK_SIZE_TO_WARN_KB configurable

[ https://issues.apache.org/jira/browse/SPARK-21349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21349: Assignee: Apache Spark > Make TASK_SIZE_TO_WARN_KB configurable >

[jira] [Updated] (SPARK-21349) Make TASK_SIZE_TO_WARN_KB configurable

[ https://issues.apache.org/jira/browse/SPARK-21349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-21349: -- Description: Although this is just a warning message, this issue tries to make

[jira] [Created] (SPARK-21349) Make TASK_SIZE_TO_WARN_KB configurable

Dongjoon Hyun created SPARK-21349: - Summary: Make TASK_SIZE_TO_WARN_KB configurable Key: SPARK-21349 URL: https://issues.apache.org/jira/browse/SPARK-21349 Project: Spark Issue Type:

[jira] [Commented] (SPARK-20660) Not able to merge Dataframes with different column orders

[ https://issues.apache.org/jira/browse/SPARK-20660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079340#comment-16079340 ] Anton Okolnychyi commented on SPARK-20660: -- I used the following code to investigate this

[jira] [Commented] (SPARK-19568) Must include class/method documentation for CRAN check

[ https://issues.apache.org/jira/browse/SPARK-19568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079325#comment-16079325 ] Felix Cheung commented on SPARK-19568: -- separate from CRAN task > Must include class/method

[jira] [Updated] (SPARK-19568) Must include class/method documentation for CRAN check

[ https://issues.apache.org/jira/browse/SPARK-19568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-19568: - Issue Type: Bug (was: Sub-task) Parent: (was: SPARK-15799) > Must include

[jira] [Updated] (SPARK-21290) R document Programmatically Specifying the Schema in SQL guide

[ https://issues.apache.org/jira/browse/SPARK-21290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-21290: - Target Version/s: 2.3.0 > R document Programmatically Specifying the Schema in SQL guide >

[jira] [Commented] (SPARK-21290) R document Programmatically Specifying the Schema in SQL guide

[ https://issues.apache.org/jira/browse/SPARK-21290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079323#comment-16079323 ] Felix Cheung commented on SPARK-21290: -- with some changes to schema string support on the way, let's

[jira] [Resolved] (SPARK-21093) Multiple gapply execution occasionally failed in SparkR

[ https://issues.apache.org/jira/browse/SPARK-21093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-21093. -- Resolution: Fixed Fix Version/s: 2.3.0 > Multiple gapply execution occasionally failed

[jira] [Commented] (SPARK-21345) SparkSessionBuilderSuite should clean up stopped sessions

[ https://issues.apache.org/jira/browse/SPARK-21345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079308#comment-16079308 ] Apache Spark commented on SPARK-21345: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Closed] (SPARK-21307) Remove SQLConf parameters from the parser-related classes.

[ https://issues.apache.org/jira/browse/SPARK-21307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li closed SPARK-21307. --- Resolution: Won't Fix > Remove SQLConf parameters from the parser-related classes. >

[jira] [Assigned] (SPARK-21343) Refine the document for spark.reducer.maxReqSizeShuffleToMem

[ https://issues.apache.org/jira/browse/SPARK-21343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21343: --- Assignee: jin xing > Refine the document for spark.reducer.maxReqSizeShuffleToMem >

[jira] [Resolved] (SPARK-21343) Refine the document for spark.reducer.maxReqSizeShuffleToMem

[ https://issues.apache.org/jira/browse/SPARK-21343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21343. - Resolution: Fixed Fix Version/s: 2.3.0 2.2.1 Issue resolved by pull

[jira] [Assigned] (SPARK-20342) DAGScheduler sends SparkListenerTaskEnd before updating task's accumulators

[ https://issues.apache.org/jira/browse/SPARK-20342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20342: --- Assignee: Marcelo Vanzin > DAGScheduler sends SparkListenerTaskEnd before updating task's

[jira] [Resolved] (SPARK-20342) DAGScheduler sends SparkListenerTaskEnd before updating task's accumulators

[ https://issues.apache.org/jira/browse/SPARK-20342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20342. - Resolution: Fixed Fix Version/s: 2.3.0 2.2.1 Issue resolved by pull

[jira] [Comment Edited] (SPARK-18859) Catalyst codegen does not mark column as nullable when it should. Causes NPE

[ https://issues.apache.org/jira/browse/SPARK-18859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076463#comment-16076463 ] Anton Okolnychyi edited comment on SPARK-18859 at 7/8/17 2:36 PM: --

[jira] [Commented] (SPARK-9476) Kafka stream loses leader after 2h of operation

[ https://issues.apache.org/jira/browse/SPARK-9476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079148#comment-16079148 ] Dharmas commented on SPARK-9476: Even I am getting the same issue !. any update on this. > Kafka stream

[jira] [Commented] (SPARK-6227) PCA and SVD for PySpark

[ https://issues.apache.org/jira/browse/SPARK-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079132#comment-16079132 ] Abou Haydar Elias commented on SPARK-6227: -- I was also wondering if we can add PCA and SVD in

[jira] [Resolved] (SPARK-21083) Store zero size and row count after analyzing empty table

[ https://issues.apache.org/jira/browse/SPARK-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21083. - Resolution: Fixed Assignee: Zhenhua Wang Fix Version/s: 2.3.0 > Store zero size

[jira] [Assigned] (SPARK-21345) SparkSessionBuilderSuite should clean up stopped sessions

[ https://issues.apache.org/jira/browse/SPARK-21345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21345: --- Assignee: Dongjoon Hyun > SparkSessionBuilderSuite should clean up stopped sessions >

[jira] [Resolved] (SPARK-21345) SparkSessionBuilderSuite should clean up stopped sessions

[ https://issues.apache.org/jira/browse/SPARK-21345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21345. - Resolution: Fixed Fix Version/s: 2.3.0 2.2.1 Issue resolved by pull

[jira] [Updated] (SPARK-21228) InSet incorrect handling of structs

[ https://issues.apache.org/jira/browse/SPARK-21228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-21228: Fix Version/s: 2.2.1 > InSet incorrect handling of structs > --- >

[jira] [Assigned] (SPARK-21344) BinaryType comparison does signed byte array comparison

[ https://issues.apache.org/jira/browse/SPARK-21344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21344: Assignee: Apache Spark > BinaryType comparison does signed byte array comparison >

[jira] [Commented] (SPARK-21344) BinaryType comparison does signed byte array comparison

[ https://issues.apache.org/jira/browse/SPARK-21344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079065#comment-16079065 ] Apache Spark commented on SPARK-21344: -- User 'kiszk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-21344) BinaryType comparison does signed byte array comparison

[ https://issues.apache.org/jira/browse/SPARK-21344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21344: Assignee: (was: Apache Spark) > BinaryType comparison does signed byte array

[jira] [Commented] (SPARK-21100) Add summary method as alternative to describe that gives quartiles similar to Pandas

[ https://issues.apache.org/jira/browse/SPARK-21100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079057#comment-16079057 ] Apache Spark commented on SPARK-21100: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079032#comment-16079032 ] Kazuaki Ishizaki commented on SPARK-21337: -- I cannot reproduce this by using the latest or v2.1

[jira] [Updated] (SPARK-21083) Store zero size and row count after analyzing empty table

[ https://issues.apache.org/jira/browse/SPARK-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-21083: - Summary: Store zero size and row count after analyzing empty table (was: Do not remove correct

[jira] [Updated] (SPARK-21083) Store zero size and row count after analyzing empty table

[ https://issues.apache.org/jira/browse/SPARK-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-21083: - Description: (was: 1. When we first analyze without `noscan` and then analyze with `noscan`,

[jira] [Commented] (SPARK-21344) BinaryType comparison does signed byte array comparison

[ https://issues.apache.org/jira/browse/SPARK-21344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079023#comment-16079023 ] Sean Owen commented on SPARK-21344: --- Bytes are signed, as are other int types in Spark SQL / Hive. Is

[jira] [Resolved] (SPARK-21347) Performance issues with KryoSerializer

[ https://issues.apache.org/jira/browse/SPARK-21347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21347. --- Resolution: Not A Problem I think this is more of question, and about a change or configuration in

[jira] [Commented] (SPARK-21341) Spark 2.1.1: I want to be able to serialize wordVectors on Word2VecModel

[ https://issues.apache.org/jira/browse/SPARK-21341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079019#comment-16079019 ] Sean Owen commented on SPARK-21341: --- Hm, I don't think you normally save a model with saveAsObjectFile

[jira] [Commented] (SPARK-21344) BinaryType comparison does signed byte array comparison

[ https://issues.apache.org/jira/browse/SPARK-21344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079017#comment-16079017 ] Dongjoon Hyun commented on SPARK-21344: --- +1 ! :) > BinaryType comparison does signed byte array

[jira] [Commented] (SPARK-21344) BinaryType comparison does signed byte array comparison

[ https://issues.apache.org/jira/browse/SPARK-21344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079010#comment-16079010 ] Kazuaki Ishizaki commented on SPARK-21344: -- I will work for this if anyone has finished a PR. >

[jira] [Assigned] (SPARK-20456) Add examples for functions collection for pyspark

[ https://issues.apache.org/jira/browse/SPARK-20456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reassigned SPARK-20456: Assignee: Michael Patterson > Add examples for functions collection for pyspark >

[jira] [Resolved] (SPARK-20456) Add examples for functions collection for pyspark

[ https://issues.apache.org/jira/browse/SPARK-20456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20456. -- Resolution: Fixed Fix Version/s: 2.3.0 Target Version/s: 2.3.0 > Add examples

[jira] [Resolved] (SPARK-20307) SparkR: pass on setHandleInvalid to spark.mllib functions that use StringIndexer

[ https://issues.apache.org/jira/browse/SPARK-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20307. -- Resolution: Fixed Assignee: Miao Wang Fix Version/s: 2.3.0 Target

[jira] [Resolved] (SPARK-21069) Add rate source to programming guide

[ https://issues.apache.org/jira/browse/SPARK-21069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-21069. -- Resolution: Fixed Assignee: Prashant Sharma Fix Version/s: 2.3.0

[jira] [Comment Edited] (SPARK-21341) Spark 2.1.1: I want to be able to serialize wordVectors on Word2VecModel

[ https://issues.apache.org/jira/browse/SPARK-21341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16078987#comment-16078987 ] Yan Facai (颜发才) edited comment on SPARK-21341 at 7/8/17 6:28 AM: - Hi,

[jira] [Comment Edited] (SPARK-21341) Spark 2.1.1: I want to be able to serialize wordVectors on Word2VecModel

[ https://issues.apache.org/jira/browse/SPARK-21341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16078987#comment-16078987 ] Yan Facai (颜发才) edited comment on SPARK-21341 at 7/8/17 6:28 AM: - Hi,

[jira] [Commented] (SPARK-21341) Spark 2.1.1: I want to be able to serialize wordVectors on Word2VecModel

[ https://issues.apache.org/jira/browse/SPARK-21341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16078987#comment-16078987 ] Yan Facai (颜发才) commented on SPARK-21341: - Hi, [~zsellami]. I guess that since the wordVectors is

[jira] [Assigned] (SPARK-20379) Allow setting SSL-related passwords through env variables

[ https://issues.apache.org/jira/browse/SPARK-20379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20379: --- Assignee: Marcelo Vanzin > Allow setting SSL-related passwords through env variables >

[jira] [Resolved] (SPARK-20379) Allow setting SSL-related passwords through env variables

[ https://issues.apache.org/jira/browse/SPARK-20379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20379. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18394

[jira] [Resolved] (SPARK-21281) cannot create empty typed array column

[ https://issues.apache.org/jira/browse/SPARK-21281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-21281. - Resolution: Fixed Assignee: Takeshi Yamamuro Fix Version/s: 2.3.0 > cannot create empty