[jira] [Resolved] (SPARK-20307) SparkR: pass on setHandleInvalid to spark.mllib functions that use StringIndexer

2017-07-07 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20307. -- Resolution: Fixed Assignee: Miao Wang Fix Version/s: 2.3.0 Target V

[jira] [Resolved] (SPARK-21069) Add rate source to programming guide

2017-07-07 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-21069. -- Resolution: Fixed Assignee: Prashant Sharma Fix Version/s: 2.3.0

[jira] [Comment Edited] (SPARK-21341) Spark 2.1.1: I want to be able to serialize wordVectors on Word2VecModel

2017-07-07 Thread 颜发才
[ https://issues.apache.org/jira/browse/SPARK-21341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078987#comment-16078987 ] Yan Facai (颜发才) edited comment on SPARK-21341 at 7/8/17 6:28 AM: --

[jira] [Comment Edited] (SPARK-21341) Spark 2.1.1: I want to be able to serialize wordVectors on Word2VecModel

2017-07-07 Thread 颜发才
[ https://issues.apache.org/jira/browse/SPARK-21341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078987#comment-16078987 ] Yan Facai (颜发才) edited comment on SPARK-21341 at 7/8/17 6:28 AM: --

[jira] [Commented] (SPARK-21341) Spark 2.1.1: I want to be able to serialize wordVectors on Word2VecModel

2017-07-07 Thread 颜发才
[ https://issues.apache.org/jira/browse/SPARK-21341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078987#comment-16078987 ] Yan Facai (颜发才) commented on SPARK-21341: - Hi, [~zsellami]. I guess that since th

[jira] [Assigned] (SPARK-20379) Allow setting SSL-related passwords through env variables

2017-07-07 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20379: --- Assignee: Marcelo Vanzin > Allow setting SSL-related passwords through env variables > -

[jira] [Resolved] (SPARK-20379) Allow setting SSL-related passwords through env variables

2017-07-07 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20379. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18394 [https://githu

[jira] [Resolved] (SPARK-21281) cannot create empty typed array column

2017-07-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-21281. - Resolution: Fixed Assignee: Takeshi Yamamuro Fix Version/s: 2.3.0 > cannot create empty t

[jira] [Resolved] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

2017-07-07 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21337. -- Resolution: Cannot Reproduce I am resolving this per above comment and I could not reproduce th

[jira] [Assigned] (SPARK-21100) Add summary method as alternative to describe that gives quartiles similar to Pandas

2017-07-07 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21100: --- Assignee: Andrew Ray > Add summary method as alternative to describe that gives quartiles si

[jira] [Resolved] (SPARK-21100) Add summary method as alternative to describe that gives quartiles similar to Pandas

2017-07-07 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21100. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18307 [https://githu

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

2017-07-07 Thread fengchaoge (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Description: when there are large 'case when ' expressions in spark sql,the CodeGenerator failed to

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

2017-07-07 Thread fengchaoge (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Description: when there are large 'case when ' expressions in spark sql,the CodeGenerator failed to

[jira] [Commented] (SPARK-21343) Refine the document for spark.reducer.maxReqSizeShuffleToMem

2017-07-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078912#comment-16078912 ] Apache Spark commented on SPARK-21343: -- User 'jinxing64' has created a pull request

[jira] [Assigned] (SPARK-21343) Refine the document for spark.reducer.maxReqSizeShuffleToMem

2017-07-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21343: Assignee: Apache Spark > Refine the document for spark.reducer.maxReqSizeShuffleToMem > --

[jira] [Assigned] (SPARK-21343) Refine the document for spark.reducer.maxReqSizeShuffleToMem

2017-07-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21343: Assignee: (was: Apache Spark) > Refine the document for spark.reducer.maxReqSizeShuffl

[jira] [Updated] (SPARK-21083) Do not remove correct stats when re-analyze

2017-07-07 Thread Zhenhua Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-21083: - Summary: Do not remove correct stats when re-analyze (was: Consider staleness when collecting co

[jira] [Updated] (SPARK-21083) Consider staleness when collecting column stats

2017-07-07 Thread Zhenhua Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-21083: - Description: 1. When we first analyze without `noscan` and then analyze with `noscan`, the table

[jira] [Updated] (SPARK-21348) Spark history Server load too slow when eventlog is large

2017-07-07 Thread duyanghao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] duyanghao updated SPARK-21348: -- Description: There is a eventlog file in hdfs as below: ``` -rwxrwx--- 3 root supergroup 9634238709 2

[jira] [Updated] (SPARK-21348) Spark history Server load too slow when eventlog is large

2017-07-07 Thread duyanghao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] duyanghao updated SPARK-21348: -- Description: There is a eventlog file in hdfs as below: ``` -rwxrwx--- 3 root supergroup 9634238709 2

[jira] [Updated] (SPARK-21348) Spark history Server load too slow when eventlog is large

2017-07-07 Thread duyanghao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] duyanghao updated SPARK-21348: -- Description: There is a eventlog file in hdfs as below: ``` -rwxrwx--- 3 root supergroup 9634238709 2

[jira] [Created] (SPARK-21348) Spark history Server load too slow when eventlog is large

2017-07-07 Thread duyanghao (JIRA)
duyanghao created SPARK-21348: - Summary: Spark history Server load too slow when eventlog is large Key: SPARK-21348 URL: https://issues.apache.org/jira/browse/SPARK-21348 Project: Spark Issue Typ

[jira] [Commented] (SPARK-21190) SPIP: Vectorized UDFs in Python

2017-07-07 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078837#comment-16078837 ] Bryan Cutler commented on SPARK-21190: -- [~rxin] I was talking about 2 different thin

[jira] [Commented] (SPARK-21341) Spark 2.1.1: I want to be able to serialize wordVectors on Word2VecModel

2017-07-07 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078832#comment-16078832 ] Dongjoon Hyun commented on SPARK-21341: --- Hi, [~zsellami]. For questions, you had be

[jira] [Commented] (SPARK-21344) BinaryType comparison does signed byte array comparison

2017-07-07 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078827#comment-16078827 ] Dongjoon Hyun commented on SPARK-21344: --- Also, this cannot be blocker because Spark

[jira] [Updated] (SPARK-21344) BinaryType comparison does signed byte array comparison

2017-07-07 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-21344: -- Affects Version/s: 2.0.0 > BinaryType comparison does signed byte array comparison > --

[jira] [Commented] (SPARK-21344) BinaryType comparison does signed byte array comparison

2017-07-07 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078825#comment-16078825 ] Dongjoon Hyun commented on SPARK-21344: --- Hi, [~shubhamc]. Thank you for reporting.

[jira] [Updated] (SPARK-21344) BinaryType comparison does signed byte array comparison

2017-07-07 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-21344: -- Priority: Major (was: Blocker) > BinaryType comparison does signed byte array comparison > ---

[jira] [Created] (SPARK-21347) Performance issues with KryoSerializer

2017-07-07 Thread Vishwanath Lakkundi (JIRA)
Vishwanath Lakkundi created SPARK-21347: --- Summary: Performance issues with KryoSerializer Key: SPARK-21347 URL: https://issues.apache.org/jira/browse/SPARK-21347 Project: Spark Issue Ty

[jira] [Created] (SPARK-21346) Spark does not use SSL for HTTP File Server and Broadcast Server

2017-07-07 Thread John (JIRA)
John created SPARK-21346: Summary: Spark does not use SSL for HTTP File Server and Broadcast Server Key: SPARK-21346 URL: https://issues.apache.org/jira/browse/SPARK-21346 Project: Spark Issue Type:

[jira] [Updated] (SPARK-20441) Within the same streaming query, one StreamingRelation should only be transformed to one StreamingExecutionRelation

2017-07-07 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-20441: - Affects Version/s: (was: 2.2.0) > Within the same streaming query, one StreamingRelat

[jira] [Updated] (SPARK-20441) Within the same streaming query, one StreamingRelation should only be transformed to one StreamingExecutionRelation

2017-07-07 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-20441: - Fix Version/s: 2.2.0 > Within the same streaming query, one StreamingRelation should only

[jira] [Assigned] (SPARK-21345) SparkSessionBuilderSuite should clean up stopped sessions

2017-07-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21345: Assignee: Apache Spark > SparkSessionBuilderSuite should clean up stopped sessions > -

[jira] [Assigned] (SPARK-21345) SparkSessionBuilderSuite should clean up stopped sessions

2017-07-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21345: Assignee: (was: Apache Spark) > SparkSessionBuilderSuite should clean up stopped sessi

[jira] [Commented] (SPARK-21345) SparkSessionBuilderSuite should clean up stopped sessions

2017-07-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078715#comment-16078715 ] Apache Spark commented on SPARK-21345: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Commented] (SPARK-21307) Remove SQLConf parameters from the parser-related classes.

2017-07-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078698#comment-16078698 ] Apache Spark commented on SPARK-21307: -- User 'gatorsmile' has created a pull request

[jira] [Updated] (SPARK-21345) SparkSessionBuilderSuite should clean up stopped sessions

2017-07-07 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-21345: -- Component/s: Tests > SparkSessionBuilderSuite should clean up stopped sessions > --

[jira] [Created] (SPARK-21345) SparkSessionBuilderSuite should clean up stopped sessions

2017-07-07 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-21345: - Summary: SparkSessionBuilderSuite should clean up stopped sessions Key: SPARK-21345 URL: https://issues.apache.org/jira/browse/SPARK-21345 Project: Spark I

[jira] [Updated] (SPARK-21344) BinaryType comparison does signed byte array comparison

2017-07-07 Thread Shubham Chopra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shubham Chopra updated SPARK-21344: --- Priority: Blocker (was: Major) > BinaryType comparison does signed byte array comparison > -

[jira] [Created] (SPARK-21344) BinaryType comparison does signed byte array comparison

2017-07-07 Thread Shubham Chopra (JIRA)
Shubham Chopra created SPARK-21344: -- Summary: BinaryType comparison does signed byte array comparison Key: SPARK-21344 URL: https://issues.apache.org/jira/browse/SPARK-21344 Project: Spark I

[jira] [Updated] (SPARK-21344) BinaryType comparison does signed byte array comparison

2017-07-07 Thread Shubham Chopra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shubham Chopra updated SPARK-21344: --- Description: BinaryType used by Spark SQL defines ordering using signed byte comparisons. Th

[jira] [Commented] (SPARK-21254) History UI: Taking over 1 minute for initial page display

2017-07-07 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078523#comment-16078523 ] Marcelo Vanzin commented on SPARK-21254: I've re-opened this because right, there

[jira] [Reopened] (SPARK-21254) History UI: Taking over 1 minute for initial page display

2017-07-07 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reopened SPARK-21254: > History UI: Taking over 1 minute for initial page display > -

[jira] [Commented] (SPARK-18085) Better History Server scalability for many / large applications

2017-07-07 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078517#comment-16078517 ] Marcelo Vanzin commented on SPARK-18085: [~rxin] this has been open with a specif

[jira] [Closed] (SPARK-21331) java.lang.NullPointerException for certain methods in classes of MLlib

2017-07-07 Thread Anirban Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirban Das closed SPARK-21331. --- Resolution: Invalid This was a version issue. I was using mllib 2.1.0 with spark 2.1.1. > java.lang.

[jira] [Commented] (SPARK-21331) java.lang.NullPointerException for certain methods in classes of MLlib

2017-07-07 Thread Anirban Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078449#comment-16078449 ] Anirban Das commented on SPARK-21331: - Thanks for pointing it out. I feel stupid now,

[jira] [Updated] (SPARK-21213) Support collecting partition-level statistics: rowCount and sizeInBytes

2017-07-07 Thread Maria (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maria updated SPARK-21213: -- Description: Support ANALYZE TABLE table PARTITION (key=value,...) COMPUTE STATISTICS [NOSCAN] SQL command to c

[jira] [Comment Edited] (SPARK-20597) KafkaSourceProvider falls back on path as synonym for topic

2017-07-07 Thread Satyajit varma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078395#comment-16078395 ] Satyajit varma edited comment on SPARK-20597 at 7/7/17 5:13 PM: ---

[jira] [Comment Edited] (SPARK-20597) KafkaSourceProvider falls back on path as synonym for topic

2017-07-07 Thread Satyajit varma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078395#comment-16078395 ] Satyajit varma edited comment on SPARK-20597 at 7/7/17 5:12 PM: ---

[jira] [Commented] (SPARK-20597) KafkaSourceProvider falls back on path as synonym for topic

2017-07-07 Thread Satyajit varma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078395#comment-16078395 ] Satyajit varma commented on SPARK-20597: Hi [~jlaskowski], I am almost done, wit

[jira] [Comment Edited] (SPARK-20597) KafkaSourceProvider falls back on path as synonym for topic

2017-07-07 Thread Satyajit varma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078395#comment-16078395 ] Satyajit varma edited comment on SPARK-20597 at 7/7/17 5:10 PM: ---

[jira] [Created] (SPARK-21343) Refine the document for spark.reducer.maxReqSizeShuffleToMem

2017-07-07 Thread jin xing (JIRA)
jin xing created SPARK-21343: Summary: Refine the document for spark.reducer.maxReqSizeShuffleToMem Key: SPARK-21343 URL: https://issues.apache.org/jira/browse/SPARK-21343 Project: Spark Issue T

[jira] [Commented] (SPARK-21342) Fix DownloadCallback to work well with RetryingBlockFetcher

2017-07-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078254#comment-16078254 ] Apache Spark commented on SPARK-21342: -- User 'jinxing64' has created a pull request

[jira] [Assigned] (SPARK-21342) Fix DownloadCallback to work well with RetryingBlockFetcher

2017-07-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21342: Assignee: Apache Spark > Fix DownloadCallback to work well with RetryingBlockFetcher > ---

[jira] [Assigned] (SPARK-21342) Fix DownloadCallback to work well with RetryingBlockFetcher

2017-07-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21342: Assignee: (was: Apache Spark) > Fix DownloadCallback to work well with RetryingBlockFe

[jira] [Created] (SPARK-21342) Fix DownloadCallback to work well with RetryingBlockFetcher

2017-07-07 Thread jin xing (JIRA)
jin xing created SPARK-21342: Summary: Fix DownloadCallback to work well with RetryingBlockFetcher Key: SPARK-21342 URL: https://issues.apache.org/jira/browse/SPARK-21342 Project: Spark Issue Ty

[jira] [Resolved] (SPARK-19358) LiveListenerBus shall log the event name when dropping them due to a fully filled queue

2017-07-07 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19358. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 16697 [https://githu

[jira] [Assigned] (SPARK-19358) LiveListenerBus shall log the event name when dropping them due to a fully filled queue

2017-07-07 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-19358: --- Assignee: Nan Zhu > LiveListenerBus shall log the event name when dropping them due to a ful

[jira] [Created] (SPARK-21341) Spark 2.1.1: I want to be able to serialize wordVectors on Word2VecModel

2017-07-07 Thread Zied Sellami (JIRA)
Zied Sellami created SPARK-21341: Summary: Spark 2.1.1: I want to be able to serialize wordVectors on Word2VecModel Key: SPARK-21341 URL: https://issues.apache.org/jira/browse/SPARK-21341 Project: Sp

[jira] [Created] (SPARK-21340) Bring PySpark MLLib evaluation metrics to parity with Scala API

2017-07-07 Thread Jake Charland (JIRA)
Jake Charland created SPARK-21340: - Summary: Bring PySpark MLLib evaluation metrics to parity with Scala API Key: SPARK-21340 URL: https://issues.apache.org/jira/browse/SPARK-21340 Project: Spark

[jira] [Created] (SPARK-21339) spark-shell --packages option does not add jars to classpath on windows

2017-07-07 Thread Goran Blankendal (JIRA)
Goran Blankendal created SPARK-21339: Summary: spark-shell --packages option does not add jars to classpath on windows Key: SPARK-21339 URL: https://issues.apache.org/jira/browse/SPARK-21339 Proje

[jira] [Resolved] (SPARK-21335) support un-aliased subquery

2017-07-07 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21335. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18559 [https://githu

[jira] [Commented] (SPARK-20870) Update the output of spark-sql -H

2017-07-07 Thread Prashant Sharma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16077988#comment-16077988 ] Prashant Sharma commented on SPARK-20870: - Is it okay to use, https://github.com/

[jira] [Commented] (SPARK-21228) InSet incorrect handling of structs

2017-07-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16077985#comment-16077985 ] Apache Spark commented on SPARK-21228: -- User 'bogdanrdc' has created a pull request

[jira] [Assigned] (SPARK-21069) Add rate source to programming guide

2017-07-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21069: Assignee: Apache Spark > Add rate source to programming guide > --

[jira] [Assigned] (SPARK-21069) Add rate source to programming guide

2017-07-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21069: Assignee: (was: Apache Spark) > Add rate source to programming guide > ---

[jira] [Commented] (SPARK-21069) Add rate source to programming guide

2017-07-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16077982#comment-16077982 ] Apache Spark commented on SPARK-21069: -- User 'ScrapCodes' has created a pull request

[jira] [Updated] (SPARK-21338) AggregatedDialect doesn't override isCascadingTruncateTable() method

2017-07-07 Thread Ostap Gonchar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ostap Gonchar updated SPARK-21338: -- Description: org.apache.spark.sql.jdbc.JdbcDialect's method: *def isCascadingTruncateTable(): O

[jira] [Commented] (SPARK-21328) Corpus creation using sparkR for textmining

2017-07-07 Thread Joe (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16077975#comment-16077975 ] Joe commented on SPARK-21328: - Hi Hyukjin Kwon, Could you please provide your suggesions?

[jira] [Created] (SPARK-21338) AggregatedDialect doesn't override isCascadingTruncateTable() method

2017-07-07 Thread Ostap Gonchar (JIRA)
Ostap Gonchar created SPARK-21338: - Summary: AggregatedDialect doesn't override isCascadingTruncateTable() method Key: SPARK-21338 URL: https://issues.apache.org/jira/browse/SPARK-21338 Project: Spark

[jira] [Commented] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

2017-07-07 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16077952#comment-16077952 ] Kazuaki Ishizaki commented on SPARK-21337: -- In the master branch, I cannot see a

[jira] [Resolved] (SPARK-21285) VectorAssembler should report the column name when data type used is not supported

2017-07-07 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang resolved SPARK-21285. - Resolution: Fixed Fix Version/s: 2.3.0 > VectorAssembler should report the column name whe

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

2017-07-07 Thread fengchaoge (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Description: when there are large 'case when ' expressions in spark sql,the CodeGenerator failed to

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

2017-07-07 Thread fengchaoge (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Description: when there are large 'case when ' expressions in spark sql,the CodeGenerator failed to

[jira] [Commented] (SPARK-21063) Spark return an empty result from remote hadoop cluster

2017-07-07 Thread YANGFEIRAN (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16077812#comment-16077812 ] YANGFEIRAN commented on SPARK-21063: https://stackoverflow.com/q/44434304/6547077 als

[jira] [Resolved] (SPARK-21313) ConsoleSink's string representation

2017-07-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21313. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18539 [https://github.co

[jira] [Assigned] (SPARK-21313) ConsoleSink's string representation

2017-07-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-21313: - Assignee: Jacek Laskowski > ConsoleSink's string representation > --

[jira] [Commented] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

2017-07-07 Thread fengchaoge (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16077730#comment-16077730 ] fengchaoge commented on SPARK-21337: !http://example.com/image.png! > SQL which has

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

2017-07-07 Thread fengchaoge (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Description: when there are large 'case when ' expressions in spark sql,the CodeGenerator failed to

[jira] [Commented] (SPARK-19680) Offsets out of range with no configured reset policy for partitions

2017-07-07 Thread Simon Kitching (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16077726#comment-16077726 ] Simon Kitching commented on SPARK-19680: Cody: thanks very much for the feedback.

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

2017-07-07 Thread fengchaoge (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Description: java.util.concurrent.ExecutionException: java.lang.Exception: failed to compile: org.cod

[jira] [Created] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

2017-07-07 Thread fengchaoge (JIRA)
fengchaoge created SPARK-21337: -- Summary: SQL which has large ‘case when’ expressions may cause code generation beyond 64KB Key: SPARK-21337 URL: https://issues.apache.org/jira/browse/SPARK-21337 Project