[jira] [Assigned] (SPARK-21250) Add a url in the table of 'Running Executors' in worker page to visit job page

2017-07-02 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21250: --- Assignee: guoxiaolongzte > Add a url in the table of 'Running Executors' in worker page to

[jira] [Resolved] (SPARK-21250) Add a url in the table of 'Running Executors' in worker page to visit job page

2017-07-02 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21250. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18464

[jira] [Resolved] (SPARK-21282) Fix test failure in 2.0

2017-07-02 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21282. - Resolution: Fixed Fix Version/s: 2.0.3 Issue resolved by pull request 18506

[jira] [Commented] (SPARK-21198) SparkSession catalog is terribly slow

2017-07-02 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071931#comment-16071931 ] Liang-Chi Hsieh commented on SPARK-21198: - I'd close this for now. If you have further question,

[jira] [Closed] (SPARK-21198) SparkSession catalog is terribly slow

2017-07-02 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh closed SPARK-21198. --- Resolution: Won't Fix > SparkSession catalog is terribly slow >

[jira] [Commented] (SPARK-21277) Spark is invoking an incorrect serializer after UDAF completion

2017-07-02 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071929#comment-16071929 ] Liang-Chi Hsieh commented on SPARK-21277: - The call to {{InternalRow.getArray}} returns an

[jira] [Updated] (SPARK-20946) simplify the config setting logic in SparkSession.getOrCreate

2017-07-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-20946: -- Fix Version/s: (was: 2.2.0) > simplify the config setting logic in

[jira] [Reopened] (SPARK-20946) simplify the config setting logic in SparkSession.getOrCreate

2017-07-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-20946: --- Hi, All. I'll reopen this issue since this is reverted from master and branch-2.2. > simplify

[jira] [Comment Edited] (SPARK-21198) SparkSession catalog is terribly slow

2017-07-02 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071897#comment-16071897 ] Liang-Chi Hsieh edited comment on SPARK-21198 at 7/3/17 4:14 AM: - I mean

[jira] [Commented] (SPARK-21283) FileOutputStream should be created as append mode

2017-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071896#comment-16071896 ] Apache Spark commented on SPARK-21283: -- User '10110346' has created a pull request for this issue:

[jira] [Commented] (SPARK-21198) SparkSession catalog is terribly slow

2017-07-02 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071897#comment-16071897 ] Liang-Chi Hsieh commented on SPARK-21198: - I mean using {{show tables}} and {{show databases}}.

[jira] [Comment Edited] (SPARK-21198) SparkSession catalog is terribly slow

2017-07-02 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071897#comment-16071897 ] Liang-Chi Hsieh edited comment on SPARK-21198 at 7/3/17 4:13 AM: - I mean

[jira] [Assigned] (SPARK-21283) FileOutputStream should be created as append mode

2017-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21283: Assignee: (was: Apache Spark) > FileOutputStream should be created as append mode >

[jira] [Assigned] (SPARK-21283) FileOutputStream should be created as append mode

2017-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21283: Assignee: Apache Spark > FileOutputStream should be created as append mode >

[jira] [Created] (SPARK-21283) FileOutputStream should be created as append mode

2017-07-02 Thread liuxian (JIRA)
liuxian created SPARK-21283: --- Summary: FileOutputStream should be created as append mode Key: SPARK-21283 URL: https://issues.apache.org/jira/browse/SPARK-21283 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-21198) SparkSession catalog is terribly slow

2017-07-02 Thread Saif Addin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071890#comment-16071890 ] Saif Addin edited comment on SPARK-21198 at 7/3/17 3:41 AM: I thought

[jira] [Comment Edited] (SPARK-21198) SparkSession catalog is terribly slow

2017-07-02 Thread Saif Addin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071890#comment-16071890 ] Saif Addin edited comment on SPARK-21198 at 7/3/17 3:41 AM: I thought

[jira] [Commented] (SPARK-21198) SparkSession catalog is terribly slow

2017-07-02 Thread Saif Addin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071890#comment-16071890 ] Saif Addin commented on SPARK-21198: I thought sqlContext was going to be deprecated soon. If it

[jira] [Commented] (SPARK-21282) Fix test failure in 2.0

2017-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071880#comment-16071880 ] Apache Spark commented on SPARK-21282: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-21282) Fix test failure in 2.0

2017-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21282: Assignee: Apache Spark (was: Xiao Li) > Fix test failure in 2.0 >

[jira] [Assigned] (SPARK-21282) Fix test failure in 2.0

2017-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21282: Assignee: Xiao Li (was: Apache Spark) > Fix test failure in 2.0 >

[jira] [Created] (SPARK-21282) Fix test failure in 2.0

2017-07-02 Thread Xiao Li (JIRA)
Xiao Li created SPARK-21282: --- Summary: Fix test failure in 2.0 Key: SPARK-21282 URL: https://issues.apache.org/jira/browse/SPARK-21282 Project: Spark Issue Type: Bug Components: SQL

[jira] [Comment Edited] (SPARK-21198) SparkSession catalog is terribly slow

2017-07-02 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071876#comment-16071876 ] Liang-Chi Hsieh edited comment on SPARK-21198 at 7/3/17 3:09 AM: - Since

[jira] [Commented] (SPARK-21198) SparkSession catalog is terribly slow

2017-07-02 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071876#comment-16071876 ] Liang-Chi Hsieh commented on SPARK-21198: - Since you can use {{sql("show tables")}} to get the

[jira] [Comment Edited] (SPARK-21198) SparkSession catalog is terribly slow

2017-07-02 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071876#comment-16071876 ] Liang-Chi Hsieh edited comment on SPARK-21198 at 7/3/17 3:08 AM: - Since

[jira] [Comment Edited] (SPARK-21281) cannot create empty typed array column

2017-07-02 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071843#comment-16071843 ] Takeshi Yamamuro edited comment on SPARK-21281 at 7/3/17 1:48 AM: --

[jira] [Commented] (SPARK-21281) cannot create empty typed array column

2017-07-02 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071843#comment-16071843 ] Takeshi Yamamuro commented on SPARK-21281: -- Honestly, I'm not sure the query in the ticket is

[jira] [Comment Edited] (SPARK-21281) cannot create empty typed array column

2017-07-02 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071843#comment-16071843 ] Takeshi Yamamuro edited comment on SPARK-21281 at 7/3/17 1:47 AM: --

[jira] [Comment Edited] (SPARK-21281) cannot create empty typed array column

2017-07-02 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071843#comment-16071843 ] Takeshi Yamamuro edited comment on SPARK-21281 at 7/3/17 1:47 AM: --

[jira] [Commented] (SPARK-21281) cannot create empty typed array column

2017-07-02 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071841#comment-16071841 ] Takeshi Yamamuro commented on SPARK-21281: -- You want to do this? {code} scala>

[jira] [Updated] (SPARK-21281) cannot create empty typed array column

2017-07-02 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-21281: - Component/s: SQL > cannot create empty typed array column >

[jira] [Updated] (SPARK-21281) cannot create empty typed array column

2017-07-02 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-21281: - Component/s: (was: Spark Core) > cannot create empty typed array column >

[jira] [Assigned] (SPARK-18004) DataFrame filter Predicate push-down fails for Oracle Timestamp type columns

2017-07-02 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-18004: --- Assignee: Rui Zha > DataFrame filter Predicate push-down fails for Oracle Timestamp type columns >

[jira] [Resolved] (SPARK-18004) DataFrame filter Predicate push-down fails for Oracle Timestamp type columns

2017-07-02 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-18004. - Resolution: Fixed Fix Version/s: 2.3.0 > DataFrame filter Predicate push-down fails for Oracle

[jira] [Resolved] (SPARK-17071) Fetch Parquet schema within driver-side when there is single file to touch without another Spark job

2017-07-02 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-17071. -- Resolution: Won't Fix I am resolving this. Please refer the discussion in the PR. > Fetch

[jira] [Updated] (SPARK-21281) cannot create empty typed array column

2017-07-02 Thread Saif Addin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saif Addin updated SPARK-21281: --- Description: Hi all I am running this piece of code {code:java} val data =

[jira] [Updated] (SPARK-21281) cannot create empty typed array column

2017-07-02 Thread Saif Addin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saif Addin updated SPARK-21281: --- Summary: cannot create empty typed array column (was: Casting empty typed array column return

[jira] [Updated] (SPARK-21281) Casting empty typed array column return different results depending on the api used

2017-07-02 Thread Saif Addin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saif Addin updated SPARK-21281: --- Priority: Minor (was: Major) > Casting empty typed array column return different results depending

[jira] [Created] (SPARK-21281) Casting empty typed array column return different results depending on the api used

2017-07-02 Thread Saif Addin (JIRA)
Saif Addin created SPARK-21281: -- Summary: Casting empty typed array column return different results depending on the api used Key: SPARK-21281 URL: https://issues.apache.org/jira/browse/SPARK-21281

[jira] [Commented] (SPARK-21198) SparkSession catalog is terribly slow

2017-07-02 Thread Saif Addin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071759#comment-16071759 ] Saif Addin commented on SPARK-21198: [~viirya] any chance you could take a look? > SparkSession

[jira] [Commented] (SPARK-21280) org.apache.spark.util.sketch.BloomFilter not bean compliant

2017-07-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071752#comment-16071752 ] Sean Owen commented on SPARK-21280: --- There are plenty of classes that are public in the bytecode, and

[jira] [Comment Edited] (SPARK-21280) org.apache.spark.util.sketch.BloomFilter not bean compliant

2017-07-02 Thread Eran Moscovici (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071749#comment-16071749 ] Eran Moscovici edited comment on SPARK-21280 at 7/2/17 6:35 PM: There is

[jira] [Commented] (SPARK-21280) org.apache.spark.util.sketch.BloomFilter not bean compliant

2017-07-02 Thread Eran Moscovici (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071749#comment-16071749 ] Eran Moscovici commented on SPARK-21280: There is a self evident use-case for using the

[jira] [Updated] (SPARK-21280) org.apache.spark.util.sketch.BloomFilter not bean compliant

2017-07-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-21280: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) I don't think it is intended

[jira] [Created] (SPARK-21280) org.apache.spark.util.sketch.BloomFilter not bean compliant

2017-07-02 Thread Eran Moscovici (JIRA)
Eran Moscovici created SPARK-21280: -- Summary: org.apache.spark.util.sketch.BloomFilter not bean compliant Key: SPARK-21280 URL: https://issues.apache.org/jira/browse/SPARK-21280 Project: Spark

[jira] [Commented] (SPARK-21206) the window slice of Dstream is wrong

2017-07-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071709#comment-16071709 ] Sean Owen commented on SPARK-21206: --- I think you are interpreting the log message to mean something it

[jira] [Commented] (SPARK-21206) the window slice of Dstream is wrong

2017-07-02 Thread Fei Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071640#comment-16071640 ] Fei Shao commented on SPARK-21206: -- About the concept of oldRDDs and newRDDs,I agree with you. About

[jira] [Closed] (SPARK-21279) stream join hive text batch not supported

2017-07-02 Thread xianyao jiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xianyao jiang closed SPARK-21279. - Resolution: Duplicate same problem > stream join hive text batch not supported >

[jira] [Updated] (SPARK-21279) stream join hive text batch not supported

2017-07-02 Thread xianyao jiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xianyao jiang updated SPARK-21279: -- Affects Version/s: (was: 2.2.0) 2.1.1 > stream join hive text batch

[jira] [Created] (SPARK-21279) stream join hive text batch not supported

2017-07-02 Thread xianyao jiang (JIRA)
xianyao jiang created SPARK-21279: - Summary: stream join hive text batch not supported Key: SPARK-21279 URL: https://issues.apache.org/jira/browse/SPARK-21279 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-15526) Shade JPMML

2017-07-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071552#comment-16071552 ] Sean Owen commented on SPARK-15526: --- I do think we should shade in 2.3.0 here. Yes, shading is the

[jira] [Assigned] (SPARK-21271) UnsafeRow.hashCode assertion when sizeInBytes not multiple of 8

2017-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21271: Assignee: Apache Spark > UnsafeRow.hashCode assertion when sizeInBytes not multiple of 8

[jira] [Assigned] (SPARK-21271) UnsafeRow.hashCode assertion when sizeInBytes not multiple of 8

2017-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21271: Assignee: (was: Apache Spark) > UnsafeRow.hashCode assertion when sizeInBytes not

[jira] [Commented] (SPARK-21271) UnsafeRow.hashCode assertion when sizeInBytes not multiple of 8

2017-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071551#comment-16071551 ] Apache Spark commented on SPARK-21271: -- User 'kiszk' has created a pull request for this issue:

[jira] [Resolved] (SPARK-19852) StringIndexer.setHandleInvalid should have another option 'new': Python API and docs

2017-07-02 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang resolved SPARK-19852. - Resolution: Fixed Fix Version/s: 2.3.0 > StringIndexer.setHandleInvalid should have

[jira] [Resolved] (SPARK-21206) the window slice of Dstream is wrong

2017-07-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21206. --- Resolution: Invalid I still don't understand what you're trying to point out. The original doc

[jira] [Resolved] (SPARK-21260) Remove the unused OutputFakerExec

2017-07-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21260. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18473

[jira] [Assigned] (SPARK-21260) Remove the unused OutputFakerExec

2017-07-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-21260: - Assignee: Jiang Xingbo Priority: Trivial (was: Minor) > Remove the unused OutputFakerExec