[jira] [Created] (SPARK-21423) MODE average aggregate function.

2017-07-14 Thread Mathew Wicks (JIRA)
Mathew Wicks created SPARK-21423: Summary: MODE average aggregate function. Key: SPARK-21423 URL: https://issues.apache.org/jira/browse/SPARK-21423 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-13860) TPCDS query 39 returns wrong results compared to TPC official result set

2017-07-14 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088305#comment-16088305 ] Dongjoon Hyun edited comment on SPARK-13860 at 7/14/17 11:52 PM: - I

[jira] [Commented] (SPARK-13860) TPCDS query 39 returns wrong results compared to TPC official result set

2017-07-14 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088305#comment-16088305 ] Dongjoon Hyun commented on SPARK-13860: --- I confirmed that the behavior of Spark 2.1.1 and 2.2.0 is

[jira] [Updated] (SPARK-13860) TPCDS query 39 returns wrong results compared to TPC official result set

2017-07-14 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-13860: -- Affects Version/s: 2.2.0 2.1.1 > TPCDS query 39 returns wrong results

[jira] [Commented] (SPARK-19208) MultivariateOnlineSummarizer performance optimization

2017-07-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088259#comment-16088259 ] Apache Spark commented on SPARK-19208: -- User 'WeichenXu123' has created a pull request for this

[jira] [Commented] (SPARK-14523) Feature parity for Statistics ML with MLlib

2017-07-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088254#comment-16088254 ] Apache Spark commented on SPARK-14523: -- User 'WeichenXu123' has created a pull request for this

[jira] [Assigned] (SPARK-21422) Depend on Apache ORC 1.4.0

2017-07-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21422: Assignee: (was: Apache Spark) > Depend on Apache ORC 1.4.0 >

[jira] [Commented] (SPARK-21422) Depend on Apache ORC 1.4.0

2017-07-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088183#comment-16088183 ] Apache Spark commented on SPARK-21422: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-21422) Depend on Apache ORC 1.4.0

2017-07-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21422: Assignee: Apache Spark > Depend on Apache ORC 1.4.0 > -- > >

[jira] [Updated] (SPARK-21422) Depend on Apache ORC 1.4.0

2017-07-14 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-21422: -- Description: Like Parquet, this issue aims to depend on the latest Apache ORC 1.4 for Apache

[jira] [Commented] (SPARK-18061) Spark Thriftserver needs to create SPNego principal

2017-07-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088164#comment-16088164 ] Apache Spark commented on SPARK-18061: -- User 'jerryshao' has created a pull request for this issue:

[jira] [Resolved] (SPARK-21407) Support spnego for ThriftServer thrift/http auth

2017-07-14 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-21407. Resolution: Duplicate > Support spnego for ThriftServer thrift/http auth >

[jira] [Commented] (SPARK-21407) Support spnego for ThriftServer thrift/http auth

2017-07-14 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088156#comment-16088156 ] Marcelo Vanzin commented on SPARK-21407: No concern other than one should be marked a duplicate

[jira] [Commented] (SPARK-21407) Support spnego for ThriftServer thrift/http auth

2017-07-14 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088151#comment-16088151 ] Saisai Shao commented on SPARK-21407: - Sorry Marcelo, haven't noticed your comment. Yes it is the

[jira] [Updated] (SPARK-21422) Depend on Apache ORC 1.4.0

2017-07-14 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-21422: -- Description: Like Parquet, this issue aims to depend on the latest Apache ORC 1.4 for Apache

[jira] [Created] (SPARK-21422) Depend on Apache ORC 1.4.0

2017-07-14 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-21422: - Summary: Depend on Apache ORC 1.4.0 Key: SPARK-21422 URL: https://issues.apache.org/jira/browse/SPARK-21422 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-21421) Add the query id as a local property to allow source and sink using it

2017-07-14 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-21421. -- Resolution: Fixed Fix Version/s: 2.3.0 > Add the query id as a local property to allow

[jira] [Resolved] (SPARK-9825) Spark overwrites remote cluster "final" properties with local config

2017-07-14 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-9825. --- Resolution: Fixed Assignee: Marcelo Vanzin Fix Version/s: 2.3.0 > Spark

[jira] [Assigned] (SPARK-21408) Default RPC dispatcher thread pool size too large for small executors

2017-07-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21408: Assignee: (was: Apache Spark) > Default RPC dispatcher thread pool size too large for

[jira] [Assigned] (SPARK-21408) Default RPC dispatcher thread pool size too large for small executors

2017-07-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21408: Assignee: Apache Spark > Default RPC dispatcher thread pool size too large for small

[jira] [Commented] (SPARK-21408) Default RPC dispatcher thread pool size too large for small executors

2017-07-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088092#comment-16088092 ] Apache Spark commented on SPARK-21408: -- User 'vanzin' has created a pull request for this issue:

[jira] [Updated] (SPARK-21398) Data on Rest end point is not updating after first fetch

2017-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-21398: -- Target Version/s: (was: 2.1.0) > Data on Rest end point is not updating after first fetch >

[jira] [Commented] (SPARK-20086) issue with pyspark 2.1.0 window function

2017-07-14 Thread Matthew Scheifer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087862#comment-16087862 ] Matthew Scheifer commented on SPARK-20086: -- Is there a way I can work around this if I'm stuck

[jira] [Assigned] (SPARK-21421) Add the query id as a local property to allow source and sink using it

2017-07-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21421: Assignee: Shixiong Zhu (was: Apache Spark) > Add the query id as a local property to

[jira] [Assigned] (SPARK-21421) Add the query id as a local property to allow source and sink using it

2017-07-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21421: Assignee: Apache Spark (was: Shixiong Zhu) > Add the query id as a local property to

[jira] [Commented] (SPARK-21421) Add the query id as a local property to allow source and sink using it

2017-07-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087784#comment-16087784 ] Apache Spark commented on SPARK-21421: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Created] (SPARK-21421) Add the query id as a local property to allow source and sink using it

2017-07-14 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-21421: Summary: Add the query id as a local property to allow source and sink using it Key: SPARK-21421 URL: https://issues.apache.org/jira/browse/SPARK-21421 Project:

[jira] [Commented] (SPARK-15526) Shade JPMML

2017-07-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087759#comment-16087759 ] Apache Spark commented on SPARK-15526: -- User 'srowen' has created a pull request for this issue:

[jira] [Updated] (SPARK-21376) Token is not renewed in yarn client process in cluster mode

2017-07-14 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-21376: --- Fix Version/s: 2.2.1 > Token is not renewed in yarn client process in cluster mode >

[jira] [Commented] (SPARK-16617) Upgrade to Avro 1.8.x

2017-07-14 Thread Zeeshan Qureshi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087696#comment-16087696 ] Zeeshan Qureshi commented on SPARK-16617: - Would be great to have Avro 1.8+ in Spark to use

[jira] [Created] (SPARK-21420) Support array type code 'q' and 'Q'

2017-07-14 Thread Xiang Gao (JIRA)
Xiang Gao created SPARK-21420: - Summary: Support array type code 'q' and 'Q' Key: SPARK-21420 URL: https://issues.apache.org/jira/browse/SPARK-21420 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-20090) Add StructType.fieldNames to Python API

2017-07-14 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087666#comment-16087666 ] Joseph K. Bradley commented on SPARK-20090: --- Sorry for not seeing this. You're right about

[jira] [Updated] (SPARK-20236) Overwrite a partitioned data source table should only overwrite related partitions

2017-07-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-20236: Labels: releasenotes (was: ) > Overwrite a partitioned data source table should only overwrite

[jira] [Commented] (SPARK-19552) Upgrade Netty version to 4.1.8 final

2017-07-14 Thread Justin Pinkul (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087581#comment-16087581 ] Justin Pinkul commented on SPARK-19552: --- I also ran into a similar problem, the gRPC client library

[jira] [Commented] (SPARK-20937) Describe spark.sql.parquet.writeLegacyFormat property in Spark SQL, DataFrames and Datasets Guide

2017-07-14 Thread Tim Armstrong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087506#comment-16087506 ] Tim Armstrong commented on SPARK-20937: --- +1 too. The documentation should also be clear that the

[jira] [Created] (SPARK-21419) Support Mesos failover_timeout in driver (Mesos cluster mode)

2017-07-14 Thread Susan X. Huynh (JIRA)
Susan X. Huynh created SPARK-21419: -- Summary: Support Mesos failover_timeout in driver (Mesos cluster mode) Key: SPARK-21419 URL: https://issues.apache.org/jira/browse/SPARK-21419 Project: Spark

[jira] [Commented] (SPARK-21405) Add LBFGS solver for GeneralizedLinearRegression

2017-07-14 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087418#comment-16087418 ] Seth Hendrickson commented on SPARK-21405: -- Good point, Nick. Though conveniently the machinery

[jira] [Commented] (SPARK-21393) spark (pyspark) crashes unpredictably when using show() or toPandas()

2017-07-14 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087391#comment-16087391 ] Kazuaki Ishizaki commented on SPARK-21393: -- Not yet, however I created a patch not to cause

[jira] [Commented] (SPARK-21393) spark (pyspark) crashes unpredictably when using show() or toPandas()

2017-07-14 Thread Zahra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087386#comment-16087386 ] Zahra commented on SPARK-21393: --- So, now I should be able to run my code without any trouble? On Fri, Jul

[jira] [Commented] (SPARK-21393) spark (pyspark) crashes unpredictably when using show() or toPandas()

2017-07-14 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087379#comment-16087379 ] Hyukjin Kwon commented on SPARK-21393: -- Yes, I am able. It looked a duplicate of the issue I opened

[jira] [Commented] (SPARK-20070) Redact datasource explain output

2017-07-14 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087378#comment-16087378 ] Daniel Darabos commented on SPARK-20070: I think this change has broken our application. Your

[jira] [Created] (SPARK-21418) NoSuchElementException: None.get on DataFrame.rdd

2017-07-14 Thread Daniel Darabos (JIRA)
Daniel Darabos created SPARK-21418: -- Summary: NoSuchElementException: None.get on DataFrame.rdd Key: SPARK-21418 URL: https://issues.apache.org/jira/browse/SPARK-21418 Project: Spark Issue

[jira] [Commented] (SPARK-21393) spark (pyspark) crashes unpredictably when using show() or toPandas()

2017-07-14 Thread Zahra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087362#comment-16087362 ] Zahra commented on SPARK-21393: --- Hi, Were you able to reproduce the error? Thanks, Zahra On Fri, Jul 14,

[jira] [Commented] (SPARK-19280) Failed Recovery from checkpoint caused by the multi-threads issue in Spark Streaming scheduler

2017-07-14 Thread manas kar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087358#comment-16087358 ] manas kar commented on SPARK-19280: --- Hi, Is there any progress on this. It seems to affect migrating

[jira] [Commented] (SPARK-21388) GBT inherit from HasStepSize & LInearSVC/Binarizer from HasThreshold

2017-07-14 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087216#comment-16087216 ] Nick Pentreath commented on SPARK-21388: What is the benefit of doing this? > GBT inherit from

[jira] [Updated] (SPARK-21417) Detect transitive join conditions via expressions

2017-07-14 Thread Claus Stadler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Claus Stadler updated SPARK-21417: -- Description: _Disclaimer: The nature of this report is similar to that of

[jira] [Created] (SPARK-21417) Detect transitive join conditions via expressions

2017-07-14 Thread Claus Stadler (JIRA)
Claus Stadler created SPARK-21417: - Summary: Detect transitive join conditions via expressions Key: SPARK-21417 URL: https://issues.apache.org/jira/browse/SPARK-21417 Project: Spark Issue

[jira] [Commented] (SPARK-21416) Emit Raw Prediction Scores in DecisionTree Classification Model

2017-07-14 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087161#comment-16087161 ] Nick Pentreath commented on SPARK-21416: This is in Spark 2.2 > Emit Raw Prediction Scores in

[jira] [Comment Edited] (SPARK-21416) Emit Raw Prediction Scores in DecisionTree Classification Model

2017-07-14 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087161#comment-16087161 ] Nick Pentreath edited comment on SPARK-21416 at 7/14/17 10:37 AM: -- This

[jira] [Closed] (SPARK-21416) Emit Raw Prediction Scores in DecisionTree Classification Model

2017-07-14 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath closed SPARK-21416. -- Resolution: Not A Problem > Emit Raw Prediction Scores in DecisionTree Classification Model >

[jira] [Commented] (SPARK-21405) Add LBFGS solver for GeneralizedLinearRegression

2017-07-14 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087138#comment-16087138 ] Nick Pentreath commented on SPARK-21405: The only thing we "lose" is the ability to get some of

[jira] [Commented] (SPARK-21415) Triage scapegoat warnings, part 1

2017-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087103#comment-16087103 ] Sean Owen commented on SPARK-21415: --- I don't know, just because I don't know when or if anyone will

[jira] [Commented] (SPARK-21415) Triage scapegoat warnings, part 1

2017-07-14 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087097#comment-16087097 ] Kazuaki Ishizaki commented on SPARK-21415: -- Thank you. Is it better to create an umbrella JIRA

[jira] [Created] (SPARK-21416) Emit Raw Prediction Scores in DecisionTree Classification Model

2017-07-14 Thread Aman Rawat (JIRA)
Aman Rawat created SPARK-21416: -- Summary: Emit Raw Prediction Scores in DecisionTree Classification Model Key: SPARK-21416 URL: https://issues.apache.org/jira/browse/SPARK-21416 Project: Spark

[jira] [Assigned] (SPARK-21415) Triage scapegoat warnings, part 1

2017-07-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21415: Assignee: Apache Spark (was: Sean Owen) > Triage scapegoat warnings, part 1 >

[jira] [Assigned] (SPARK-21415) Triage scapegoat warnings, part 1

2017-07-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21415: Assignee: Sean Owen (was: Apache Spark) > Triage scapegoat warnings, part 1 >

[jira] [Commented] (SPARK-21415) Triage scapegoat warnings, part 1

2017-07-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087004#comment-16087004 ] Apache Spark commented on SPARK-21415: -- User 'srowen' has created a pull request for this issue:

[jira] [Created] (SPARK-21415) Triage scapegoat warnings, part 1

2017-07-14 Thread Sean Owen (JIRA)
Sean Owen created SPARK-21415: - Summary: Triage scapegoat warnings, part 1 Key: SPARK-21415 URL: https://issues.apache.org/jira/browse/SPARK-21415 Project: Spark Issue Type: Improvement

[jira] [Reopened] (SPARK-19293) Spark 2.1.x unstable with spark.speculation=true

2017-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-19293: --- > Spark 2.1.x unstable with spark.speculation=true > >

[jira] [Resolved] (SPARK-19293) Spark 2.1.x unstable with spark.speculation=true

2017-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19293. --- Resolution: Duplicate Fix Version/s: (was: 2.2.0) > Spark 2.1.x unstable with

[jira] [Commented] (SPARK-15423) why it is very slow to clean resources in Spark-2.0.0-preview

2017-07-14 Thread Kannu Gupta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086973#comment-16086973 ] Kannu Gupta commented on SPARK-15423: - [~srowen] I am facing the same issue with spark-2.1.0. Is the

[jira] [Commented] (SPARK-20712) [SQL] Spark can't read Hive table when column type has length greater than 4000 bytes

2017-07-14 Thread Elliot West (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086963#comment-16086963 ] Elliot West commented on SPARK-20712: - Related perhaps: HIVE-12274 > [SQL] Spark can't read Hive

[jira] [Commented] (SPARK-12606) Scala/Java compatibility issue Re: how to extend java transformer from Scala UnaryTransformer ?

2017-07-14 Thread Davis (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086961#comment-16086961 ] Davis commented on SPARK-12606: --- This happens on spark 2.2.0. In essence we can not a write a Transformer

[jira] [Commented] (SPARK-21413) Multiple projections with CASE WHEN fails to run generated codes

2017-07-14 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086931#comment-16086931 ] Hyukjin Kwon commented on SPARK-21413: -- Sure, thank you for double checking and working on this. >

[jira] [Commented] (SPARK-21413) Multiple projections with CASE WHEN fails to run generated codes

2017-07-14 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086927#comment-16086927 ] Kazuaki Ishizaki commented on SPARK-21413: -- Thank you for preparing a good repro. I can

[jira] [Comment Edited] (SPARK-21399) Difficult to debug code in spark-class

2017-07-14 Thread Mahesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086885#comment-16086885 ] Mahesh edited comment on SPARK-21399 at 7/14/17 6:30 AM: - Marcelo, Yes, I

[jira] [Comment Edited] (SPARK-21399) Difficult to debug code in spark-class

2017-07-14 Thread Mahesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086898#comment-16086898 ] Mahesh edited comment on SPARK-21399 at 7/14/17 6:26 AM: - Closing the issue based

[jira] [Comment Edited] (SPARK-21399) Difficult to debug code in spark-class

2017-07-14 Thread Mahesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086898#comment-16086898 ] Mahesh edited comment on SPARK-21399 at 7/14/17 6:25 AM: - Closing the issue based

[jira] [Commented] (SPARK-21414) Buffer in SlidingWindowFunctionFrame could be big though window is small

2017-07-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086921#comment-16086921 ] Apache Spark commented on SPARK-21414: -- User 'jinxing64' has created a pull request for this issue:

[jira] [Assigned] (SPARK-21414) Buffer in SlidingWindowFunctionFrame could be big though window is small

2017-07-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21414: Assignee: (was: Apache Spark) > Buffer in SlidingWindowFunctionFrame could be big

[jira] [Assigned] (SPARK-21414) Buffer in SlidingWindowFunctionFrame could be big though window is small

2017-07-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21414: Assignee: Apache Spark > Buffer in SlidingWindowFunctionFrame could be big though window

[jira] [Updated] (SPARK-20706) Spark-shell not overriding method/variable definition

2017-07-14 Thread Raphael Roth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raphael Roth updated SPARK-20706: - Attachment: screenshot-1.png > Spark-shell not overriding method/variable definition >

[jira] [Updated] (SPARK-20706) Spark-shell not overriding method/variable definition

2017-07-14 Thread Raphael Roth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raphael Roth updated SPARK-20706: - Affects Version/s: 2.2.0 2.1.1 > Spark-shell not overriding

[jira] [Updated] (SPARK-20706) Spark-shell not overriding method/variable definition

2017-07-14 Thread Raphael Roth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raphael Roth updated SPARK-20706: - Priority: Major (was: Minor) > Spark-shell not overriding method/variable definition >

[jira] [Updated] (SPARK-20706) Spark-shell not overriding method/variable definition

2017-07-14 Thread Raphael Roth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raphael Roth updated SPARK-20706: - Description: !screenshot-1.png!In the following example, the definition of myMethod is not

[jira] [Created] (SPARK-21414) Buffer in SlidingWindowFunctionFrame could be big though window is small

2017-07-14 Thread jin xing (JIRA)
jin xing created SPARK-21414: Summary: Buffer in SlidingWindowFunctionFrame could be big though window is small Key: SPARK-21414 URL: https://issues.apache.org/jira/browse/SPARK-21414 Project: Spark