[jira] [Updated] (SPARK-10384) Univariate statistics as UDAFs

2015-10-05 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10384: -- Description: It would be nice to define univariate statistics as UDAFs. This JIRA discusses

[jira] [Commented] (SPARK-10868) monotonicallyIncreasingId() supports offset for indexing

2015-10-05 Thread Martin Senne (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944137#comment-14944137 ] Martin Senne commented on SPARK-10868: -- I will do (and give my best!) Thx for offering this

[jira] [Updated] (SPARK-10384) Univariate statistics as UDAFs

2015-10-05 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10384: -- Description: It would be nice to define univariate statistics as UDAFs. This JIRA discusses

[jira] [Updated] (SPARK-10384) Univariate statistics as UDAFs

2015-10-05 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10384: -- Description: It would be nice to define univariate statistics as UDAFs. This JIRA discusses

[jira] [Assigned] (SPARK-10384) Univariate statistics as UDAFs

2015-10-05 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-10384: - Assignee: Xiangrui Meng (was: Burak Yavuz) > Univariate statistics as UDAFs >

[jira] [Assigned] (SPARK-10337) Views are broken

2015-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10337: Assignee: Wenchen Fan (was: Apache Spark) > Views are broken > > >

[jira] [Commented] (SPARK-10337) Views are broken

2015-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944259#comment-14944259 ] Apache Spark commented on SPARK-10337: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10337) Views are broken

2015-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10337: Assignee: Apache Spark (was: Wenchen Fan) > Views are broken > > >

[jira] [Reopened] (SPARK-10937) java.lang.NoSuchMethodError when instantiating sqlContext in spark-shell using hive 0.12.x, 0.13.x

2015-10-05 Thread Curtis Wilde (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Curtis Wilde reopened SPARK-10937: -- In spark-defaults.conf I've added: spark.sql.hive.metastore.version 0.12.0

[jira] [Issue Comment Deleted] (SPARK-10937) java.lang.NoSuchMethodError when instantiating sqlContext in spark-shell using hive 0.12.x, 0.13.x

2015-10-05 Thread Curtis Wilde (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Curtis Wilde updated SPARK-10937: - Comment: was deleted (was: In spark-defaults.conf I've added: spark.sql.hive.metastore.version

[jira] [Commented] (SPARK-10937) java.lang.NoSuchMethodError when instantiating sqlContext in spark-shell using hive 0.12.x, 0.13.x

2015-10-05 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944271#comment-14944271 ] Marcelo Vanzin commented on SPARK-10937: And I assume that you checked the classpath of the

[jira] [Created] (SPARK-10939) Misaligned data with RDD.zip after repartition

2015-10-05 Thread Dan Brown (JIRA)
Dan Brown created SPARK-10939: - Summary: Misaligned data with RDD.zip after repartition Key: SPARK-10939 URL: https://issues.apache.org/jira/browse/SPARK-10939 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-10685) Misaligned data with RDD.zip and DataFrame.withColumn after repartition

2015-10-05 Thread Dan Brown (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944282#comment-14944282 ] Dan Brown commented on SPARK-10685: --- [~davies] [~joshrosen] Ok, I've split out the

[jira] [Updated] (SPARK-10940) Too many open files Spark Shuffle

2015-10-05 Thread Sandeep Pal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Pal updated SPARK-10940: Description: Executing terasort by Spark-SQL on the data generated by teragen in hadoop. Data

[jira] [Created] (SPARK-10940) Too many open files Spark Shuffle

2015-10-05 Thread Sandeep Pal (JIRA)
Sandeep Pal created SPARK-10940: --- Summary: Too many open files Spark Shuffle Key: SPARK-10940 URL: https://issues.apache.org/jira/browse/SPARK-10940 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-10925) Exception when joining DataFrames

2015-10-05 Thread Jason C Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944303#comment-14944303 ] Jason C Lee commented on SPARK-10925: - I removed your 2nd step "apply an UDF on column "name"" and

[jira] [Issue Comment Deleted] (SPARK-10925) Exception when joining DataFrames

2015-10-05 Thread Jason C Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason C Lee updated SPARK-10925: Comment: was deleted (was: I removed your 2nd step "apply an UDF on column "name"" and was able to

[jira] [Commented] (SPARK-10560) Make StreamingLogisticRegressionWithSGD Python API equals with Scala one

2015-10-05 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944305#comment-14944305 ] Bryan Cutler commented on SPARK-10560: -- Hi [~yanboliang]], I just want to make sure I'm on the same

[jira] [Commented] (SPARK-10940) Too many open files Spark Shuffle

2015-10-05 Thread Sandeep Pal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944307#comment-14944307 ] Sandeep Pal commented on SPARK-10940: - I have already read similar issue as below but this does not

[jira] [Resolved] (SPARK-10934) hashCode of unsafe array may crush

2015-10-05 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-10934. Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8987

[jira] [Updated] (SPARK-10934) hashCode of unsafe array may crush

2015-10-05 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-10934: --- Fix Version/s: 1.5.2 > hashCode of unsafe array may crush > -- > >

[jira] [Updated] (SPARK-10934) hashCode of unsafe array may crush

2015-10-05 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-10934: --- Assignee: Wenchen Fan > hashCode of unsafe array may crush > -- > >

[jira] [Commented] (SPARK-9776) Another instance of Derby may have already booted the database

2015-10-05 Thread Alexander Pivovarov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944335#comment-14944335 ] Alexander Pivovarov commented on SPARK-9776: To reproduce the issue 1. start emr-4.1.0

[jira] [Updated] (SPARK-10384) Univariate statistics as UDAFs

2015-10-05 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10384: -- Description: It would be nice to define univariate statistics as UDAFs. This JIRA discusses

[jira] [Created] (SPARK-10936) UDAF "mode" for categorical variables

2015-10-05 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-10936: - Summary: UDAF "mode" for categorical variables Key: SPARK-10936 URL: https://issues.apache.org/jira/browse/SPARK-10936 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-10604) Univariate statistics as UDAFs: categorical stats

2015-10-05 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-10604. --- Resolution: Duplicate Marking this as duplicated in favor of concrete JIRAs. > Univariate

[jira] [Closed] (SPARK-10602) Univariate statistics as UDAFs: single-pass continuous stats

2015-10-05 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng closed SPARK-10602. - Resolution: Duplicate Marking this as duplicated in favor of concrete JIRAs. Please continue the

[jira] [Updated] (SPARK-10912) Improve Spark metrics executor.filesystem

2015-10-05 Thread Yongjia Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongjia Wang updated SPARK-10912: - Attachment: s3a_metrics.patch Adding s3a is fairly straightforward. I guess the reason it's not

[jira] [Comment Edited] (SPARK-10474) TungstenAggregation cannot acquire memory for pointer array after switching to sort-based

2015-10-05 Thread Hans van den Bogert (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944166#comment-14944166 ] Hans van den Bogert edited comment on SPARK-10474 at 10/5/15 10:36 PM:

[jira] [Commented] (SPARK-10384) Univariate statistics as UDAFs

2015-10-05 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944168#comment-14944168 ] Xiangrui Meng commented on SPARK-10384: --- I marked small umbrella JIRAs duplicated in favor of

[jira] [Updated] (SPARK-10937) java.lang.NoSuchMethodError when instantiating sqlContext in spark-shell using hive 0.12.x, 0.13.x

2015-10-05 Thread Curtis Wilde (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Curtis Wilde updated SPARK-10937: - Description: Running spark-shell with Hive 0.12.0 (HDP 2.0) causes an exception and sqlContext

[jira] [Commented] (SPARK-10937) java.lang.NoSuchMethodError when instantiating sqlContext in spark-shell using hive 0.12.x, 0.13.x

2015-10-05 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944211#comment-14944211 ] Marcelo Vanzin commented on SPARK-10937: Did you set {{spark.sql.hive.metastore.version}} to

[jira] [Commented] (SPARK-10937) java.lang.NoSuchMethodError when instantiating sqlContext in spark-shell using hive 0.12.x, 0.13.x

2015-10-05 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944215#comment-14944215 ] Marcelo Vanzin commented on SPARK-10937: Nevermind. The exception is coming from the execution

[jira] [Comment Edited] (SPARK-10937) java.lang.NoSuchMethodError when instantiating sqlContext in spark-shell using hive 0.12.x, 0.13.x

2015-10-05 Thread Curtis Wilde (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944218#comment-14944218 ] Curtis Wilde edited comment on SPARK-10937 at 10/5/15 11:13 PM: Yes, I

[jira] [Resolved] (SPARK-10937) java.lang.NoSuchMethodError when instantiating sqlContext in spark-shell using hive 0.12.x, 0.13.x

2015-10-05 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-10937. Resolution: Invalid Then that's your problem. You're causing the error by overriding the

[jira] [Assigned] (SPARK-10938) Remove typeId in columnar cache

2015-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10938: Assignee: Apache Spark (was: Davies Liu) > Remove typeId in columnar cache >

[jira] [Commented] (SPARK-10938) Remove typeId in columnar cache

2015-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944238#comment-14944238 ] Apache Spark commented on SPARK-10938: -- User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10938) Remove typeId in columnar cache

2015-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10938: Assignee: Davies Liu (was: Apache Spark) > Remove typeId in columnar cache >

[jira] [Commented] (SPARK-10877) Assertions fail straightforward DataFrame job due to word alignment

2015-10-05 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944150#comment-14944150 ] Matt Cheah commented on SPARK-10877: Does spark-submit enable assertions? I'm not sure how SBT passes

[jira] [Updated] (SPARK-10384) Univariate statistics as UDAFs

2015-10-05 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10384: -- Description: It would be nice to define univariate statistics as UDAFs. This JIRA discusses

[jira] [Comment Edited] (SPARK-10603) Univariate statistics as UDAFs: multi-pass continuous stats

2015-10-05 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944153#comment-14944153 ] Xiangrui Meng edited comment on SPARK-10603 at 10/5/15 10:32 PM: - Marking

[jira] [Closed] (SPARK-10603) Univariate statistics as UDAFs: multi-pass continuous stats

2015-10-05 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng closed SPARK-10603. - Resolution: Duplicate Marking this as duplicated in favor of concrete JIRAs. > Univariate

[jira] [Commented] (SPARK-10641) skewness and kurtosis support

2015-10-05 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944155#comment-14944155 ] Xiangrui Meng commented on SPARK-10641: --- Any updates? > skewness and kurtosis support >

[jira] [Commented] (SPARK-10474) TungstenAggregation cannot acquire memory for pointer array after switching to sort-based

2015-10-05 Thread Hans van den Bogert (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944166#comment-14944166 ] Hans van den Bogert commented on SPARK-10474: - Had this patch against tag 1.5.1:

[jira] [Commented] (SPARK-5575) Artificial neural networks for MLlib deep learning

2015-10-05 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944173#comment-14944173 ] Alexander Ulanov commented on SPARK-5575: - Weide, These are major features and some of them are

[jira] [Comment Edited] (SPARK-10474) TungstenAggregation cannot acquire memory for pointer array after switching to sort-based

2015-10-05 Thread Hans van den Bogert (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944166#comment-14944166 ] Hans van den Bogert edited comment on SPARK-10474 at 10/5/15 10:39 PM:

[jira] [Commented] (SPARK-8848) Write Parquet LISTs and MAPs conforming to Parquet format spec

2015-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944197#comment-14944197 ] Apache Spark commented on SPARK-8848: - User 'liancheng' has created a pull request for this issue:

[jira] [Commented] (SPARK-10937) java.lang.NoSuchMethodError when instantiating sqlContext in spark-shell using hive 0.12.x, 0.13.x

2015-10-05 Thread Curtis Wilde (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944213#comment-14944213 ] Curtis Wilde commented on SPARK-10937: -- Yes > java.lang.NoSuchMethodError when instantiating

[jira] [Comment Edited] (SPARK-10937) java.lang.NoSuchMethodError when instantiating sqlContext in spark-shell using hive 0.12.x, 0.13.x

2015-10-05 Thread Curtis Wilde (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944213#comment-14944213 ] Curtis Wilde edited comment on SPARK-10937 at 10/5/15 11:06 PM: Yes

[jira] [Created] (SPARK-10937) java.lang.NoSuchMethodError when instantiating sqlContext in spark-shell using hive 0.12.x, 0.13.x

2015-10-05 Thread Curtis Wilde (JIRA)
Curtis Wilde created SPARK-10937: Summary: java.lang.NoSuchMethodError when instantiating sqlContext in spark-shell using hive 0.12.x, 0.13.x Key: SPARK-10937 URL:

[jira] [Commented] (SPARK-10937) java.lang.NoSuchMethodError when instantiating sqlContext in spark-shell using hive 0.12.x, 0.13.x

2015-10-05 Thread Curtis Wilde (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944218#comment-14944218 ] Curtis Wilde commented on SPARK-10937: -- Yes, I added the following jars to the classpath:

[jira] [Assigned] (SPARK-9999) RDD-like API on top of Catalyst/DataFrame

2015-10-05 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reassigned SPARK-: --- Assignee: Michael Armbrust > RDD-like API on top of Catalyst/DataFrame >

[jira] [Updated] (SPARK-9999) RDD-like API on top of Catalyst/DataFrame

2015-10-05 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-: Description: The RDD API is very flexible, and as a result harder to optimize its execution

[jira] [Commented] (SPARK-10937) java.lang.NoSuchMethodError when instantiating sqlContext in spark-shell using hive 0.12.x, 0.13.x

2015-10-05 Thread Curtis Wilde (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944258#comment-14944258 ] Curtis Wilde commented on SPARK-10937: -- In spark-defaults.conf I've added:

[jira] [Commented] (SPARK-9239) HiveUDAF support for AggregateFunction2

2015-10-05 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944405#comment-14944405 ] Josh Rosen commented on SPARK-9239: --- Is this duplicated by SPARK-10765? > HiveUDAF support for

[jira] [Assigned] (SPARK-10941) .Refactor AggregateFunction2 and AlgebraicAggregate interfaces to improve code clarity

2015-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10941: Assignee: Apache Spark (was: Josh Rosen) > .Refactor AggregateFunction2 and

[jira] [Created] (SPARK-10941) .Refactor AggregateFunction2 and AlgebraicAggregate interfaces to improve code clarity

2015-10-05 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-10941: -- Summary: .Refactor AggregateFunction2 and AlgebraicAggregate interfaces to improve code clarity Key: SPARK-10941 URL: https://issues.apache.org/jira/browse/SPARK-10941

[jira] [Commented] (SPARK-10941) .Refactor AggregateFunction2 and AlgebraicAggregate interfaces to improve code clarity

2015-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944411#comment-14944411 ] Apache Spark commented on SPARK-10941: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10941) .Refactor AggregateFunction2 and AlgebraicAggregate interfaces to improve code clarity

2015-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10941: Assignee: Josh Rosen (was: Apache Spark) > .Refactor AggregateFunction2 and

[jira] [Resolved] (SPARK-10900) Add output operation events to StreamingListener

2015-10-05 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-10900. --- Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 1.6.0 > Add output

[jira] [Commented] (SPARK-6723) Model import/export for ChiSqSelector

2015-10-05 Thread Jayant Shekhar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944415#comment-14944415 ] Jayant Shekhar commented on SPARK-6723: --- [~fliang] Hey Feyman, I made changes to the PR based on

[jira] [Comment Edited] (SPARK-6723) Model import/export for ChiSqSelector

2015-10-05 Thread Jayant Shekhar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944415#comment-14944415 ] Jayant Shekhar edited comment on SPARK-6723 at 10/6/15 2:25 AM: [~fliang]

[jira] [Created] (SPARK-10942) Not all cached RDDs are unpersisted

2015-10-05 Thread Nick Pritchard (JIRA)
Nick Pritchard created SPARK-10942: -- Summary: Not all cached RDDs are unpersisted Key: SPARK-10942 URL: https://issues.apache.org/jira/browse/SPARK-10942 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-10534) ORDER BY clause allows only columns that are present in SELECT statement

2015-10-05 Thread Dilip Biswal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944462#comment-14944462 ] Dilip Biswal commented on SPARK-10534: -- I would like to work on this. > ORDER BY clause allows only

[jira] [Commented] (SPARK-10942) Not all cached RDDs are unpersisted

2015-10-05 Thread Rekha Joshi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944465#comment-14944465 ] Rekha Joshi commented on SPARK-10942: - [~pnpritchard] hi.did you try rdd.unpersist()? > Not all

[jira] [Comment Edited] (SPARK-10942) Not all cached RDDs are unpersisted

2015-10-05 Thread Rekha Joshi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944465#comment-14944465 ] Rekha Joshi edited comment on SPARK-10942 at 10/6/15 4:04 AM: --

[jira] [Commented] (SPARK-10942) Not all cached RDDs are unpersisted

2015-10-05 Thread Nick Pritchard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944477#comment-14944477 ] Nick Pritchard commented on SPARK-10942: [~rekhajoshm] Yes, but calling {{rdd2.unpersist()}}

[jira] [Updated] (SPARK-10919) Assosiation rules class should return the support of each rule

2015-10-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10919: -- Component/s: MLlib > Assosiation rules class should return the support of each rule >

[jira] [Comment Edited] (SPARK-10925) Exception when joining DataFrames

2015-10-05 Thread Alexis Seigneurin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943318#comment-14943318 ] Alexis Seigneurin edited comment on SPARK-10925 at 10/5/15 12:44 PM: -

[jira] [Commented] (SPARK-7135) Expression for monotonically increasing IDs

2015-10-05 Thread Martin Senne (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943151#comment-14943151 ] Martin Senne commented on SPARK-7135: - @[~rxin]: Please see my description in SPARK-10868 . >

[jira] [Updated] (SPARK-10925) Exception when joining DataFrames

2015-10-05 Thread Alexis Seigneurin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexis Seigneurin updated SPARK-10925: -- Attachment: (was: Photo 05-10-2015 14 31 16.jpg) > Exception when joining

[jira] [Updated] (SPARK-10925) Exception when joining DataFrames

2015-10-05 Thread Alexis Seigneurin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexis Seigneurin updated SPARK-10925: -- Attachment: Photo 05-10-2015 14 31 16.jpg > Exception when joining DataFrames >

[jira] [Updated] (SPARK-10868) monotonicallyIncreasingId() supports offset for indexing

2015-10-05 Thread Martin Senne (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Senne updated SPARK-10868: - Description: With SPARK-7135 and https://github.com/apache/spark/pull/5709

[jira] [Updated] (SPARK-10925) Exception when joining DataFrames

2015-10-05 Thread Alexis Seigneurin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexis Seigneurin updated SPARK-10925: -- Attachment: TestCase2.scala > Exception when joining DataFrames >

[jira] [Created] (SPARK-10925) Exception when joining DataFrames

2015-10-05 Thread Alexis Seigneurin (JIRA)
Alexis Seigneurin created SPARK-10925: - Summary: Exception when joining DataFrames Key: SPARK-10925 URL: https://issues.apache.org/jira/browse/SPARK-10925 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-10925) Exception when joining DataFrames

2015-10-05 Thread Alexis Seigneurin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexis Seigneurin updated SPARK-10925: -- Attachment: Photo 05-10-2015 14 31 16.jpg The workflow. > Exception when joining

[jira] [Comment Edited] (SPARK-6404) Call broadcast() in each interval for spark streaming programs.

2015-10-05 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-6404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943143#comment-14943143 ] Kåre Blakstad edited comment on SPARK-6404 at 10/5/15 10:41 AM: I do

[jira] [Comment Edited] (SPARK-6404) Call broadcast() in each interval for spark streaming programs.

2015-10-05 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-6404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943143#comment-14943143 ] Kåre Blakstad edited comment on SPARK-6404 at 10/5/15 10:42 AM:

[jira] [Updated] (SPARK-10868) monotonicallyIncreasingId() supports offset for indexing

2015-10-05 Thread Martin Senne (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Senne updated SPARK-10868: - Description: With SPARK-7135 and https://github.com/apache/spark/pull/5709

[jira] [Commented] (SPARK-10617) Leap year miscalculated in sql query

2015-10-05 Thread Alex Rovner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943300#comment-14943300 ] Alex Rovner commented on SPARK-10617: - Good catch! Looks like there is a bug in

[jira] [Commented] (SPARK-10923) Spark handling parallel requests

2015-10-05 Thread Tarek Abouzeid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943342#comment-14943342 ] Tarek Abouzeid commented on SPARK-10923: oh , sorry , thanks > Spark handling parallel requests

[jira] [Commented] (SPARK-10797) RDD's coalesce should not write out the temporary key

2015-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943369#comment-14943369 ] Apache Spark commented on SPARK-10797: -- User 'ehnalis' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10797) RDD's coalesce should not write out the temporary key

2015-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10797: Assignee: Apache Spark > RDD's coalesce should not write out the temporary key >

[jira] [Assigned] (SPARK-10797) RDD's coalesce should not write out the temporary key

2015-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10797: Assignee: (was: Apache Spark) > RDD's coalesce should not write out the temporary key

[jira] [Commented] (SPARK-5783) Include filename, line number in eventlog-parsing error message

2015-10-05 Thread arnab chatterjee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943417#comment-14943417 ] arnab chatterjee commented on SPARK-5783: - Similar issue here on spark 1.3.1. Any update on this

[jira] [Commented] (SPARK-5783) Include filename, line number in eventlog-parsing error message

2015-10-05 Thread Michael England (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943426#comment-14943426 ] Michael England commented on SPARK-5783: Interesting, thanks Sean. > Include filename, line

[jira] [Commented] (SPARK-10797) RDD's coalesce should not write out the temporary key

2015-10-05 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943366#comment-14943366 ] Zoltán Zvara commented on SPARK-10797: -- I think we have the following options to solve this problem:

[jira] [Comment Edited] (SPARK-10792) Spark + YARN – executor is not re-created

2015-10-05 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943405#comment-14943405 ] Daniel Darabos edited comment on SPARK-10792 at 10/5/15 2:17 PM: - I work

[jira] [Commented] (SPARK-5783) Include filename, line number in eventlog-parsing error message

2015-10-05 Thread Michael England (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943419#comment-14943419 ] Michael England commented on SPARK-5783: Similar issue here on spark 1.3.1. Any update on this

[jira] [Issue Comment Deleted] (SPARK-5783) Include filename, line number in eventlog-parsing error message

2015-10-05 Thread arnab chatterjee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] arnab chatterjee updated SPARK-5783: Comment: was deleted (was: Similar issue here on spark 1.3.1. Any update on this pull

[jira] [Updated] (SPARK-10792) Spark + YARN – executor is not re-created

2015-10-05 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos updated SPARK-10792: --- Priority: Critical (was: Minor) Component/s: (was: Streaming) Summary:

[jira] [Commented] (SPARK-10797) RDD's coalesce should not write out the temporary key

2015-10-05 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943411#comment-14943411 ] Zoltán Zvara commented on SPARK-10797: -- On a simple repartitioning of {{(Int, String)}} types I've

[jira] [Commented] (SPARK-10792) Spark streaming + YARN – executor is not re-created on machine restart

2015-10-05 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943405#comment-14943405 ] Daniel Darabos commented on SPARK-10792: I work with Andras and can now substantiate his account

[jira] [Commented] (SPARK-10792) Spark + YARN – executor is not re-created

2015-10-05 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943407#comment-14943407 ] Daniel Darabos commented on SPARK-10792: I've edited the priority, the title, and the component

[jira] [Commented] (SPARK-6823) Add a model.matrix like capability to DataFrames (modelDataFrame)

2015-10-05 Thread Deborah Siegel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943410#comment-14943410 ] Deborah Siegel commented on SPARK-6823: --- does SPARK-9681 address RFormula supporting identity? eg.

[jira] [Resolved] (SPARK-5783) Include filename, line number in eventlog-parsing error message

2015-10-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5783. -- Resolution: Fixed Assignee: Ryan Williams Fix Version/s: 1.4.0 1.3.2

[jira] [Issue Comment Deleted] (SPARK-10617) Leap year miscalculated in sql query

2015-10-05 Thread Alex Rovner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rovner updated SPARK-10617: Comment: was deleted (was: https://github.com/apache/spark/pull/8980) > Leap year miscalculated

[jira] [Created] (SPARK-10927) Spark history uses the application name instead of the ID

2015-10-05 Thread JIRA
Jean-Baptiste Onofré created SPARK-10927: Summary: Spark history uses the application name instead of the ID Key: SPARK-10927 URL: https://issues.apache.org/jira/browse/SPARK-10927 Project:

[jira] [Commented] (SPARK-10847) Pyspark - DataFrame - Optional Metadata with `None` triggers cryptic failure

2015-10-05 Thread Jason C Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943670#comment-14943670 ] Jason C Lee commented on SPARK-10847: - You're welcome! > Pyspark - DataFrame - Optional Metadata

[jira] [Assigned] (SPARK-10926) Refactor ContextCleaner to allow weak reference cleaning to be done outside of the driver

2015-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10926: Assignee: Apache Spark > Refactor ContextCleaner to allow weak reference cleaning to be

<    1   2   3   >