[jira] [Updated] (SPARK-16205) dict -> StructType conversion is undocumented

2016-06-24 Thread Max Moroz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Moroz updated SPARK-16205: -- Description: According to the docs, StructType is equivalent only to python list and tuple. I accident

[jira] [Updated] (SPARK-16205) dict -> StructType conversion is undocumented

2016-06-24 Thread Max Moroz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Moroz updated SPARK-16205: -- Description: According to the docs, StructType is equivalent only to python list and tuple. I accident

[jira] [Created] (SPARK-16205) dict -> StructType conversion is undocumented

2016-06-24 Thread Max Moroz (JIRA)
Max Moroz created SPARK-16205: - Summary: dict -> StructType conversion is undocumented Key: SPARK-16205 URL: https://issues.apache.org/jira/browse/SPARK-16205 Project: Spark Issue Type: Documenta

[jira] [Created] (SPARK-16204) Row() interfact

2016-06-24 Thread Max Moroz (JIRA)
Max Moroz created SPARK-16204: - Summary: Row() interfact Key: SPARK-16204 URL: https://issues.apache.org/jira/browse/SPARK-16204 Project: Spark Issue Type: Improvement Components: PySpa

[jira] [Created] (SPARK-16203) regexp_extract to return an ArrayType(StringType())

2016-06-24 Thread Max Moroz (JIRA)
Max Moroz created SPARK-16203: - Summary: regexp_extract to return an ArrayType(StringType()) Key: SPARK-16203 URL: https://issues.apache.org/jira/browse/SPARK-16203 Project: Spark Issue Type: Imp

[jira] [Assigned] (SPARK-16202) Misleading Description of CreatableRelationProvider's createRelation

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16202: Assignee: (was: Apache Spark) > Misleading Description of CreatableRelationProvider's

[jira] [Assigned] (SPARK-16202) Misleading Description of CreatableRelationProvider's createRelation

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16202: Assignee: Apache Spark > Misleading Description of CreatableRelationProvider's createRelat

[jira] [Commented] (SPARK-16202) Misleading Description of CreatableRelationProvider's createRelation

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349093#comment-15349093 ] Apache Spark commented on SPARK-16202: -- User 'gatorsmile' has created a pull request

[jira] [Created] (SPARK-16202) Misleading Description of CreatableRelationProvider's createRelation

2016-06-24 Thread Xiao Li (JIRA)
Xiao Li created SPARK-16202: --- Summary: Misleading Description of CreatableRelationProvider's createRelation Key: SPARK-16202 URL: https://issues.apache.org/jira/browse/SPARK-16202 Project: Spark I

[jira] [Commented] (SPARK-16183) Large Spark SQL commands cause StackOverflowError in parser when using sqlContext.sql

2016-06-24 Thread Matthew Porter (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349084#comment-15349084 ] Matthew Porter commented on SPARK-16183: Much of the length is due to using colum

[jira] [Commented] (SPARK-16183) Large Spark SQL commands cause StackOverflowError in parser when using sqlContext.sql

2016-06-24 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349078#comment-15349078 ] Dongjoon Hyun commented on SPARK-16183: --- Hi, [~hvanhovell]. Does Spark have SQL Que

[jira] [Updated] (SPARK-16173) Can't join describe() of DataFrame in Scala 2.10

2016-06-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16173: --- Fix Version/s: 1.5.3 > Can't join describe() of DataFrame in Scala 2.10 > ---

[jira] [Updated] (SPARK-16173) Can't join describe() of DataFrame in Scala 2.10

2016-06-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16173: --- Fix Version/s: 2.0.1 > Can't join describe() of DataFrame in Scala 2.10 > ---

[jira] [Updated] (SPARK-16173) Can't join describe() of DataFrame in Scala 2.10

2016-06-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16173: --- Assignee: Dongjoon Hyun > Can't join describe() of DataFrame in Scala 2.10 >

[jira] [Resolved] (SPARK-16173) Can't join describe() of DataFrame in Scala 2.10

2016-06-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-16173. Resolution: Fixed Fix Version/s: 1.6.2 Issue resolved by pull request 13902 [https://github.

[jira] [Commented] (SPARK-16183) Large Spark SQL commands cause StackOverflowError in parser when using sqlContext.sql

2016-06-24 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349069#comment-15349069 ] Herman van Hovell commented on SPARK-16183: --- In Spark 1.6 the HiveContext uses

[jira] [Commented] (SPARK-16183) Large Spark SQL commands cause StackOverflowError in parser when using sqlContext.sql

2016-06-24 Thread Matthew Porter (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349057#comment-15349057 ] Matthew Porter commented on SPARK-16183: We use AWS EMR, so I was planning on wai

[jira] [Resolved] (SPARK-16192) Improve the type check of CollectSet in CheckAnalysis

2016-06-24 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-16192. --- Resolution: Resolved Assignee: Takeshi Yamamuro Fix Version/s

[jira] [Updated] (SPARK-16192) Improve the type check of CollectSet in CheckAnalysis

2016-06-24 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-16192: -- Affects Version/s: (was: 1.6.1) 2.0.0 > Improve the type che

[jira] [Created] (SPARK-16201) Expose information schema

2016-06-24 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-16201: - Summary: Expose information schema Key: SPARK-16201 URL: https://issues.apache.org/jira/browse/SPARK-16201 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-16173) Can't join describe() of DataFrame in Scala 2.10

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349024#comment-15349024 ] Apache Spark commented on SPARK-16173: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Commented] (SPARK-16200) Rename AggregateFunction#supportsPartial

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349004#comment-15349004 ] Apache Spark commented on SPARK-16200: -- User 'maropu' has created a pull request for

[jira] [Assigned] (SPARK-16200) Rename AggregateFunction#supportsPartial

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16200: Assignee: (was: Apache Spark) > Rename AggregateFunction#supportsPartial > ---

[jira] [Assigned] (SPARK-16200) Rename AggregateFunction#supportsPartial

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16200: Assignee: Apache Spark > Rename AggregateFunction#supportsPartial > --

[jira] [Created] (SPARK-16200) Rename AggregateFunction#supportsPartial

2016-06-24 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-16200: Summary: Rename AggregateFunction#supportsPartial Key: SPARK-16200 URL: https://issues.apache.org/jira/browse/SPARK-16200 Project: Spark Issue Type:

[jira] [Commented] (SPARK-16183) Large Spark SQL commands cause StackOverflowError in parser when using sqlContext.sql

2016-06-24 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348986#comment-15348986 ] Herman van Hovell commented on SPARK-16183: --- We have overhauled the parser in S

[jira] [Resolved] (SPARK-16195) Allow users to specify empty over clause in window expressions through dataset API

2016-06-24 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-16195. --- Resolution: Resolved Assignee: Dilip Biswal > Allow users to specify empty over

[jira] [Resolved] (SPARK-16186) Support partition batch pruning with `IN` predicate in InMemoryTableScanExec

2016-06-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-16186. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 13887 [https://github.

[jira] [Updated] (SPARK-16186) Support partition batch pruning with `IN` predicate in InMemoryTableScanExec

2016-06-24 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16186: -- Description: One of the most frequent usage patterns for Spark SQL is using **cached tables**.

[jira] [Assigned] (SPARK-16199) Add a method to list the referenced columns in data source Filter

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16199: Assignee: Apache Spark (was: Reynold Xin) > Add a method to list the referenced columns i

[jira] [Assigned] (SPARK-16199) Add a method to list the referenced columns in data source Filter

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16199: Assignee: Reynold Xin (was: Apache Spark) > Add a method to list the referenced columns i

[jira] [Commented] (SPARK-16199) Add a method to list the referenced columns in data source Filter

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348820#comment-15348820 ] Apache Spark commented on SPARK-16199: -- User 'rxin' has created a pull request for t

[jira] [Created] (SPARK-16199) Add a method to list the referenced columns in data source Filter

2016-06-24 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-16199: --- Summary: Add a method to list the referenced columns in data source Filter Key: SPARK-16199 URL: https://issues.apache.org/jira/browse/SPARK-16199 Project: Spark

[jira] [Updated] (SPARK-16179) UDF explosion yielding empty dataframe fails

2016-06-24 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16179: Fix Version/s: (was: 2.0.0) 2.0.1 > UDF explosion yielding empty dataframe f

[jira] [Resolved] (SPARK-16179) UDF explosion yielding empty dataframe fails

2016-06-24 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-16179. - Resolution: Fixed Fix Version/s: 2.0.0 > UDF explosion yielding empty dataframe fails > --

[jira] [Created] (SPARK-16198) Change the access level of the predict method in spark.ml.Predictor to public

2016-06-24 Thread Hussein Hazimeh (JIRA)
Hussein Hazimeh created SPARK-16198: --- Summary: Change the access level of the predict method in spark.ml.Predictor to public Key: SPARK-16198 URL: https://issues.apache.org/jira/browse/SPARK-16198 P

[jira] [Commented] (SPARK-16183) Large Spark SQL commands cause StackOverflowError in parser when using sqlContext.sql

2016-06-24 Thread Matthew Porter (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348726#comment-15348726 ] Matthew Porter commented on SPARK-16183: The query has a bit of proprietary infor

[jira] [Updated] (SPARK-16195) Allow users to specify empty over clause in window expressions through dataset API

2016-06-24 Thread Dilip Biswal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dilip Biswal updated SPARK-16195: - Description: In SQL, its allowed to specify an empty OVER clause in the window expression. {code

[jira] [Assigned] (SPARK-16173) Can't join describe() of DataFrame in Scala 2.10

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16173: Assignee: Apache Spark > Can't join describe() of DataFrame in Scala 2.10 > --

[jira] [Commented] (SPARK-16173) Can't join describe() of DataFrame in Scala 2.10

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348713#comment-15348713 ] Apache Spark commented on SPARK-16173: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Assigned] (SPARK-16173) Can't join describe() of DataFrame in Scala 2.10

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16173: Assignee: (was: Apache Spark) > Can't join describe() of DataFrame in Scala 2.10 > ---

[jira] [Resolved] (SPARK-16077) Python UDF may fail because of six

2016-06-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-16077. Resolution: Fixed Fix Version/s: 2.0.1 1.6.3 Issue resolved by pull reque

[jira] [Assigned] (SPARK-16077) Python UDF may fail because of six

2016-06-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-16077: -- Assignee: Davies Liu > Python UDF may fail because of six > --

[jira] [Assigned] (SPARK-16196) Optimize in-memory scan performance using ColumnarBatches

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16196: Assignee: Andrew Or (was: Apache Spark) > Optimize in-memory scan performance using Colum

[jira] [Commented] (SPARK-16196) Optimize in-memory scan performance using ColumnarBatches

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348703#comment-15348703 ] Apache Spark commented on SPARK-16196: -- User 'andrewor14' has created a pull request

[jira] [Assigned] (SPARK-16196) Optimize in-memory scan performance using ColumnarBatches

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16196: Assignee: Apache Spark (was: Andrew Or) > Optimize in-memory scan performance using Colum

[jira] [Commented] (SPARK-16173) Can't join describe() of DataFrame in Scala 2.10

2016-06-24 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348685#comment-15348685 ] Dongjoon Hyun commented on SPARK-16173: --- Hi, [~davies] and [~bomeng]. If you don't

[jira] [Commented] (SPARK-16173) Can't join describe() of DataFrame in Scala 2.10

2016-06-24 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348688#comment-15348688 ] Dongjoon Hyun commented on SPARK-16173: --- Of course, with Scala 2.10. > Can't join

[jira] [Assigned] (SPARK-16197) Cleanup PySpark status api and example

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16197: Assignee: Apache Spark > Cleanup PySpark status api and example >

[jira] [Assigned] (SPARK-16197) Cleanup PySpark status api and example

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16197: Assignee: (was: Apache Spark) > Cleanup PySpark status api and example > -

[jira] [Commented] (SPARK-16197) Cleanup PySpark status api and example

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348679#comment-15348679 ] Apache Spark commented on SPARK-16197: -- User 'BryanCutler' has created a pull reques

[jira] [Commented] (SPARK-16194) No way to dynamically set env vars on driver in cluster mode

2016-06-24 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348650#comment-15348650 ] Michael Gummelt commented on SPARK-16194: - Ah, yea, that's what I need. I'd like

[jira] [Created] (SPARK-16196) Optimize in-memory scan performance using ColumnarBatches

2016-06-24 Thread Andrew Or (JIRA)
Andrew Or created SPARK-16196: - Summary: Optimize in-memory scan performance using ColumnarBatches Key: SPARK-16196 URL: https://issues.apache.org/jira/browse/SPARK-16196 Project: Spark Issue Typ

[jira] [Created] (SPARK-16197) Cleanup PySpark status api and example

2016-06-24 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-16197: Summary: Cleanup PySpark status api and example Key: SPARK-16197 URL: https://issues.apache.org/jira/browse/SPARK-16197 Project: Spark Issue Type: Improvemen

[jira] [Commented] (SPARK-16194) No way to dynamically set env vars on driver in cluster mode

2016-06-24 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348645#comment-15348645 ] Michael Gummelt commented on SPARK-16194: - > Env variables are pretty much from o

[jira] [Commented] (SPARK-16194) No way to dynamically set env vars on driver in cluster mode

2016-06-24 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348643#comment-15348643 ] Marcelo Vanzin commented on SPARK-16194: For YARN you have {{spark.yarn.appMaster

[jira] [Assigned] (SPARK-16195) Allow users to specify empty over clause in window expressions through dataset API

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16195: Assignee: Apache Spark > Allow users to specify empty over clause in window expressions th

[jira] [Commented] (SPARK-16195) Allow users to specify empty over clause in window expressions through dataset API

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348628#comment-15348628 ] Apache Spark commented on SPARK-16195: -- User 'dilipbiswal' has created a pull reques

[jira] [Assigned] (SPARK-16195) Allow users to specify empty over clause in window expressions through dataset API

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16195: Assignee: (was: Apache Spark) > Allow users to specify empty over clause in window exp

[jira] [Created] (SPARK-16195) Allow users to specify empty over clause in window expressions through dataset API

2016-06-24 Thread Dilip Biswal (JIRA)
Dilip Biswal created SPARK-16195: Summary: Allow users to specify empty over clause in window expressions through dataset API Key: SPARK-16195 URL: https://issues.apache.org/jira/browse/SPARK-16195 Pr

[jira] [Commented] (SPARK-16193) Address flaky ExternalAppendOnlyMapSuite spilling tests

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348598#comment-15348598 ] Apache Spark commented on SPARK-16193: -- User 'srowen' has created a pull request for

[jira] [Updated] (SPARK-16194) No way to dynamically set env vars on driver in cluster mode

2016-06-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16194: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) Env variables are pretty much f

[jira] [Created] (SPARK-16194) No way to dynamically set env vars on driver in cluster mode

2016-06-24 Thread Michael Gummelt (JIRA)
Michael Gummelt created SPARK-16194: --- Summary: No way to dynamically set env vars on driver in cluster mode Key: SPARK-16194 URL: https://issues.apache.org/jira/browse/SPARK-16194 Project: Spark

[jira] [Assigned] (SPARK-16193) Address flaky ExternalAppendOnlyMapSuite spilling tests

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16193: Assignee: Apache Spark (was: Sean Owen) > Address flaky ExternalAppendOnlyMapSuite spilli

[jira] [Assigned] (SPARK-16193) Address flaky ExternalAppendOnlyMapSuite spilling tests

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16193: Assignee: Sean Owen (was: Apache Spark) > Address flaky ExternalAppendOnlyMapSuite spilli

[jira] [Created] (SPARK-16193) Address flaky ExternalAppendOnlyMapSuite spilling tests

2016-06-24 Thread Sean Owen (JIRA)
Sean Owen created SPARK-16193: - Summary: Address flaky ExternalAppendOnlyMapSuite spilling tests Key: SPARK-16193 URL: https://issues.apache.org/jira/browse/SPARK-16193 Project: Spark Issue Type:

[jira] [Updated] (SPARK-16112) R programming guide update for gapply and gapplyCollect

2016-06-24 Thread Kai Jiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Jiang updated SPARK-16112: -- Summary: R programming guide update for gapply and gapplyCollect (was: R programming guide update for

[jira] [Commented] (SPARK-16112) R programming guide update for gapply

2016-06-24 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348480#comment-15348480 ] Shivaram Venkataraman commented on SPARK-16112: --- Feel free to include both

[jira] [Commented] (SPARK-10073) Python withColumn for existing column name not consistent with scala

2016-06-24 Thread Russell Bradberry (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348468#comment-15348468 ] Russell Bradberry commented on SPARK-10073: --- with this, you added: {code}asser

[jira] [Commented] (SPARK-16164) CombineFilters should keep the ordering in the logical plan

2016-06-24 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348438#comment-15348438 ] Xiangrui Meng commented on SPARK-16164: --- [~lian cheng] See my last comment on GitHu

[jira] [Commented] (SPARK-16112) R programming guide update for gapply

2016-06-24 Thread Narine Kokhlikyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348430#comment-15348430 ] Narine Kokhlikyan commented on SPARK-16112: --- [~felixcheung], [~shivaram], [~sun

[jira] [Updated] (SPARK-15963) `TaskKilledException` is not correctly caught in `Executor.TaskRunner`

2016-06-24 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-15963: - Description: Before this change, if either of the following cases happened to a task , the task

[jira] [Resolved] (SPARK-15963) `TaskKilledException` is not correctly caught in `Executor.TaskRunner`

2016-06-24 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-15963. -- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 13685 [https://git

[jira] [Updated] (SPARK-15963) `TaskKilledException` is not correctly caught in `Executor.TaskRunner`

2016-06-24 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-15963: - Assignee: Liwei Lin > `TaskKilledException` is not correctly caught in `Executor.TaskRunner` > --

[jira] [Assigned] (SPARK-15254) Improve ML pipeline Cross Validation Scaladoc & PyDoc

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15254: Assignee: (was: Apache Spark) > Improve ML pipeline Cross Validation Scaladoc & PyDoc

[jira] [Assigned] (SPARK-15254) Improve ML pipeline Cross Validation Scaladoc & PyDoc

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15254: Assignee: Apache Spark > Improve ML pipeline Cross Validation Scaladoc & PyDoc > -

[jira] [Commented] (SPARK-15254) Improve ML pipeline Cross Validation Scaladoc & PyDoc

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348325#comment-15348325 ] Apache Spark commented on SPARK-15254: -- User 'krishnakalyan3' has created a pull req

[jira] [Commented] (SPARK-15955) Failed Spark application returns with exitcode equals to zero

2016-06-24 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348220#comment-15348220 ] Thomas Graves commented on SPARK-15955: --- there are some corner cases in spark 1.x t

[jira] [Assigned] (SPARK-14172) Hive table partition predicate not passed down correctly

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14172: Assignee: Apache Spark > Hive table partition predicate not passed down correctly > --

[jira] [Assigned] (SPARK-14172) Hive table partition predicate not passed down correctly

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14172: Assignee: (was: Apache Spark) > Hive table partition predicate not passed down correct

[jira] [Commented] (SPARK-14172) Hive table partition predicate not passed down correctly

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348184#comment-15348184 ] Apache Spark commented on SPARK-14172: -- User 'jiangxb1987' has created a pull reques

[jira] [Commented] (SPARK-16149) API consistency discussion: CountVectorizer.{minDF -> minDocFreq, minTF -> minTermFreq}

2016-06-24 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348173#comment-15348173 ] Nick Pentreath commented on SPARK-16149: I'd generally vote for: * if it's a new

[jira] [Resolved] (SPARK-15997) Audit ml.feature Update documentation for ml feature transformers

2016-06-24 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath resolved SPARK-15997. Resolution: Fixed Fix Version/s: 2.0.1 Issue resolved by pull request 13745 [https:/

[jira] [Updated] (SPARK-16188) Spark sql create a lot of small files

2016-06-24 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-16188: -- Priority: Major (was: Minor) > Spark sql create a lot of small files > ---

[jira] [Commented] (SPARK-16192) Improve the type check of CollectSet in CheckAnalysis

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348119#comment-15348119 ] Apache Spark commented on SPARK-16192: -- User 'maropu' has created a pull request for

[jira] [Assigned] (SPARK-16192) Improve the type check of CollectSet in CheckAnalysis

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16192: Assignee: (was: Apache Spark) > Improve the type check of CollectSet in CheckAnalysis

[jira] [Assigned] (SPARK-16192) Improve the type check of CollectSet in CheckAnalysis

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16192: Assignee: Apache Spark > Improve the type check of CollectSet in CheckAnalysis > -

[jira] [Assigned] (SPARK-6685) Use DSYRK to compute AtA in ALS

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6685: --- Assignee: Apache Spark > Use DSYRK to compute AtA in ALS > --- >

[jira] [Commented] (SPARK-6685) Use DSYRK to compute AtA in ALS

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348116#comment-15348116 ] Apache Spark commented on SPARK-6685: - User 'hqzizania' has created a pull request for

[jira] [Assigned] (SPARK-6685) Use DSYRK to compute AtA in ALS

2016-06-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6685: --- Assignee: (was: Apache Spark) > Use DSYRK to compute AtA in ALS > ---

[jira] [Created] (SPARK-16192) Improve the type check of CollectSet in CheckAnalysis

2016-06-24 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-16192: Summary: Improve the type check of CollectSet in CheckAnalysis Key: SPARK-16192 URL: https://issues.apache.org/jira/browse/SPARK-16192 Project: Spark

[jira] [Commented] (SPARK-15917) Define the number of executors in standalone mode with an easy-to-use property

2016-06-24 Thread Jonathan Taws (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348108#comment-15348108 ] Jonathan Taws commented on SPARK-15917: --- I made a change to the *StandaloneSchedule

[jira] [Created] (SPARK-16191) Code-Generated SpecificColumnarIterator fails for wide pivot with caching

2016-06-24 Thread Matthew Livesey (JIRA)
Matthew Livesey created SPARK-16191: --- Summary: Code-Generated SpecificColumnarIterator fails for wide pivot with caching Key: SPARK-16191 URL: https://issues.apache.org/jira/browse/SPARK-16191 Proje

[jira] [Updated] (SPARK-16190) Worker registration failed: Duplicate worker ID

2016-06-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16190: -- Priority: Minor (was: Critical) How did they stop and how were they restarted? > Worker registration

[jira] [Updated] (SPARK-16190) Worker registration failed: Duplicate worker ID

2016-06-24 Thread Thomas Huang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Huang updated SPARK-16190: - Attachment: spark-mqq-org.apache.spark.deploy.worker.Worker-1-slave2.out worker log of slave 2 >

[jira] [Updated] (SPARK-16190) Worker registration failed: Duplicate worker ID

2016-06-24 Thread Thomas Huang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Huang updated SPARK-16190: - Attachment: spark-mqq-org.apache.spark.deploy.worker.Worker-1-slave19.out worker log of slave 19

[jira] [Updated] (SPARK-16190) Worker registration failed: Duplicate worker ID

2016-06-24 Thread Thomas Huang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Huang updated SPARK-16190: - Attachment: spark-mqq-org.apache.spark.deploy.worker.Worker-1-slave8.out worker log of slave 8 >

[jira] [Updated] (SPARK-16190) Worker registration failed: Duplicate worker ID

2016-06-24 Thread Thomas Huang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Huang updated SPARK-16190: - Attachment: spark-mqq-org.apache.spark.deploy.worker.Worker-1-slave7.out worker log of slave7 >

[jira] [Created] (SPARK-16190) Worker registration failed: Duplicate worker ID

2016-06-24 Thread Thomas Huang (JIRA)
Thomas Huang created SPARK-16190: Summary: Worker registration failed: Duplicate worker ID Key: SPARK-16190 URL: https://issues.apache.org/jira/browse/SPARK-16190 Project: Spark Issue Type: B

[jira] [Updated] (SPARK-16188) Spark sql create a lot of small files

2016-06-24 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-16188: -- Affects Version/s: (was: 2.0.0) > Spark sql create a lot of small files > -

  1   2   >