[jira] [Commented] (SPARK-14176) Add processing time trigger

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212828#comment-15212828 ] Apache Spark commented on SPARK-14176: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-14176) Add processing time trigger

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14176: Assignee: Apache Spark (was: Shixiong Zhu) > Add processing time trigger >

[jira] [Assigned] (SPARK-14176) Add processing time trigger

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14176: Assignee: Shixiong Zhu (was: Apache Spark) > Add processing time trigger >

[jira] [Created] (SPARK-14176) Add processing time trigger

2016-03-25 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-14176: Summary: Add processing time trigger Key: SPARK-14176 URL: https://issues.apache.org/jira/browse/SPARK-14176 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-14175) Simplify whole stage codegen interface

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14175: Assignee: Apache Spark (was: Davies Liu) > Simplify whole stage codegen interface >

[jira] [Commented] (SPARK-14175) Simplify whole stage codegen interface

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212820#comment-15212820 ] Apache Spark commented on SPARK-14175: -- User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-14175) Simplify whole stage codegen interface

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14175: Assignee: Davies Liu (was: Apache Spark) > Simplify whole stage codegen interface >

[jira] [Created] (SPARK-14175) Simplify whole stage codegen interface

2016-03-25 Thread Davies Liu (JIRA)
Davies Liu created SPARK-14175: -- Summary: Simplify whole stage codegen interface Key: SPARK-14175 URL: https://issues.apache.org/jira/browse/SPARK-14175 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-14174) Accelerate KMeans via Mini-Batch EM

2016-03-25 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212776#comment-15212776 ] zhengruifeng commented on SPARK-14174: -- There is another sklean example for MiniBatch KMeans:

[jira] [Commented] (SPARK-14174) Accelerate KMeans via Mini-Batch EM

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212772#comment-15212772 ] Apache Spark commented on SPARK-14174: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-14174) Accelerate KMeans via Mini-Batch EM

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14174: Assignee: (was: Apache Spark) > Accelerate KMeans via Mini-Batch EM >

[jira] [Assigned] (SPARK-14174) Accelerate KMeans via Mini-Batch EM

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14174: Assignee: Apache Spark > Accelerate KMeans via Mini-Batch EM >

[jira] [Created] (SPARK-14174) Accelerate KMeans via Mini-Batch EM

2016-03-25 Thread zhengruifeng (JIRA)
zhengruifeng created SPARK-14174: Summary: Accelerate KMeans via Mini-Batch EM Key: SPARK-14174 URL: https://issues.apache.org/jira/browse/SPARK-14174 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-14109) HDFSMetadataLog throws AbstractFilesSystem exception with common schemes like s3n

2016-03-25 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-14109. --- Resolution: Fixed > HDFSMetadataLog throws AbstractFilesSystem exception with common schemes

[jira] [Commented] (SPARK-14139) Dataset loses nullability in operations with RowEncoder

2016-03-25 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212743#comment-15212743 ] koert kuipers commented on SPARK-14139: --- it is not clear to me if the goal should remain to derive

[jira] [Updated] (SPARK-14173) Ignoring config property “spark.executor.extraJavaOptions”

2016-03-25 Thread liyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyan updated SPARK-14173: -- Description: when i submit streaming application on *yarn cluster* mode, i can't find

[jira] [Updated] (SPARK-14173) Ignoring config property “spark.executor.extraJavaOptions”

2016-03-25 Thread liyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyan updated SPARK-14173: -- Description: when i submit streaming application on *yarn cluster* , i can't find

[jira] [Updated] (SPARK-14173) Ignoring config property “spark.executor.extraJavaOptions”

2016-03-25 Thread liyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyan updated SPARK-14173: -- Description: when i submit streaming application on *yarn cluster* , i can't find

[jira] [Commented] (SPARK-4743) Use SparkEnv.serializer instead of closureSerializer in aggregateByKey and foldByKey

2016-03-25 Thread Jack Franson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212726#comment-15212726 ] Jack Franson commented on SPARK-4743: - Hi, I'm still running into "Task not serializable" errors with

[jira] [Created] (SPARK-14173) Ignoring config property “spark.executor.extraJavaOptions”

2016-03-25 Thread liyan (JIRA)
liyan created SPARK-14173: - Summary: Ignoring config property “spark.executor.extraJavaOptions” Key: SPARK-14173 URL: https://issues.apache.org/jira/browse/SPARK-14173 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-14172) Hive table partition predicate not passed down correctly

2016-03-25 Thread Yingji Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingji Zhang updated SPARK-14172: - Description: When the hive sql contains nondeterministic fields, spark plan will not push down

[jira] [Updated] (SPARK-14171) UDAF aggregates argument object inspector not parsed correctly

2016-03-25 Thread Jianfeng Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jianfeng Hu updated SPARK-14171: Description: For example, when using percentile_approx and count distinct together, it raises an

[jira] [Updated] (SPARK-14171) UDAF aggregates argument object inspector not parsed correctly

2016-03-25 Thread Jianfeng Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jianfeng Hu updated SPARK-14171: Description: For example, when using percentile_approx and count distinct together, it raises an

[jira] [Created] (SPARK-14172) Hive table partition predicate not passed down correctly

2016-03-25 Thread Yingji Zhang (JIRA)
Yingji Zhang created SPARK-14172: Summary: Hive table partition predicate not passed down correctly Key: SPARK-14172 URL: https://issues.apache.org/jira/browse/SPARK-14172 Project: Spark

[jira] [Created] (SPARK-14171) UDAF aggregates argument object inspector not parsed correctly

2016-03-25 Thread Jianfeng Hu (JIRA)
Jianfeng Hu created SPARK-14171: --- Summary: UDAF aggregates argument object inspector not parsed correctly Key: SPARK-14171 URL: https://issues.apache.org/jira/browse/SPARK-14171 Project: Spark

[jira] [Commented] (SPARK-12436) If all values of a JSON field is null, JSON's inferSchema should return NullType instead of StringType

2016-03-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212671#comment-15212671 ] Reynold Xin commented on SPARK-12436: - I don't have everything page in, but why isn't an empty string

[jira] [Commented] (SPARK-1153) Generalize VertexId in GraphX so that UUIDs can be used as vertex IDs.

2016-03-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212670#comment-15212670 ] Reynold Xin commented on SPARK-1153: [~ntietz] changing this will very likely make performance regress

[jira] [Resolved] (SPARK-14073) Move streaming-flume back to Spark

2016-03-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-14073. - Resolution: Fixed Fix Version/s: 2.0.0 > Move streaming-flume back to Spark >

[jira] [Assigned] (SPARK-14170) Remove the PR template before pushing changes

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14170: Assignee: Apache Spark > Remove the PR template before pushing changes >

[jira] [Commented] (SPARK-14170) Remove the PR template before pushing changes

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212653#comment-15212653 ] Apache Spark commented on SPARK-14170: -- User 'vanzin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-14170) Remove the PR template before pushing changes

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14170: Assignee: (was: Apache Spark) > Remove the PR template before pushing changes >

[jira] [Created] (SPARK-14170) Remove the PR template before pushing changes

2016-03-25 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-14170: -- Summary: Remove the PR template before pushing changes Key: SPARK-14170 URL: https://issues.apache.org/jira/browse/SPARK-14170 Project: Spark Issue

[jira] [Assigned] (SPARK-14013) Properly implement temporary functions in SessionCatalog

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14013: Assignee: (was: Apache Spark) > Properly implement temporary functions in

[jira] [Commented] (SPARK-14013) Properly implement temporary functions in SessionCatalog

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212634#comment-15212634 ] Apache Spark commented on SPARK-14013: -- User 'andrewor14' has created a pull request for this issue:

[jira] [Assigned] (SPARK-14013) Properly implement temporary functions in SessionCatalog

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14013: Assignee: Apache Spark > Properly implement temporary functions in SessionCatalog >

[jira] [Assigned] (SPARK-13955) Spark in yarn mode fails

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13955: Assignee: Apache Spark > Spark in yarn mode fails > > >

[jira] [Commented] (SPARK-13955) Spark in yarn mode fails

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212609#comment-15212609 ] Apache Spark commented on SPARK-13955: -- User 'vanzin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13955) Spark in yarn mode fails

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13955: Assignee: (was: Apache Spark) > Spark in yarn mode fails > >

[jira] [Commented] (SPARK-14169) Add UninterruptibleThread

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212608#comment-15212608 ] Apache Spark commented on SPARK-14169: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Created] (SPARK-14169) Add UninterruptibleThread

2016-03-25 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-14169: Summary: Add UninterruptibleThread Key: SPARK-14169 URL: https://issues.apache.org/jira/browse/SPARK-14169 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-14169) Add UninterruptibleThread

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14169: Assignee: Apache Spark (was: Shixiong Zhu) > Add UninterruptibleThread >

[jira] [Assigned] (SPARK-14169) Add UninterruptibleThread

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14169: Assignee: Shixiong Zhu (was: Apache Spark) > Add UninterruptibleThread >

[jira] [Commented] (SPARK-14141) Let user specify datatypes of pandas dataframe in toPandas()

2016-03-25 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212590#comment-15212590 ] holdenk commented on SPARK-14141: - So with RDDs there is `toLocalIterator` which you could use to do this

[jira] [Commented] (SPARK-14141) Let user specify datatypes of pandas dataframe in toPandas()

2016-03-25 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212587#comment-15212587 ] holdenk commented on SPARK-14141: - The more I look at this the more I think its not a good fit for Spark.

[jira] [Commented] (SPARK-14141) Let user specify datatypes of pandas dataframe in toPandas()

2016-03-25 Thread Luke Miner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212563#comment-15212563 ] Luke Miner commented on SPARK-14141: Is there any way to do this process in chunks: read a chunk of

[jira] [Updated] (SPARK-14159) StringIndexerModel sets output column metadata incorrectly

2016-03-25 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-14159: -- Target Version/s: 1.6.2, 2.0.0 (was: 2.0.0) > StringIndexerModel sets output column metadata

[jira] [Reopened] (SPARK-14159) StringIndexerModel sets output column metadata incorrectly

2016-03-25 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reopened SPARK-14159: --- > StringIndexerModel sets output column metadata incorrectly >

[jira] [Assigned] (SPARK-14159) StringIndexerModel sets output column metadata incorrectly

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14159: Assignee: Joseph K. Bradley (was: Apache Spark) > StringIndexerModel sets output column

[jira] [Assigned] (SPARK-14159) StringIndexerModel sets output column metadata incorrectly

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14159: Assignee: Apache Spark (was: Joseph K. Bradley) > StringIndexerModel sets output column

[jira] [Resolved] (SPARK-14159) StringIndexerModel sets output column metadata incorrectly

2016-03-25 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-14159. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11965

[jira] [Commented] (SPARK-14141) Let user specify datatypes of pandas dataframe in toPandas()

2016-03-25 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212548#comment-15212548 ] holdenk commented on SPARK-14141: - So following up, `from_records` doesn't take dtypes although we could

[jira] [Assigned] (SPARK-14168) Managed Memory Leak Msg Should Only Be a Warning

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14168: Assignee: Imran Rashid (was: Apache Spark) > Managed Memory Leak Msg Should Only Be a

[jira] [Commented] (SPARK-14168) Managed Memory Leak Msg Should Only Be a Warning

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212501#comment-15212501 ] Apache Spark commented on SPARK-14168: -- User 'squito' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-11293) Spillable collections leak shuffle memory

2016-03-25 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212497#comment-15212497 ] Imran Rashid edited comment on SPARK-11293 at 3/25/16 10:27 PM: I've seen

[jira] [Assigned] (SPARK-14168) Managed Memory Leak Msg Should Only Be a Warning

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14168: Assignee: Apache Spark (was: Imran Rashid) > Managed Memory Leak Msg Should Only Be a

[jira] [Commented] (SPARK-11293) Spillable collections leak shuffle memory

2016-03-25 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212497#comment-15212497 ] Imran Rashid commented on SPARK-11293: -- I've seen a few people misled by the error msg, so I'd like

[jira] [Created] (SPARK-14168) Managed Memory Leak Msg Should Only Be a Warning

2016-03-25 Thread Imran Rashid (JIRA)
Imran Rashid created SPARK-14168: Summary: Managed Memory Leak Msg Should Only Be a Warning Key: SPARK-14168 URL: https://issues.apache.org/jira/browse/SPARK-14168 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-14091) Improve performance of SparkContext.getCallSite()

2016-03-25 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-14091. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11911

[jira] [Updated] (SPARK-14091) Improve performance of SparkContext.getCallSite()

2016-03-25 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-14091: --- Summary: Improve performance of SparkContext.getCallSite() (was: Consider improving performance of

[jira] [Updated] (SPARK-14091) Consider improving performance of SparkContext.getCallSite()

2016-03-25 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-14091: --- Assignee: Rajesh Balamohan > Consider improving performance of SparkContext.getCallSite() >

[jira] [Resolved] (SPARK-14167) Remove redundant `return` in Scala code

2016-03-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-14167. --- Resolution: Won't Fix I close this issue based on the comment of [~joshrosen]. "I'm

[jira] [Commented] (SPARK-13783) Model export/import for spark.ml: GBTs

2016-03-25 Thread Gayathri Murali (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212442#comment-15212442 ] Gayathri Murali commented on SPARK-13783: - Thanks [~josephkb]. I can go first, as I am almost

[jira] [Commented] (SPARK-14167) Remove redundant `return` in Scala code

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212441#comment-15212441 ] Apache Spark commented on SPARK-14167: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-14167) Remove redundant `return` in Scala code

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14167: Assignee: Apache Spark > Remove redundant `return` in Scala code >

[jira] [Assigned] (SPARK-14167) Remove redundant `return` in Scala code

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14167: Assignee: (was: Apache Spark) > Remove redundant `return` in Scala code >

[jira] [Commented] (SPARK-13842) Consider __iter__ and __getitem__ methods for pyspark.sql.types.StructType

2016-03-25 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212438#comment-15212438 ] holdenk commented on SPARK-13842: - This makes some additional sense when we consider that `StructType` in

[jira] [Created] (SPARK-14167) Remove redundant `return` in Scala code

2016-03-25 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-14167: - Summary: Remove redundant `return` in Scala code Key: SPARK-14167 URL: https://issues.apache.org/jira/browse/SPARK-14167 Project: Spark Issue Type: Task

[jira] [Commented] (SPARK-13783) Model export/import for spark.ml: GBTs

2016-03-25 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212428#comment-15212428 ] Joseph K. Bradley commented on SPARK-13783: --- I'd prefer what [~GayathriMurali] mentioned;

[jira] [Issue Comment Deleted] (SPARK-6725) Model export/import for Pipeline API (Scala)

2016-03-25 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6725: - Comment: was deleted (was: Ping! Is anyone interested in picking up the GBT or

[jira] [Commented] (SPARK-14141) Let user specify datatypes of pandas dataframe in toPandas()

2016-03-25 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212410#comment-15212410 ] holdenk commented on SPARK-14141: - I can take a crack at this, seems pretty reasonable & small. > Let

[jira] [Updated] (SPARK-13783) Model export/import for spark.ml: GBTs

2016-03-25 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-13783: -- Target Version/s: 2.0.0 > Model export/import for spark.ml: GBTs >

[jira] [Updated] (SPARK-13784) Model export/import for spark.ml: RandomForests

2016-03-25 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-13784: -- Target Version/s: 2.0.0 > Model export/import for spark.ml: RandomForests >

[jira] [Updated] (SPARK-6725) Model export/import for Pipeline API (Scala)

2016-03-25 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6725: - Target Version/s: 2.0.0 > Model export/import for Pipeline API (Scala) >

[jira] [Updated] (SPARK-6725) Model export/import for Pipeline API (Scala)

2016-03-25 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6725: - Summary: Model export/import for Pipeline API (Scala) (was: Model export/import for

[jira] [Commented] (SPARK-6725) Model export/import for Pipeline API

2016-03-25 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212402#comment-15212402 ] Joseph K. Bradley commented on SPARK-6725: -- Ping! Is anyone interested in picking up the GBT or

[jira] [Assigned] (SPARK-14081) DataFrameNaFunctions fill should not convert float fields to double

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14081: Assignee: Apache Spark > DataFrameNaFunctions fill should not convert float fields to

[jira] [Commented] (SPARK-14081) DataFrameNaFunctions fill should not convert float fields to double

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212395#comment-15212395 ] Apache Spark commented on SPARK-14081: -- User 'traviscrawford' has created a pull request for this

[jira] [Assigned] (SPARK-14081) DataFrameNaFunctions fill should not convert float fields to double

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14081: Assignee: (was: Apache Spark) > DataFrameNaFunctions fill should not convert float

[jira] [Updated] (SPARK-14165) NoSuchElementException: None.get when joining DataFrames with Seq of fields of different case

2016-03-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-14165: -- Issue Type: Bug (was: Improvement) Hm yeah this is a problem then. I'm not 100% sure which case

[jira] [Commented] (SPARK-14159) StringIndexerModel sets output column metadata incorrectly

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212369#comment-15212369 ] Apache Spark commented on SPARK-14159: -- User 'jkbradley' has created a pull request for this issue:

[jira] [Commented] (SPARK-14108) calling count() on empty dataframe throws java.util.NoSuchElementException

2016-03-25 Thread Jacek Laskowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212368#comment-15212368 ] Jacek Laskowski commented on SPARK-14108: - I'd like to see the code to show case it since:

[jira] [Assigned] (SPARK-14159) StringIndexerModel sets output column metadata incorrectly

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14159: Assignee: Joseph K. Bradley (was: Apache Spark) > StringIndexerModel sets output column

[jira] [Assigned] (SPARK-14159) StringIndexerModel sets output column metadata incorrectly

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14159: Assignee: Apache Spark (was: Joseph K. Bradley) > StringIndexerModel sets output column

[jira] [Commented] (SPARK-14165) NoSuchElementException: None.get when joining DataFrames with Seq of fields of different case

2016-03-25 Thread Jacek Laskowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212365#comment-15212365 ] Jacek Laskowski commented on SPARK-14165: - Right, but: {code} scala> left.join(right, $"abc" ===

[jira] [Commented] (SPARK-13786) Pyspark ml.tuning support export/import

2016-03-25 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212364#comment-15212364 ] Xusen Yin commented on SPARK-13786: --- I'll work on it. > Pyspark ml.tuning support export/import >

[jira] [Issue Comment Deleted] (SPARK-11666) Find the best `k` by cutting bisecting k-means cluster tree without recomputation

2016-03-25 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak KÖSE updated SPARK-11666: --- Comment: was deleted (was: Hi, can you share links for references about that?) > Find the best `k`

[jira] [Created] (SPARK-14166) Add deterministic sampling like in Hive

2016-03-25 Thread Ruslan Dautkhanov (JIRA)
Ruslan Dautkhanov created SPARK-14166: - Summary: Add deterministic sampling like in Hive Key: SPARK-14166 URL: https://issues.apache.org/jira/browse/SPARK-14166 Project: Spark Issue

[jira] [Comment Edited] (SPARK-14108) calling count() on empty dataframe throws java.util.NoSuchElementException

2016-03-25 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-14108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212363#comment-15212363 ] Burak KÖSE edited comment on SPARK-14108 at 3/25/16 8:38 PM: - Please give an

[jira] [Commented] (SPARK-14108) calling count() on empty dataframe throws java.util.NoSuchElementException

2016-03-25 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-14108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212363#comment-15212363 ] Burak KÖSE commented on SPARK-14108: Please give a test case. > calling count() on empty dataframe

[jira] [Updated] (SPARK-14165) NoSuchElementException: None.get when joining DataFrames with Seq of fields of different case

2016-03-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-14165: -- Issue Type: Improvement (was: Bug) It's case sensitive right? your tables don't actually both have a

[jira] [Resolved] (SPARK-14131) Add a workaround for HADOOP-10622 to fix DataFrameReaderWriterSuite

2016-03-25 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-14131. -- Resolution: Fixed Fix Version/s: 2.0.0 > Add a workaround for HADOOP-10622 to fix

[jira] [Commented] (SPARK-14123) Function related commands

2016-03-25 Thread Bo Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212356#comment-15212356 ] Bo Meng commented on SPARK-14123: - I will be working on this. Thanks. > Function related commands >

[jira] [Updated] (SPARK-14041) Locate possible duplicates and group them into subtasks

2016-03-25 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xusen Yin updated SPARK-14041: -- Description: To find out all examples of ml/mllib that don't contain "example on": {code}grep -L

[jira] [Updated] (SPARK-14164) Improve input layer validation of MultilayerPerceptronClassifier

2016-03-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-14164: -- Summary: Improve input layer validation of MultilayerPerceptronClassifier (was: Improve input

[jira] [Updated] (SPARK-14164) Improve input layer validation of MultilayerPerceptronClassifier

2016-03-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-14164: -- Description: This issue improves an input layer validation and adds related testcases to

[jira] [Assigned] (SPARK-14164) Improve input layer validation to MultilayerPerceptronClassifier

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14164: Assignee: (was: Apache Spark) > Improve input layer validation to

[jira] [Assigned] (SPARK-14164) Improve input layer validation to MultilayerPerceptronClassifier

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14164: Assignee: Apache Spark > Improve input layer validation to MultilayerPerceptronClassifier

[jira] [Commented] (SPARK-14164) Improve input layer validation to MultilayerPerceptronClassifier

2016-03-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212337#comment-15212337 ] Apache Spark commented on SPARK-14164: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Created] (SPARK-14165) NoSuchElementException: None.get when joining DataFrames with Seq of fields of different case

2016-03-25 Thread Jacek Laskowski (JIRA)
Jacek Laskowski created SPARK-14165: --- Summary: NoSuchElementException: None.get when joining DataFrames with Seq of fields of different case Key: SPARK-14165 URL:

[jira] [Updated] (SPARK-14164) Improve input layer validation to MultilayerPerceptronClassifier

2016-03-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-14164: -- Summary: Improve input layer validation to MultilayerPerceptronClassifier (was: Add input

  1   2   3   >