[jira] [Commented] (SPARK-16863) ProbabilisticClassifier.fit check threshoulds' length

2016-08-02 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405333#comment-15405333 ] zhengruifeng commented on SPARK-16863: -- [SPARK-16851] add checking for

[jira] [Comment Edited] (SPARK-7146) Should ML sharedParams be a public API?

2016-08-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405300#comment-15405300 ] Nicholas Chammas edited comment on SPARK-7146 at 8/3/16 4:45 AM: - A quick

[jira] [Commented] (SPARK-7146) Should ML sharedParams be a public API?

2016-08-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405300#comment-15405300 ] Nicholas Chammas commented on SPARK-7146: - A quick update from a PySpark user: I am using

[jira] [Commented] (SPARK-16863) ProbabilisticClassifier.fit check threshoulds' length

2016-08-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405276#comment-15405276 ] Sean Owen commented on SPARK-16863: --- Wait, how is this different from SPARK-16851? I don't see why

[jira] [Assigned] (SPARK-16862) Configurable buffer size in `UnsafeSorterSpillReader`

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16862: Assignee: (was: Apache Spark) > Configurable buffer size in `UnsafeSorterSpillReader`

[jira] [Commented] (SPARK-16862) Configurable buffer size in `UnsafeSorterSpillReader`

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405245#comment-15405245 ] Apache Spark commented on SPARK-16862: -- User 'tejasapatil' has created a pull request for this

[jira] [Assigned] (SPARK-16862) Configurable buffer size in `UnsafeSorterSpillReader`

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16862: Assignee: Apache Spark > Configurable buffer size in `UnsafeSorterSpillReader` >

[jira] [Assigned] (SPARK-16853) Analysis error for DataSet typed selection

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16853: Assignee: (was: Apache Spark) > Analysis error for DataSet typed selection >

[jira] [Commented] (SPARK-16853) Analysis error for DataSet typed selection

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405220#comment-15405220 ] Apache Spark commented on SPARK-16853: -- User 'clockfly' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16853) Analysis error for DataSet typed selection

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16853: Assignee: Apache Spark > Analysis error for DataSet typed selection >

[jira] [Commented] (SPARK-16495) Add ADMM optimizer in mllib package

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405217#comment-15405217 ] Apache Spark commented on SPARK-16495: -- User 'ZunwenYou' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16866) Basic infrastructure for file-based SQL end-to-end tests

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16866: Assignee: Apache Spark > Basic infrastructure for file-based SQL end-to-end tests >

[jira] [Commented] (SPARK-16866) Basic infrastructure for file-based SQL end-to-end tests

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405203#comment-15405203 ] Apache Spark commented on SPARK-16866: -- User 'petermaxlee' has created a pull request for this

[jira] [Assigned] (SPARK-16866) Basic infrastructure for file-based SQL end-to-end tests

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16866: Assignee: (was: Apache Spark) > Basic infrastructure for file-based SQL end-to-end

[jira] [Created] (SPARK-16866) Basic infrastructure for file-based SQL end-to-end tests

2016-08-02 Thread Peter Lee (JIRA)
Peter Lee created SPARK-16866: - Summary: Basic infrastructure for file-based SQL end-to-end tests Key: SPARK-16866 URL: https://issues.apache.org/jira/browse/SPARK-16866 Project: Spark Issue

[jira] [Created] (SPARK-16865) A file-based end-to-end SQL query suite

2016-08-02 Thread Peter Lee (JIRA)
Peter Lee created SPARK-16865: - Summary: A file-based end-to-end SQL query suite Key: SPARK-16865 URL: https://issues.apache.org/jira/browse/SPARK-16865 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-14387) Exceptions thrown when querying ORC tables

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405192#comment-15405192 ] Apache Spark commented on SPARK-14387: -- User 'rajeshbalamohan' has created a pull request for this

[jira] [Created] (SPARK-16864) Comprehensive version info

2016-08-02 Thread jay vyas (JIRA)
jay vyas created SPARK-16864: Summary: Comprehensive version info Key: SPARK-16864 URL: https://issues.apache.org/jira/browse/SPARK-16864 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-16863) ProbabilisticClassifier.fit check threshoulds' length

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16863: Assignee: Apache Spark > ProbabilisticClassifier.fit check threshoulds' length >

[jira] [Commented] (SPARK-16863) ProbabilisticClassifier.fit check threshoulds' length

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405182#comment-15405182 ] Apache Spark commented on SPARK-16863: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-16863) ProbabilisticClassifier.fit check threshoulds' length

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16863: Assignee: (was: Apache Spark) > ProbabilisticClassifier.fit check threshoulds' length

[jira] [Created] (SPARK-16863) ProbabilisticClassifier.fit check threshoulds' length

2016-08-02 Thread zhengruifeng (JIRA)
zhengruifeng created SPARK-16863: Summary: ProbabilisticClassifier.fit check threshoulds' length Key: SPARK-16863 URL: https://issues.apache.org/jira/browse/SPARK-16863 Project: Spark Issue

[jira] [Created] (SPARK-16862) Configurable buffer size in `UnsafeSorterSpillReader`

2016-08-02 Thread Tejas Patil (JIRA)
Tejas Patil created SPARK-16862: --- Summary: Configurable buffer size in `UnsafeSorterSpillReader` Key: SPARK-16862 URL: https://issues.apache.org/jira/browse/SPARK-16862 Project: Spark Issue

[jira] [Commented] (SPARK-14453) Remove SPARK_JAVA_OPTS environment variable

2016-08-02 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405112#comment-15405112 ] Saisai Shao commented on SPARK-14453: - If you want to fix this issue, it would be better target to

[jira] [Commented] (SPARK-16610) When writing ORC files, orc.compress should not be overridden if users do not set "compression" in the options

2016-08-02 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405071#comment-15405071 ] Hyukjin Kwon commented on SPARK-16610: -- One thought is, we might have to document that we don't

[jira] [Commented] (SPARK-16857) CrossValidator and KMeans throws IllegalArgumentException

2016-08-02 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405069#comment-15405069 ] Xusen Yin commented on SPARK-16857: --- I agree the cluster assignments could be arbitrary. Yes under this

[jira] [Commented] (SPARK-16320) Spark 2.0 slower than 1.6 when querying nested columns

2016-08-02 Thread Sean Zhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405057#comment-15405057 ] Sean Zhong commented on SPARK-16320: [~maver1ck] Did you use the test case in this jira {code} select

[jira] [Commented] (SPARK-16857) CrossValidator and KMeans throws IllegalArgumentException

2016-08-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405055#comment-15405055 ] Sean Owen commented on SPARK-16857: --- I don't think that in general it makes sense to use

[jira] [Commented] (SPARK-16857) CrossValidator and KMeans throws IllegalArgumentException

2016-08-02 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405050#comment-15405050 ] Xusen Yin commented on SPARK-16857: --- Using CrossValidator with KMeans should be supported. As a kind of

[jira] [Commented] (SPARK-16700) StructType doesn't accept Python dicts anymore

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405044#comment-15405044 ] Apache Spark commented on SPARK-16700: -- User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16700) StructType doesn't accept Python dicts anymore

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16700: Assignee: (was: Apache Spark) > StructType doesn't accept Python dicts anymore >

[jira] [Commented] (SPARK-16700) StructType doesn't accept Python dicts anymore

2016-08-02 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405043#comment-15405043 ] Davies Liu commented on SPARK-16700: Sent PR https://github.com/apache/spark/pull/14469 to address

[jira] [Assigned] (SPARK-16700) StructType doesn't accept Python dicts anymore

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16700: Assignee: Apache Spark > StructType doesn't accept Python dicts anymore >

[jira] [Assigned] (SPARK-16700) StructType doesn't accept Python dicts anymore

2016-08-02 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-16700: -- Assignee: Davies Liu > StructType doesn't accept Python dicts anymore >

[jira] [Updated] (SPARK-15541) SparkContext.stop throws error

2016-08-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15541: -- Fix Version/s: 2.1.0 2.0.1 1.6.3 > SparkContext.stop throws

[jira] [Assigned] (SPARK-16671) Merge variable substitution code in core and SQL

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16671: Assignee: (was: Apache Spark) > Merge variable substitution code in core and SQL >

[jira] [Assigned] (SPARK-16671) Merge variable substitution code in core and SQL

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16671: Assignee: Apache Spark > Merge variable substitution code in core and SQL >

[jira] [Commented] (SPARK-16671) Merge variable substitution code in core and SQL

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405022#comment-15405022 ] Apache Spark commented on SPARK-16671: -- User 'vanzin' has created a pull request for this issue:

[jira] [Resolved] (SPARK-16796) Visible passwords on Spark environment page

2016-08-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16796. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14409

[jira] [Assigned] (SPARK-16861) Refactor PySpark accumulator API to be on top of AccumulatorV2 API

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16861: Assignee: (was: Apache Spark) > Refactor PySpark accumulator API to be on top of

[jira] [Commented] (SPARK-16861) Refactor PySpark accumulator API to be on top of AccumulatorV2 API

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404891#comment-15404891 ] Apache Spark commented on SPARK-16861: -- User 'holdenk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16861) Refactor PySpark accumulator API to be on top of AccumulatorV2 API

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16861: Assignee: Apache Spark > Refactor PySpark accumulator API to be on top of AccumulatorV2

[jira] [Created] (SPARK-16861) Refactor PySpark accumulator API to be on top of AccumulatorV2 API

2016-08-02 Thread holdenk (JIRA)
holdenk created SPARK-16861: --- Summary: Refactor PySpark accumulator API to be on top of AccumulatorV2 API Key: SPARK-16861 URL: https://issues.apache.org/jira/browse/SPARK-16861 Project: Spark

[jira] [Resolved] (SPARK-16858) Removal of TestHiveSharedState

2016-08-02 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-16858. - Resolution: Fixed Assignee: Xiao Li Fix Version/s: 2.1.0 > Removal of

[jira] [Commented] (SPARK-16321) Spark 2.0 performance drop vs Spark 1.6 when reading parquet file

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404676#comment-15404676 ] Apache Spark commented on SPARK-16321: -- User 'maver1ck' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16321) Spark 2.0 performance drop vs Spark 1.6 when reading parquet file

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16321: Assignee: (was: Apache Spark) > Spark 2.0 performance drop vs Spark 1.6 when reading

[jira] [Assigned] (SPARK-16321) Spark 2.0 performance drop vs Spark 1.6 when reading parquet file

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16321: Assignee: Apache Spark > Spark 2.0 performance drop vs Spark 1.6 when reading parquet

[jira] [Commented] (SPARK-16320) Spark 2.0 slower than 1.6 when querying nested columns

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404675#comment-15404675 ] Apache Spark commented on SPARK-16320: -- User 'maver1ck' has created a pull request for this issue:

[jira] [Commented] (SPARK-16321) Spark 2.0 performance drop vs Spark 1.6 when reading parquet file

2016-08-02 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404654#comment-15404654 ] Maciej Bryński commented on SPARK-16321: [~smilegator] spark.sql.parquet.filterPushdown has true

[jira] [Created] (SPARK-16860) UDT Stringification Incorrect in PySpark

2016-08-02 Thread Vladimir Feinberg (JIRA)
Vladimir Feinberg created SPARK-16860: - Summary: UDT Stringification Incorrect in PySpark Key: SPARK-16860 URL: https://issues.apache.org/jira/browse/SPARK-16860 Project: Spark Issue

[jira] [Commented] (SPARK-16321) Spark 2.0 performance drop vs Spark 1.6 when reading parquet file

2016-08-02 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404652#comment-15404652 ] Xiao Li commented on SPARK-16321: - Can you set `spark.sql.parquet.enableVectorizedReader` to false and

[jira] [Comment Edited] (SPARK-16320) Spark 2.0 slower than 1.6 when querying nested columns

2016-08-02 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404646#comment-15404646 ] Maciej Bryński edited comment on SPARK-16320 at 8/2/16 7:38 PM:

[jira] [Comment Edited] (SPARK-16320) Spark 2.0 slower than 1.6 when querying nested columns

2016-08-02 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404646#comment-15404646 ] Maciej Bryński edited comment on SPARK-16320 at 8/2/16 7:37 PM:

[jira] [Commented] (SPARK-16320) Spark 2.0 slower than 1.6 when querying nested columns

2016-08-02 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404646#comment-15404646 ] Maciej Bryński commented on SPARK-16320: [~michael], [~yhuai] I think this is smallest change

[jira] [Commented] (SPARK-16320) Spark 2.0 slower than 1.6 when querying nested columns

2016-08-02 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404620#comment-15404620 ] Maciej Bryński commented on SPARK-16320: I think that problem is already resolved by

[jira] [Commented] (SPARK-16320) Spark 2.0 slower than 1.6 when querying nested columns

2016-08-02 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404619#comment-15404619 ] Maciej Bryński commented on SPARK-16320: Yes. That's it. With this PR Spark 2.0 is faster than

[jira] [Commented] (SPARK-16802) joins.LongToUnsafeRowMap crashes with ArrayIndexOutOfBoundsException

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404611#comment-15404611 ] Apache Spark commented on SPARK-16802: -- User 'davies' has created a pull request for this issue:

[jira] [Resolved] (SPARK-16787) SparkContext.addFile() should not fail if called twice with the same file

2016-08-02 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-16787. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Commented] (SPARK-16838) Add PMML export for ML KMeans in PySpark

2016-08-02 Thread Gayathri Murali (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404591#comment-15404591 ] Gayathri Murali commented on SPARK-16838: - I can work on this > Add PMML export for ML KMeans in

[jira] [Updated] (SPARK-16859) History Server storage information is missing

2016-08-02 Thread Andrey Ivanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Ivanov updated SPARK-16859: -- Description: It looks like job history storage tab in history server is broken for completed

[jira] [Resolved] (SPARK-6399) Code compiled against 1.3.0 may not run against older Spark versions

2016-08-02 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-6399. --- Resolution: Won't Fix I think at this point it's pretty clear we won't do anything here. >

[jira] [Created] (SPARK-16859) History Server storage information is missing

2016-08-02 Thread Andrey Ivanov (JIRA)
Andrey Ivanov created SPARK-16859: - Summary: History Server storage information is missing Key: SPARK-16859 URL: https://issues.apache.org/jira/browse/SPARK-16859 Project: Spark Issue Type:

[jira] [Updated] (SPARK-16855) move Greatest and Least from conditionalExpressions.scala to arithmetic.scala

2016-08-02 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16855: Fix Version/s: (was: 2.0.1) > move Greatest and Least from conditionalExpressions.scala to

[jira] [Resolved] (SPARK-16855) move Greatest and Least from conditionalExpressions.scala to arithmetic.scala

2016-08-02 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-16855. - Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 > move Greatest and Least

[jira] [Updated] (SPARK-16796) Visible passwords on Spark environment page

2016-08-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16796: -- Assignee: Artur > Visible passwords on Spark environment page >

[jira] [Assigned] (SPARK-16858) Removal of TestHiveSharedState

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16858: Assignee: (was: Apache Spark) > Removal of TestHiveSharedState >

[jira] [Commented] (SPARK-16858) Removal of TestHiveSharedState

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404464#comment-15404464 ] Apache Spark commented on SPARK-16858: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16858) Removal of TestHiveSharedState

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16858: Assignee: Apache Spark > Removal of TestHiveSharedState > --

[jira] [Created] (SPARK-16858) Removal of TestHiveSharedState

2016-08-02 Thread Xiao Li (JIRA)
Xiao Li created SPARK-16858: --- Summary: Removal of TestHiveSharedState Key: SPARK-16858 URL: https://issues.apache.org/jira/browse/SPARK-16858 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-15639) Try to push down filter at RowGroups level for parquet reader

2016-08-02 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15639: --- Target Version/s: 2.0.1 > Try to push down filter at RowGroups level for parquet reader >

[jira] [Updated] (SPARK-15639) Try to push down filter at RowGroups level for parquet reader

2016-08-02 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15639: --- Priority: Blocker (was: Major) > Try to push down filter at RowGroups level for parquet reader >

[jira] [Updated] (SPARK-16816) Add documentation to create JavaSparkContext from SparkSession

2016-08-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16816: -- Assignee: sandeep purohit > Add documentation to create JavaSparkContext from SparkSession >

[jira] [Resolved] (SPARK-16816) Add documentation to create JavaSparkContext from SparkSession

2016-08-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16816. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14436

[jira] [Updated] (SPARK-16850) Improve error message for greatest/least

2016-08-02 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16850: Fix Version/s: 2.0.1 > Improve error message for greatest/least >

[jira] [Created] (SPARK-16857) CrossValidator and KMeans throws IllegalArgumentException

2016-08-02 Thread Ryan Claussen (JIRA)
Ryan Claussen created SPARK-16857: - Summary: CrossValidator and KMeans throws IllegalArgumentException Key: SPARK-16857 URL: https://issues.apache.org/jira/browse/SPARK-16857 Project: Spark

[jira] [Resolved] (SPARK-16836) Hive date/time function error

2016-08-02 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-16836. - Resolution: Fixed Assignee: Herman van Hovell Fix Version/s: 2.1.0

[jira] [Resolved] (SPARK-16062) PySpark SQL python-only UDTs don't work well

2016-08-02 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-16062. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Resolved] (SPARK-15989) PySpark SQL python-only UDTs don't support nested types

2016-08-02 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15989. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Commented] (SPARK-16320) Spark 2.0 slower than 1.6 when querying nested columns

2016-08-02 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404369#comment-15404369 ] Yin Huai commented on SPARK-16320: -- Can you also try https://github.com/apache/spark/pull/13701 and see

[jira] [Comment Edited] (SPARK-16320) Spark 2.0 slower than 1.6 when querying nested columns

2016-08-02 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404369#comment-15404369 ] Yin Huai edited comment on SPARK-16320 at 8/2/16 4:52 PM: -- [~maver1ck] Can you

[jira] [Commented] (SPARK-16320) Spark 2.0 slower than 1.6 when querying nested columns

2016-08-02 Thread Michael Allman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404361#comment-15404361 ] Michael Allman commented on SPARK-16320: [~maver1ck] I'm having trouble reproducing your problem.

[jira] [Commented] (SPARK-16856) Link application summary page and detail page to the master page

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404331#comment-15404331 ] Apache Spark commented on SPARK-16856: -- User 'nblintao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16856) Link application summary page and detail page to the master page

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16856: Assignee: (was: Apache Spark) > Link application summary page and detail page to the

[jira] [Assigned] (SPARK-16856) Link application summary page and detail page to the master page

2016-08-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16856: Assignee: Apache Spark > Link application summary page and detail page to the master page

[jira] [Resolved] (SPARK-16835) LinearRegression LogisticRegression AFTSuvivalRegression should unpersist input training data when exception throws

2016-08-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16835. --- Resolution: Won't Fix > LinearRegression LogisticRegression AFTSuvivalRegression should unpersist >

[jira] [Updated] (SPARK-16837) TimeWindow incorrectly drops slideDuration in constructors

2016-08-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16837: -- Assignee: Tom Magrino > TimeWindow incorrectly drops slideDuration in constructors >

[jira] [Resolved] (SPARK-16837) TimeWindow incorrectly drops slideDuration in constructors

2016-08-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16837. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Updated] (SPARK-16822) Support latex in scaladoc with MathJax

2016-08-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16822: -- Assignee: Shuai Lin > Support latex in scaladoc with MathJax > --

[jira] [Resolved] (SPARK-16822) Support latex in scaladoc with MathJax

2016-08-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16822. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14438

[jira] [Created] (SPARK-16856) Link application summary page and detail page to the master page

2016-08-02 Thread Tao Lin (JIRA)
Tao Lin created SPARK-16856: --- Summary: Link application summary page and detail page to the master page Key: SPARK-16856 URL: https://issues.apache.org/jira/browse/SPARK-16856 Project: Spark

[jira] [Updated] (SPARK-16520) Link executors to corresponding worker pages

2016-08-02 Thread Tao Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Lin updated SPARK-16520: Priority: Major (was: Minor) > Link executors to corresponding worker pages >

[jira] [Resolved] (SPARK-15541) SparkContext.stop throws error

2016-08-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15541. --- Resolution: Resolved Assignee: Maciej Bryński Resolved by

[jira] [Commented] (SPARK-16854) mapWithState Support for Python

2016-08-02 Thread Boaz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404217#comment-15404217 ] Boaz commented on SPARK-16854: -- IMHO, streaming in python would be incomplete without mapWithState. Finding

[jira] [Updated] (SPARK-12650) No means to specify Xmx settings for spark-submit in cluster deploy mode for Spark on YARN

2016-08-02 Thread Jacek Laskowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Laskowski updated SPARK-12650: Summary: No means to specify Xmx settings for spark-submit in cluster deploy mode for

[jira] [Updated] (SPARK-12650) No means to specify Xmx settings for SparkSubmit in cluster deploy mode for Spark on YARN

2016-08-02 Thread Jacek Laskowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Laskowski updated SPARK-12650: Summary: No means to specify Xmx settings for SparkSubmit in cluster deploy mode for Spark

[jira] [Commented] (SPARK-14453) Remove SPARK_JAVA_OPTS environment variable

2016-08-02 Thread Jacek Laskowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404175#comment-15404175 ] Jacek Laskowski commented on SPARK-14453: - Is anyone working on it? Just found few places in

[jira] [Updated] (SPARK-14453) Remove SPARK_JAVA_OPTS environment variable

2016-08-02 Thread Jacek Laskowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Laskowski updated SPARK-14453: Issue Type: Task (was: Bug) > Remove SPARK_JAVA_OPTS environment variable >

[jira] [Updated] (SPARK-14453) Remove SPARK_JAVA_OPTS environment variable

2016-08-02 Thread Jacek Laskowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Laskowski updated SPARK-14453: Summary: Remove SPARK_JAVA_OPTS environment variable (was: Consider removing

[jira] [Commented] (SPARK-16852) RejectedExecutionException when exit at some times

2016-08-02 Thread Weizhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404166#comment-15404166 ] Weizhong commented on SPARK-16852: -- I run 2T tpcds, and some times will print the stack. {noformat}

[jira] [Comment Edited] (SPARK-16320) Spark 2.0 slower than 1.6 when querying nested columns

2016-08-02 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404114#comment-15404114 ] Maciej Bryński edited comment on SPARK-16320 at 8/2/16 3:05 PM:

  1   2   >