[jira] [Updated] (SPARK-6599) Improve usability and reliability of Kinesis-based Spark Streaming

2015-08-01 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-6599: - Summary: Improve usability and reliability of Kinesis-based Spark Streaming (was: Improve

[jira] [Resolved] (SPARK-8232) complex function: sort_array

2015-08-01 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-8232. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7581

[jira] [Resolved] (SPARK-9515) Creating JavaSparkContext throws NPE

2015-08-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-9515. -- Resolution: Invalid Yes I think one of the problem is that you aren't using spark-submit. You haven't

[jira] [Commented] (SPARK-8263) string function: substr/substring should also support binary type

2015-08-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650187#comment-14650187 ] Apache Spark commented on SPARK-8263: - User 'davies' has created a pull request for

[jira] [Resolved] (SPARK-9517) BytesToBytesMap should encode data the same way as UnsafeExternalSorter

2015-08-01 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-9517. Resolution: Fixed Fix Version/s: 1.5.0 BytesToBytesMap should encode data the same way as

[jira] [Updated] (SPARK-6599) Improve usability and reliability of Kinesis stream

2015-08-01 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-6599: - Summary: Improve usability and reliability of Kinesis stream (was: Improve usability and

[jira] [Updated] (SPARK-6599) Improve usability and reliability of Kinesis stream

2015-08-01 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-6599: - Description: Usability improvements: API improvements, AWS SDK upgrades, etc. Reliability

[jira] [Updated] (SPARK-9480) Create an map abstract class MapData and a default implementation backed by 2 ArrayData

2015-08-01 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9480: --- Parent Issue: SPARK-9413 (was: SPARK-9389) Create an map abstract class MapData and a default

[jira] [Resolved] (SPARK-9480) Create an map abstract class MapData and a default implementation backed by 2 ArrayData

2015-08-01 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-9480. Resolution: Fixed Assignee: Wenchen Fan Fix Version/s: 1.5.0 Create an map

[jira] [Commented] (SPARK-8887) Explicitly define which data types can be used as dynamic partition columns

2015-08-01 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650196#comment-14650196 ] Reynold Xin commented on SPARK-8887: [~lian cheng] can we put this in 1.5?

[jira] [Commented] (SPARK-6873) Some Hive-Catalyst comparison tests fail due to unimportant order of some printed elements

2015-08-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650268#comment-14650268 ] Cheng Lian commented on SPARK-6873: --- It's not important. Internally, Hive just traverses

[jira] [Assigned] (SPARK-9525) Optimize SparseVector initializations in linalg

2015-08-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9525: --- Assignee: (was: Apache Spark) Optimize SparseVector initializations in linalg

[jira] [Commented] (SPARK-9525) Optimize SparseVector initializations in linalg

2015-08-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650392#comment-14650392 ] Apache Spark commented on SPARK-9525: - User 'MechCoder' has created a pull request for

[jira] [Assigned] (SPARK-9525) Optimize SparseVector initializations in linalg

2015-08-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9525: --- Assignee: Apache Spark Optimize SparseVector initializations in linalg

[jira] [Updated] (SPARK-9526) Utilize randomized tests to reveal potential bugs in sql expressions

2015-08-01 Thread Yijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yijie Shen updated SPARK-9526: -- Summary: Utilize randomized tests to reveal potential bugs in sql expressions (was: Utilize randomized

[jira] [Created] (SPARK-9527) PrefixSpan.run should return a PrefixSpanModel instead of an RDD

2015-08-01 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-9527: Summary: PrefixSpan.run should return a PrefixSpanModel instead of an RDD Key: SPARK-9527 URL: https://issues.apache.org/jira/browse/SPARK-9527 Project: Spark

[jira] [Commented] (SPARK-6873) Some Hive-Catalyst comparison tests fail due to unimportant order of some printed elements

2015-08-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650480#comment-14650480 ] Sean Owen commented on SPARK-6873: -- [~rxin] [~lian cheng] It's still a problem. Yes I'm

[jira] [Commented] (SPARK-8505) Add settings to kick `lint-r` from `./dev/run-test.py`

2015-08-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650262#comment-14650262 ] Sean Owen commented on SPARK-8505: -- If you open a pull request, the JIRA is marked as In

[jira] [Created] (SPARK-9524) Latest Spark (8765665015ef47a23e00f7d01d4d280c31bb236d) breaks (pyspark)

2015-08-01 Thread Samuel Marks (JIRA)
Samuel Marks created SPARK-9524: --- Summary: Latest Spark (8765665015ef47a23e00f7d01d4d280c31bb236d) breaks (pyspark) Key: SPARK-9524 URL: https://issues.apache.org/jira/browse/SPARK-9524 Project: Spark

[jira] [Commented] (SPARK-8069) Add support for cutoff to RandomForestClassifier

2015-08-01 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650477#comment-14650477 ] holdenk commented on SPARK-8069: So I was looking at

[jira] [Commented] (SPARK-8999) Support non-temporal sequence in PrefixSpan

2015-08-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650415#comment-14650415 ] Xiangrui Meng commented on SPARK-8999: -- [~srowen] Thanks for your feedback!

[jira] [Created] (SPARK-9526) Utilize ScalaCheck to reveal potential bugs in sql expressions

2015-08-01 Thread Yijie Shen (JIRA)
Yijie Shen created SPARK-9526: - Summary: Utilize ScalaCheck to reveal potential bugs in sql expressions Key: SPARK-9526 URL: https://issues.apache.org/jira/browse/SPARK-9526 Project: Spark

[jira] [Commented] (SPARK-9526) Utilize randomized tests to reveal potential bugs in sql expressions

2015-08-01 Thread Yijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650423#comment-14650423 ] Yijie Shen commented on SPARK-9526: --- [~srowen] thanks for reminding, the current

[jira] [Assigned] (SPARK-9526) Utilize randomized tests to reveal potential bugs in sql expressions

2015-08-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9526: --- Assignee: Apache Spark Utilize randomized tests to reveal potential bugs in sql expressions

[jira] [Commented] (SPARK-9526) Utilize randomized tests to reveal potential bugs in sql expressions

2015-08-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650420#comment-14650420 ] Apache Spark commented on SPARK-9526: - User 'yjshen' has created a pull request for

[jira] [Assigned] (SPARK-9526) Utilize randomized tests to reveal potential bugs in sql expressions

2015-08-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9526: --- Assignee: (was: Apache Spark) Utilize randomized tests to reveal potential bugs in sql

[jira] [Commented] (SPARK-3166) Custom serialisers can't be shipped in application jars

2015-08-01 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650486#comment-14650486 ] Josh Rosen commented on SPARK-3166: --- Does anyone know if this is still an issue in newer

[jira] [Commented] (SPARK-6227) PCA and SVD for PySpark

2015-08-01 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650314#comment-14650314 ] Manoj Kumar commented on SPARK-6227: [~mengxr] Can this be assigned to me? Since the

[jira] [Updated] (SPARK-9524) Latest Spark (8765665015ef47a23e00f7d01d4d280c31bb236d) breaks (pyspark)

2015-08-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9524: - Affects Version/s: (was: 1.5.0) Priority: Major (was: Blocker) [~SamuelMarks] please

[jira] [Resolved] (SPARK-9524) Latest Spark (8765665015ef47a23e00f7d01d4d280c31bb236d) breaks (pyspark)

2015-08-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-9524. -- Resolution: Invalid This doesn't appear to be related to Spark. At least none of the error here shows

[jira] [Updated] (SPARK-9525) Optimize SparseVector initializations in linalg

2015-08-01 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Kumar updated SPARK-9525: --- Priority: Major (was: Minor) Optimize SparseVector initializations in linalg

[jira] [Commented] (SPARK-9526) Utilize randomized tests to reveal potential bugs in sql expressions

2015-08-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650421#comment-14650421 ] Sean Owen commented on SPARK-9526: -- [~yijieshen] at this point should we really be making

[jira] [Commented] (SPARK-9524) Latest Spark (8765665015ef47a23e00f7d01d4d280c31bb236d) breaks (pyspark)

2015-08-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650448#comment-14650448 ] Sean Owen commented on SPARK-9524: -- That's an error from ipython though, not Spark. It

[jira] [Commented] (SPARK-9000) Support generic item type in PrefixSpan

2015-08-01 Thread Masaki Rikitoku (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650301#comment-14650301 ] Masaki Rikitoku commented on SPARK-9000: Hi Xiangrui Meng Thanks for your

[jira] [Updated] (SPARK-9526) Utilize randomized testing to reveal potential bugs in sql expressions

2015-08-01 Thread Yijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yijie Shen updated SPARK-9526: -- Summary: Utilize randomized testing to reveal potential bugs in sql expressions (was: Utilize

[jira] [Created] (SPARK-9525) Optimize SparseVector initializations in linalg

2015-08-01 Thread Manoj Kumar (JIRA)
Manoj Kumar created SPARK-9525: -- Summary: Optimize SparseVector initializations in linalg Key: SPARK-9525 URL: https://issues.apache.org/jira/browse/SPARK-9525 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-8263) string function: substr/substring should also support binary type

2015-08-01 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-8263. --- Resolution: Fixed Fix Version/s: 1.5.0 Target Version/s: (was: ) Issue resolved by

[jira] [Commented] (SPARK-9524) Latest Spark (8765665015ef47a23e00f7d01d4d280c31bb236d) breaks (pyspark)

2015-08-01 Thread Samuel Marks (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650413#comment-14650413 ] Samuel Marks commented on SPARK-9524: - You're welcome to close the issue, it was only

[jira] [Commented] (SPARK-5754) Spark AM not launching on Windows

2015-08-01 Thread Carsten Blank (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650444#comment-14650444 ] Carsten Blank commented on SPARK-5754: -- Okay so I have thought about this more and

[jira] [Created] (SPARK-9529) Improve sort on Decimal

2015-08-01 Thread Davies Liu (JIRA)
Davies Liu created SPARK-9529: - Summary: Improve sort on Decimal Key: SPARK-9529 URL: https://issues.apache.org/jira/browse/SPARK-9529 Project: Spark Issue Type: Sub-task Components:

[jira] [Updated] (SPARK-7146) Should ML sharedParams be a public API?

2015-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7146: - Target Version/s: 1.6.0 (was: 1.5.0) Should ML sharedParams be a public API?

[jira] [Updated] (SPARK-5114) Should Evaluator be a PipelineStage

2015-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-5114: - Issue Type: Sub-task (was: Question) Parent: SPARK-5874 Should Evaluator be a

[jira] [Updated] (SPARK-7809) MultivariateOnlineSummarizer should allow users to configure what to compute

2015-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7809: - Target Version/s: 1.6.0 (was: 1.5.0) MultivariateOnlineSummarizer should allow users to

[jira] [Updated] (SPARK-5535) Add parameter for storage levels.

2015-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-5535: - Target Version/s: 1.6.0 (was: 1.5.0) Add parameter for storage levels.

[jira] [Created] (SPARK-9532) Standardize checks for Predictor and other prediction abstractions

2015-08-01 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-9532: Summary: Standardize checks for Predictor and other prediction abstractions Key: SPARK-9532 URL: https://issues.apache.org/jira/browse/SPARK-9532 Project:

[jira] [Commented] (SPARK-4526) Gradient should be added batch computing interface

2015-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650598#comment-14650598 ] Joseph K. Bradley commented on SPARK-4526: -- I'm going to unset the target

[jira] [Updated] (SPARK-5256) Improving MLlib optimization APIs

2015-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-5256: - Affects Version/s: (was: 1.2.0) Target Version/s: 1.6.0 Improving MLlib

[jira] [Updated] (SPARK-4526) Gradient should be added batch computing interface

2015-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-4526: - Target Version/s: (was: 1.5.0) Gradient should be added batch computing interface

[jira] [Resolved] (SPARK-9459) avoid copy for UTF8String during unsafe sorting

2015-08-01 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9459. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7840

[jira] [Updated] (SPARK-6517) Bisecting k-means clustering

2015-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6517: - Target Version/s: 1.6.0 (was: 1.5.0) Bisecting k-means clustering

[jira] [Commented] (SPARK-6346) Use faster converging optimization method in MLlib

2015-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650596#comment-14650596 ] Joseph K. Bradley commented on SPARK-6346: -- Unsetting target version from 1.5.

[jira] [Updated] (SPARK-6346) Use faster converging optimization method in MLlib

2015-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6346: - Target Version/s: (was: 1.5.0) Use faster converging optimization method in MLlib

[jira] [Updated] (SPARK-9191) Add ml.PCA user guide and code examples

2015-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-9191: - Shepherd: Joseph K. Bradley Add ml.PCA user guide and code examples

[jira] [Resolved] (SPARK-8185) date/time function: datediff

2015-08-01 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-8185. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7847

[jira] [Resolved] (SPARK-8191) date/time function: to_utc_timestamp

2015-08-01 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-8191. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7847

[jira] [Resolved] (SPARK-8188) date/time function: from_utc_timestamp

2015-08-01 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-8188. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7847

[jira] [Commented] (SPARK-7402) JSON serialization of params

2015-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650594#comment-14650594 ] Joseph K. Bradley commented on SPARK-7402: -- I'm switching this to 1.6. JSON

[jira] [Commented] (SPARK-5832) Add Affinity Propagation clustering algorithm

2015-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650600#comment-14650600 ] Joseph K. Bradley commented on SPARK-5832: -- It sounds like we need some more

[jira] [Updated] (SPARK-7376) Python: Add validation functionality to individual Param

2015-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7376: - Target Version/s: 1.6.0 (was: 1.5.0) Python: Add validation functionality to individual

[jira] [Updated] (SPARK-5832) Add Affinity Propagation clustering algorithm

2015-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-5832: - Target Version/s: 1.6.0 (was: 1.5.0) Add Affinity Propagation clustering algorithm

[jira] [Updated] (SPARK-7402) JSON serialization of params

2015-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7402: - Target Version/s: 1.6.0 (was: 1.5.0) JSON serialization of params

[jira] [Updated] (SPARK-8520) Improve GLM's scalability on number of features

2015-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-8520: - Target Version/s: 1.6.0 (was: 1.5.0) Improve GLM's scalability on number of features

[jira] [Commented] (SPARK-6517) Bisecting k-means clustering

2015-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650595#comment-14650595 ] Joseph K. Bradley commented on SPARK-6517: -- I'm sorry, but I think we'll need to

[jira] [Resolved] (SPARK-8269) string function: initcap

2015-08-01 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-8269. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7850

[jira] [Resolved] (SPARK-9520) UnsafeFixedWidthAggregationMap should support in-place sorting of its own records

2015-08-01 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-9520. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7849

[jira] [Assigned] (SPARK-9529) Improve sort on Decimal

2015-08-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9529: --- Assignee: Davies Liu (was: Apache Spark) Improve sort on Decimal ---

[jira] [Commented] (SPARK-9529) Improve sort on Decimal

2015-08-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650540#comment-14650540 ] Apache Spark commented on SPARK-9529: - User 'davies' has created a pull request for

[jira] [Updated] (SPARK-9530) ScalaDoc should not indicate LDAModel.describeTopics and DistributedLDAModel.topDocumentsPerTopic as approximate.

2015-08-01 Thread Meihua Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Meihua Wu updated SPARK-9530: - Summary: ScalaDoc should not indicate LDAModel.describeTopics and

[jira] [Resolved] (SPARK-9530) ScalaDoc should not indicate LDAModel.describeTopics and DistributedLDAModel.topDocumentsPerTopic as approximate.

2015-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-9530. -- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7858

[jira] [Updated] (SPARK-9530) ScalaDoc should not indicate LDAModel.describeTopics and DistributedLDAModel.topDocumentsPerTopic as approximate.

2015-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-9530: - Target Version/s: 1.5.0 (was: 1.3.2, 1.4.2, 1.5.0) ScalaDoc should not indicate

[jira] [Issue Comment Deleted] (SPARK-1902) Spark shell prints error when :4040 port already in use

2015-08-01 Thread Eugene Morozov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Morozov updated SPARK-1902: -- Comment: was deleted (was: It looks like package name has changed since and now

[jira] [Comment Edited] (SPARK-9000) Support generic item type in PrefixSpan

2015-08-01 Thread Masaki Rikitoku (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650301#comment-14650301 ] Masaki Rikitoku edited comment on SPARK-9000 at 8/1/15 12:33 PM:

[jira] [Commented] (SPARK-9514) Add EventHubsReceiver to support Spark Streaming using Azure EventHubs

2015-08-01 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650308#comment-14650308 ] Nan Zhu commented on SPARK-9514: I think the best way to do it is to add a new component

[jira] [Closed] (SPARK-4751) Support dynamic allocation for standalone mode

2015-08-01 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-4751. Resolution: Fixed Fix Version/s: 1.5.0 Support dynamic allocation for standalone mode

[jira] [Created] (SPARK-9528) RandomForestClassifier should extend ProbabilisticClassifier

2015-08-01 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-9528: Summary: RandomForestClassifier should extend ProbabilisticClassifier Key: SPARK-9528 URL: https://issues.apache.org/jira/browse/SPARK-9528 Project: Spark

[jira] [Created] (SPARK-9530) ScalaDoc should not indicate LDAModel.descripeTopic and DistributedLDAModel.topDocumentsPerTopic as approximate.

2015-08-01 Thread Meihua Wu (JIRA)
Meihua Wu created SPARK-9530: Summary: ScalaDoc should not indicate LDAModel.descripeTopic and DistributedLDAModel.topDocumentsPerTopic as approximate. Key: SPARK-9530 URL:

[jira] [Assigned] (SPARK-9530) ScalaDoc should not indicate LDAModel.describeTopics and DistributedLDAModel.topDocumentsPerTopic as approximate.

2015-08-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9530: --- Assignee: Apache Spark ScalaDoc should not indicate LDAModel.describeTopics and

[jira] [Commented] (SPARK-9530) ScalaDoc should not indicate LDAModel.describeTopics and DistributedLDAModel.topDocumentsPerTopic as approximate.

2015-08-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650543#comment-14650543 ] Apache Spark commented on SPARK-9530: - User 'rotationsymmetry' has created a pull

[jira] [Assigned] (SPARK-9530) ScalaDoc should not indicate LDAModel.describeTopics and DistributedLDAModel.topDocumentsPerTopic as approximate.

2015-08-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9530: --- Assignee: (was: Apache Spark) ScalaDoc should not indicate LDAModel.describeTopics and

[jira] [Updated] (SPARK-9530) ScalaDoc should not indicate LDAModel.describeTopics and DistributedLDAModel.topDocumentsPerTopic as approximate.

2015-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-9530: - Assignee: Meihua Wu Target Version/s: 1.3.2, 1.4.2, 1.5.0 ScalaDoc should

[jira] [Assigned] (SPARK-9528) RandomForestClassifier should extend ProbabilisticClassifier

2015-08-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9528: --- Assignee: Apache Spark (was: Joseph K. Bradley) RandomForestClassifier should extend

[jira] [Commented] (SPARK-9528) RandomForestClassifier should extend ProbabilisticClassifier

2015-08-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650558#comment-14650558 ] Apache Spark commented on SPARK-9528: - User 'jkbradley' has created a pull request for

[jira] [Assigned] (SPARK-9528) RandomForestClassifier should extend ProbabilisticClassifier

2015-08-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9528: --- Assignee: Joseph K. Bradley (was: Apache Spark) RandomForestClassifier should extend

[jira] [Commented] (SPARK-8612) Yarn application status is misreported for failed PySpark apps.

2015-08-01 Thread Neelesh Srinivas Salian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650564#comment-14650564 ] Neelesh Srinivas Salian commented on SPARK-8612: Similar to SPARK-7736?

[jira] [Updated] (SPARK-9514) Add EventHubsReceiver to support Spark Streaming using Azure EventHubs

2015-08-01 Thread shanyu zhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shanyu zhao updated SPARK-9514: --- Attachment: SPARK-9514.patch Patch attached. I put EventHubsReceiver in external folder and added an

[jira] [Updated] (SPARK-8874) Add missing methods in Word2Vec ML

2015-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-8874: - Assignee: Manoj Kumar Add missing methods in Word2Vec ML

[jira] [Commented] (SPARK-9514) Add EventHubsReceiver to support Spark Streaming using Azure EventHubs

2015-08-01 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650568#comment-14650568 ] Nan Zhu commented on SPARK-9514: Hi, [~shanyu], in Spark, we usually submit patches via

[jira] [Resolved] (SPARK-9495) Support prefix generation for date / timestamp data type

2015-08-01 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-9495. Resolution: Fixed Assignee: Davies Liu Fix Version/s: 1.5.0 Support prefix

[jira] [Commented] (SPARK-8069) Add support for cutoff to RandomForestClassifier

2015-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650514#comment-14650514 ] Joseph K. Bradley commented on SPARK-8069: -- My final plan was to only have it for

[jira] [Updated] (SPARK-9491) App running on secure YARN with no HBase config will hang

2015-08-01 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-9491: -- Assignee: Marcelo Vanzin Affects Version/s: 1.4.0 Target Version/s: 1.4.2,

[jira] [Commented] (SPARK-9531) UnsafeFixedWidthAggregationMap should be able to turn itself into an UnsafeKVExternalSorter

2015-08-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650562#comment-14650562 ] Apache Spark commented on SPARK-9531: - User 'rxin' has created a pull request for this

[jira] [Assigned] (SPARK-9531) UnsafeFixedWidthAggregationMap should be able to turn itself into an UnsafeKVExternalSorter

2015-08-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9531: --- Assignee: Reynold Xin (was: Apache Spark) UnsafeFixedWidthAggregationMap should be able to

[jira] [Assigned] (SPARK-9531) UnsafeFixedWidthAggregationMap should be able to turn itself into an UnsafeKVExternalSorter

2015-08-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9531: --- Assignee: Apache Spark (was: Reynold Xin) UnsafeFixedWidthAggregationMap should be able to

[jira] [Comment Edited] (SPARK-8874) Add missing methods in Word2Vec ML

2015-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650570#comment-14650570 ] Joseph K. Bradley edited comment on SPARK-8874 at 8/2/15 1:14 AM:

[jira] [Commented] (SPARK-9526) Utilize randomized tests to reveal potential bugs in sql expressions

2015-08-01 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650498#comment-14650498 ] Reynold Xin commented on SPARK-9526: I downgraded it to major. While it is great to

[jira] [Resolved] (SPARK-9491) App running on secure YARN with no HBase config will hang

2015-08-01 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-9491. --- Resolution: Fixed App running on secure YARN with no HBase config will hang

[jira] [Assigned] (SPARK-9495) Support prefix generation for date / timestamp data type

2015-08-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9495: --- Assignee: (was: Apache Spark) Support prefix generation for date / timestamp data type

[jira] [Assigned] (SPARK-9483) UTF8String.getPrefix only works in little-endian order

2015-08-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9483: --- Assignee: Apache Spark (was: Matthew Brandyberry) UTF8String.getPrefix only works in

[jira] [Commented] (SPARK-5754) Spark AM not launching on Windows

2015-08-01 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650551#comment-14650551 ] Inigo Goiri commented on SPARK-5754: I overwrote the existing escapeForShell to use

[jira] [Commented] (SPARK-8333) Spark failed to delete temp directory created by HiveContext

2015-08-01 Thread Sudhakar Thota (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650555#comment-14650555 ] Sudhakar Thota commented on SPARK-8333: --- Thanks for the clarification. I have used

  1   2   >