[jira] [Commented] (SPARK-3995) [PYSPARK] PySpark's sample methods do not work with NumPy 1.9

2014-11-26 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225886#comment-14225886 ] Xiangrui Meng commented on SPARK-3995: -- I think we can backport SPARK-4477, which

[jira] [Created] (SPARK-4619) Double ms in ShuffleBlockFetcherIterator log

2014-11-26 Thread maji2014 (JIRA)
maji2014 created SPARK-4619: --- Summary: Double ms in ShuffleBlockFetcherIterator log Key: SPARK-4619 URL: https://issues.apache.org/jira/browse/SPARK-4619 Project: Spark Issue Type: Bug Affects

[jira] [Commented] (SPARK-4584) 2x Performance regression for Spark-on-YARN

2014-11-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225899#comment-14225899 ] Sean Owen commented on SPARK-4584: -- {{SecurityManager}} is something that loads of the

[jira] [Commented] (SPARK-4619) Double ms in ShuffleBlockFetcherIterator log

2014-11-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225910#comment-14225910 ] Apache Spark commented on SPARK-4619: - User 'maji2014' has created a pull request for

[jira] [Created] (SPARK-4620) Add unpersist in Graph/GraphImpl

2014-11-26 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-4620: --- Summary: Add unpersist in Graph/GraphImpl Key: SPARK-4620 URL: https://issues.apache.org/jira/browse/SPARK-4620 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-4620) Add unpersist in Graph/GraphImpl

2014-11-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225924#comment-14225924 ] Apache Spark commented on SPARK-4620: - User 'maropu' has created a pull request for

[jira] [Updated] (SPARK-4620) Add unpersist in Graph/GraphImpl

2014-11-26 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-4620: Description: Add an IF to uncache both vertices and edges of Graph/GraphImpl. This IF is

[jira] [Commented] (SPARK-4537) Add 'processing delay' and 'totalDelay' to the metrics reported by the Spark Streaming subsystem

2014-11-26 Thread Gerard Maas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225945#comment-14225945 ] Gerard Maas commented on SPARK-4537: I was waiting for internal clearance to put a PR

[jira] [Resolved] (SPARK-4002) KafkaStreamSuite Kafka input stream case fails on OSX

2014-11-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4002. -- Resolution: Cannot Reproduce Yes I haven't seen this in a while so let's mark it CannotReproduce

[jira] [Created] (SPARK-4621) when sort- based shuffle, Cache recently used shuffle index can reduce indexFile's io

2014-11-26 Thread Lianhui Wang (JIRA)
Lianhui Wang created SPARK-4621: --- Summary: when sort- based shuffle, Cache recently used shuffle index can reduce indexFile's io Key: SPARK-4621 URL: https://issues.apache.org/jira/browse/SPARK-4621

[jira] [Updated] (SPARK-4621) when sort- based shuffle, Cache recently finished shuffle index can reduce indexFile's io

2014-11-26 Thread Lianhui Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianhui Wang updated SPARK-4621: Summary: when sort- based shuffle, Cache recently finished shuffle index can reduce indexFile's io

[jira] [Commented] (SPARK-2468) Netty-based block server / client module

2014-11-26 Thread zzc (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225960#comment-14225960 ] zzc commented on SPARK-2468: AH, Aaron Davidson, With patch #3465, I can run successful

[jira] [Commented] (SPARK-4528) [SQL] add comment support for Spark SQL CLI

2014-11-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225989#comment-14225989 ] Apache Spark commented on SPARK-4528: - User 'tsingfu' has created a pull request for

[jira] [Commented] (SPARK-4613) Make JdbcRDD easier to use from Java

2014-11-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226007#comment-14226007 ] Apache Spark commented on SPARK-4613: - User 'liancheng' has created a pull request for

[jira] [Updated] (SPARK-3546) InputStream of ManagedBuffer is not closed and causes running out of file descriptor

2014-11-26 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3546: -- Target Version/s: 1.2.0 (was: 1.1.1, 1.2.0) InputStream of ManagedBuffer is not closed and

[jira] [Commented] (SPARK-3546) InputStream of ManagedBuffer is not closed and causes running out of file descriptor

2014-11-26 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226051#comment-14226051 ] Kousuke Saruta commented on SPARK-3546: --- [~andrewor14] I don't think we need this

[jira] [Commented] (SPARK-4615) Cannot disconnect from spark-shell

2014-11-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226097#comment-14226097 ] Sean Owen commented on SPARK-4615: -- I can't reproduce this, although I'm using Java 7,

[jira] [Created] (SPARK-4622) Add the some error infomation if using spark-sql in yarn-cluster mode

2014-11-26 Thread hzw (JIRA)
hzw created SPARK-4622: -- Summary: Add the some error infomation if using spark-sql in yarn-cluster mode Key: SPARK-4622 URL: https://issues.apache.org/jira/browse/SPARK-4622 Project: Spark Issue Type:

[jira] [Created] (SPARK-4623) Add the some error infomation if using spark-sql in yarn-cluster mode

2014-11-26 Thread hzw (JIRA)
hzw created SPARK-4623: -- Summary: Add the some error infomation if using spark-sql in yarn-cluster mode Key: SPARK-4623 URL: https://issues.apache.org/jira/browse/SPARK-4623 Project: Spark Issue Type:

[jira] [Created] (SPARK-4624) Errors when reading/writtign to S3 large object files

2014-11-26 Thread Kriton Tsintaris (JIRA)
Kriton Tsintaris created SPARK-4624: --- Summary: Errors when reading/writtign to S3 large object files Key: SPARK-4624 URL: https://issues.apache.org/jira/browse/SPARK-4624 Project: Spark

[jira] [Commented] (SPARK-2192) Examples Data Not in Binary Distribution

2014-11-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226151#comment-14226151 ] Sean Owen commented on SPARK-2192: -- Oops, on further inspection I see that the file is

[jira] [Commented] (SPARK-2192) Examples Data Not in Binary Distribution

2014-11-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226175#comment-14226175 ] Apache Spark commented on SPARK-2192: - User 'srowen' has created a pull request for

[jira] [Updated] (SPARK-4621) when sort- based shuffle, Cache recently finished shuffle index can reduce indexFile's io

2014-11-26 Thread Lianhui Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianhui Wang updated SPARK-4621: Priority: Minor (was: Critical) when sort- based shuffle, Cache recently finished shuffle index

[jira] [Commented] (SPARK-4196) Streaming + checkpointing + saveAsNewAPIHadoopFiles = NotSerializableException for Hadoop Configuration

2014-11-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226192#comment-14226192 ] Sean Owen commented on SPARK-4196: -- Looks good. At least, this commit got me past this

[jira] [Created] (SPARK-4625) Support Sort By in both DSL SimpleSQLParser

2014-11-26 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-4625: Summary: Support Sort By in both DSL SimpleSQLParser Key: SPARK-4625 URL: https://issues.apache.org/jira/browse/SPARK-4625 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-4625) Support Sort By in both DSL SimpleSQLParser

2014-11-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226280#comment-14226280 ] Apache Spark commented on SPARK-4625: - User 'chenghao-intel' has created a pull

[jira] [Resolved] (SPARK-4612) Configuration object gets created for every task even if not new file/jar is added

2014-11-26 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-4612. -- Resolution: Fixed Fix Version/s: 1.2.0 Target Version/s: 1.2.0 (was: 1.3.0)

[jira] [Updated] (SPARK-4046) Incorrect Java example on site

2014-11-26 Thread Brennon York (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brennon York updated SPARK-4046: Attachment: SPARK-4046.diff Pulled down the site at the [Apache

[jira] [Resolved] (SPARK-4604) Make MatrixFactorizationModel constructor public

2014-11-26 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-4604. -- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 3473

[jira] [Commented] (SPARK-4614) Slight API changes in Matrix and Matrices

2014-11-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226396#comment-14226396 ] Apache Spark commented on SPARK-4614: - User 'mengxr' has created a pull request for

[jira] [Created] (SPARK-4626) NoSuchElementException in CoarseGrainedSchedulerBackend

2014-11-26 Thread Victor Tso (JIRA)
Victor Tso created SPARK-4626: - Summary: NoSuchElementException in CoarseGrainedSchedulerBackend Key: SPARK-4626 URL: https://issues.apache.org/jira/browse/SPARK-4626 Project: Spark Issue Type:

[jira] [Commented] (SPARK-4626) NoSuchElementException in CoarseGrainedSchedulerBackend

2014-11-26 Thread Victor Tso (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226466#comment-14226466 ] Victor Tso commented on SPARK-4626: --- Apologies if Blocker is the wrong level. We're

[jira] [Updated] (SPARK-4626) NoSuchElementException in CoarseGrainedSchedulerBackend

2014-11-26 Thread Victor Tso (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Victor Tso updated SPARK-4626: -- Affects Version/s: (was: 1.1.0) 1.0.2 NoSuchElementException in

[jira] [Updated] (SPARK-4626) NoSuchElementException in CoarseGrainedSchedulerBackend

2014-11-26 Thread Victor Tso (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Victor Tso updated SPARK-4626: -- Priority: Major (was: Blocker) NoSuchElementException in CoarseGrainedSchedulerBackend

[jira] [Issue Comment Deleted] (SPARK-4626) NoSuchElementException in CoarseGrainedSchedulerBackend

2014-11-26 Thread Victor Tso (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Victor Tso updated SPARK-4626: -- Comment: was deleted (was: Apologies if Blocker is the wrong level. We're seeing this in production so

[jira] [Commented] (SPARK-4626) NoSuchElementException in CoarseGrainedSchedulerBackend

2014-11-26 Thread Victor Tso (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226480#comment-14226480 ] Victor Tso commented on SPARK-4626: --- It looks like a race where the KillTask message

[jira] [Commented] (SPARK-4584) 2x Performance regression for Spark-on-YARN

2014-11-26 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226506#comment-14226506 ] Marcelo Vanzin commented on SPARK-4584: --- [~sowen] that was going to be my

[jira] [Commented] (SPARK-4626) NoSuchElementException in CoarseGrainedSchedulerBackend

2014-11-26 Thread Victor Tso (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226513#comment-14226513 ] Victor Tso commented on SPARK-4626: --- https://github.com/apache/spark/pull/3483

[jira] [Commented] (SPARK-4626) NoSuchElementException in CoarseGrainedSchedulerBackend

2014-11-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226514#comment-14226514 ] Apache Spark commented on SPARK-4626: - User 'roxchkplusony' has created a pull request

[jira] [Commented] (SPARK-4001) Add Apriori algorithm to Spark MLlib

2014-11-26 Thread Jacky Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226562#comment-14226562 ] Jacky Li commented on SPARK-4001: - Thanks for your suggestion, Daniel. Here is the

[jira] [Commented] (SPARK-4584) 2x Performance regression for Spark-on-YARN

2014-11-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226637#comment-14226637 ] Apache Spark commented on SPARK-4584: - User 'vanzin' has created a pull request for

[jira] [Updated] (SPARK-4626) NoSuchElementException in CoarseGrainedSchedulerBackend

2014-11-26 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4626: - Assignee: Victor Tso NoSuchElementException in CoarseGrainedSchedulerBackend

[jira] [Created] (SPARK-4627) Too many TripletFields

2014-11-26 Thread Joseph E. Gonzalez (JIRA)
Joseph E. Gonzalez created SPARK-4627: - Summary: Too many TripletFields Key: SPARK-4627 URL: https://issues.apache.org/jira/browse/SPARK-4627 Project: Spark Issue Type: Bug

[jira] [Closed] (SPARK-4627) Too many TripletFields

2014-11-26 Thread Joseph E. Gonzalez (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph E. Gonzalez closed SPARK-4627. - Resolution: Fixed Fix Version/s: 1.2.0 Fixed in PR #3472. Too many TripletFields

[jira] [Commented] (SPARK-4627) Too many TripletFields

2014-11-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226652#comment-14226652 ] Apache Spark commented on SPARK-4627: - User 'jegonzal' has created a pull request for

[jira] [Created] (SPARK-4628) Put all external projects behind a build flag

2014-11-26 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-4628: -- Summary: Put all external projects behind a build flag Key: SPARK-4628 URL: https://issues.apache.org/jira/browse/SPARK-4628 Project: Spark Issue Type:

[jira] [Commented] (SPARK-4628) Put all external projects behind a build flag

2014-11-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226691#comment-14226691 ] Apache Spark commented on SPARK-4628: - User 'pwendell' has created a pull request for

[jira] [Resolved] (SPARK-4614) Slight API changes in Matrix and Matrices

2014-11-26 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-4614. -- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 3482

[jira] [Commented] (SPARK-2450) Provide link to YARN executor logs on UI

2014-11-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226706#comment-14226706 ] Apache Spark commented on SPARK-2450: - User 'ksakellis' has created a pull request for

[jira] [Resolved] (SPARK-4376) Put external modules behind build profiles

2014-11-26 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza resolved SPARK-4376. --- Resolution: Duplicate Put external modules behind build profiles

[jira] [Commented] (SPARK-4628) Put all external projects behind a build flag

2014-11-26 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226722#comment-14226722 ] Sandy Ryza commented on SPARK-4628: --- This looks like a duplicate of SPARK-4376.

[jira] [Commented] (SPARK-4001) Add Apriori algorithm to Spark MLlib

2014-11-26 Thread Daniel Erenrich (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226793#comment-14226793 ] Daniel Erenrich commented on SPARK-4001: First a minor point: I'd suggest making

[jira] [Created] (SPARK-4629) Spark SQL uses Hadoop Configuration in a thread-unsafe manner when writing Parquet files

2014-11-26 Thread Michael Allman (JIRA)
Michael Allman created SPARK-4629: - Summary: Spark SQL uses Hadoop Configuration in a thread-unsafe manner when writing Parquet files Key: SPARK-4629 URL: https://issues.apache.org/jira/browse/SPARK-4629

[jira] [Resolved] (SPARK-4583) GradientBoostedTrees error logging should use loss being minimized

2014-11-26 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-4583. -- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 3474

[jira] [Commented] (SPARK-2192) Examples Data Not in Binary Distribution

2014-11-26 Thread Pat McDonough (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226868#comment-14226868 ] Pat McDonough commented on SPARK-2192: -- Thanks [~srowen]. And yes, Xiangrui confirmed

[jira] [Created] (SPARK-4630) Dynamically determine optimal number of partitions

2014-11-26 Thread Kostas Sakellis (JIRA)
Kostas Sakellis created SPARK-4630: -- Summary: Dynamically determine optimal number of partitions Key: SPARK-4630 URL: https://issues.apache.org/jira/browse/SPARK-4630 Project: Spark Issue

[jira] [Updated] (SPARK-4630) Dynamically determine optimal number of partitions

2014-11-26 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-4630: -- Assignee: Kostas Sakellis Dynamically determine optimal number of partitions

[jira] [Commented] (SPARK-1503) Implement Nesterov's accelerated first-order method

2014-11-26 Thread Aaron Staple (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226958#comment-14226958 ] Aaron Staple commented on SPARK-1503: - [~rezazadeh] Thanks for your feedback. Your

[jira] [Commented] (SPARK-4452) Shuffle data structures can starve others on the same thread for memory

2014-11-26 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226969#comment-14226969 ] Sandy Ryza commented on SPARK-4452: --- Thinking about the current change a little more, an

[jira] [Comment Edited] (SPARK-1503) Implement Nesterov's accelerated first-order method

2014-11-26 Thread Aaron Staple (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226958#comment-14226958 ] Aaron Staple edited comment on SPARK-1503 at 11/26/14 11:09 PM:

[jira] [Comment Edited] (SPARK-1503) Implement Nesterov's accelerated first-order method

2014-11-26 Thread Aaron Staple (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226958#comment-14226958 ] Aaron Staple edited comment on SPARK-1503 at 11/26/14 11:10 PM:

[jira] [Comment Edited] (SPARK-1503) Implement Nesterov's accelerated first-order method

2014-11-26 Thread Aaron Staple (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226958#comment-14226958 ] Aaron Staple edited comment on SPARK-1503 at 11/26/14 11:14 PM:

[jira] [Commented] (SPARK-1503) Implement Nesterov's accelerated first-order method

2014-11-26 Thread Reza Zadeh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227023#comment-14227023 ] Reza Zadeh commented on SPARK-1503: --- Thanks Aaron. From an implementation perspective,

[jira] [Resolved] (SPARK-4046) Incorrect Java example on site

2014-11-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-4046. Resolution: Fixed Assignee: Reynold Xin Incorrect Java example on site

[jira] [Commented] (SPARK-4046) Incorrect Java example on site

2014-11-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227053#comment-14227053 ] Reynold Xin commented on SPARK-4046: Thanks - I pushed the change and updated the

[jira] [Created] (SPARK-4631) Add real unit test for MQTT

2014-11-26 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-4631: Summary: Add real unit test for MQTT Key: SPARK-4631 URL: https://issues.apache.org/jira/browse/SPARK-4631 Project: Spark Issue Type: Test

[jira] [Resolved] (SPARK-3628) Don't apply accumulator updates multiple times for tasks in result stages

2014-11-26 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-3628. -- Resolution: Fixed Fix Version/s: 1.2.0 Target Version/s: 1.1.2 (was: 0.9.3,

[jira] [Commented] (SPARK-3628) Don't apply accumulator updates multiple times for tasks in result stages

2014-11-26 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227077#comment-14227077 ] Matei Zaharia commented on SPARK-3628: -- FYI I merged this into 1.2.0, since the patch

[jira] [Commented] (SPARK-4631) Add real unit test for MQTT

2014-11-26 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227076#comment-14227076 ] Tathagata Das commented on SPARK-4631: -- [~prabeeshk] Could you please take a crack at

[jira] [Commented] (SPARK-4632) Upgrade MQTT dependency to use latest mqtt-client

2014-11-26 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227091#comment-14227091 ] Tathagata Das commented on SPARK-4632: -- This is a less scary alternative to the JIRA

[jira] [Commented] (SPARK-732) Recomputation of RDDs may result in duplicated accumulator updates

2014-11-26 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227108#comment-14227108 ] Matei Zaharia commented on SPARK-732: - As discussed on

[jira] [Reopened] (SPARK-3628) Don't apply accumulator updates multiple times for tasks in result stages

2014-11-26 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia reopened SPARK-3628: -- Don't apply accumulator updates multiple times for tasks in result stages

[jira] [Commented] (SPARK-4631) Add real unit test for MQTT

2014-11-26 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227121#comment-14227121 ] Tathagata Das commented on SPARK-4631: -- I am optimistically going ahead and upgrading

[jira] [Updated] (SPARK-4632) Upgrade MQTT dependency to use latest mqtt-client

2014-11-26 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-4632: - Issue Type: Improvement (was: Test) Upgrade MQTT dependency to use latest mqtt-client

[jira] [Created] (SPARK-4633) Support gzip in spark.compression.io.codec

2014-11-26 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-4633: --- Summary: Support gzip in spark.compression.io.codec Key: SPARK-4633 URL: https://issues.apache.org/jira/browse/SPARK-4633 Project: Spark Issue Type:

[jira] [Commented] (SPARK-4633) Support gzip in spark.compression.io.codec

2014-11-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227133#comment-14227133 ] Apache Spark commented on SPARK-4633: - User 'maropu' has created a pull request for

[jira] [Created] (SPARK-4634) Enable metrics for each application to be gathered in one node

2014-11-26 Thread Masayoshi TSUZUKI (JIRA)
Masayoshi TSUZUKI created SPARK-4634: Summary: Enable metrics for each application to be gathered in one node Key: SPARK-4634 URL: https://issues.apache.org/jira/browse/SPARK-4634 Project: Spark

[jira] [Commented] (SPARK-4634) Enable metrics for each application to be gathered in one node

2014-11-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227141#comment-14227141 ] Apache Spark commented on SPARK-4634: - User 'tsudukim' has created a pull request for

[jira] [Updated] (SPARK-3704) Short (TINYINT) incorrectly handled in thrift JDBC/ODBC server

2014-11-26 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-3704: Summary: Short (TINYINT) incorrectly handled in thrift JDBC/ODBC server (was: the types

[jira] [Commented] (SPARK-3995) [PYSPARK] PySpark's sample methods do not work with NumPy 1.9

2014-11-26 Thread Jeremy Freeman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227266#comment-14227266 ] Jeremy Freeman commented on SPARK-3995: --- Agree with [~mengxr], that's a better

[jira] [Commented] (SPARK-4630) Dynamically determine optimal number of partitions

2014-11-26 Thread Lianhui Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227280#comment-14227280 ] Lianhui Wang commented on SPARK-4630: - i am interesting in this. and we can learn some

[jira] [Comment Edited] (SPARK-4630) Dynamically determine optimal number of partitions

2014-11-26 Thread Lianhui Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227280#comment-14227280 ] Lianhui Wang edited comment on SPARK-4630 at 11/27/14 5:34 AM:

[jira] [Created] (SPARK-4635) Delete the val that never used in HashOuterJoin.

2014-11-26 Thread DoingDone9 (JIRA)
DoingDone9 created SPARK-4635: - Summary: Delete the val that never used in HashOuterJoin. Key: SPARK-4635 URL: https://issues.apache.org/jira/browse/SPARK-4635 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-4635) Delete the val that never used in HashOuterJoin.

2014-11-26 Thread DoingDone9 (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DoingDone9 updated SPARK-4635: -- Description: The val boundCondition is created in execute(),but it never be used in execute(); Delete

[jira] [Commented] (SPARK-4631) Add real unit test for MQTT

2014-11-26 Thread Prabeesh K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227309#comment-14227309 ] Prabeesh K commented on SPARK-4631: --- [~tdas] Please update to new

[jira] [Updated] (SPARK-4635) Delete the val that never used in execute() of HashOuterJoin.

2014-11-26 Thread DoingDone9 (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DoingDone9 updated SPARK-4635: -- Summary: Delete the val that never used in execute() of HashOuterJoin. (was: Delete the val that

[jira] [Updated] (SPARK-4635) Delete the val that never used in execute() of HashOuterJoin.

2014-11-26 Thread DoingDone9 (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DoingDone9 updated SPARK-4635: -- Description: The val boundCondition is created in execute(),but it never be used in execute(). (was:

[jira] [Commented] (SPARK-4170) Closure problems when running Scala app that extends App

2014-11-26 Thread Brennon York (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227319#comment-14227319 ] Brennon York commented on SPARK-4170: - [~srowen] this bug seems to be an issue with

[jira] [Comment Edited] (SPARK-4170) Closure problems when running Scala app that extends App

2014-11-26 Thread Brennon York (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227319#comment-14227319 ] Brennon York edited comment on SPARK-4170 at 11/27/14 6:48 AM:

[jira] [Commented] (SPARK-4635) Delete the val that never used in execute() of HashOuterJoin.

2014-11-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227321#comment-14227321 ] Apache Spark commented on SPARK-4635: - User 'DoingDone9' has created a pull request

[jira] [Commented] (SPARK-4632) Upgrade MQTT dependency to use latest mqtt-client

2014-11-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227326#comment-14227326 ] Apache Spark commented on SPARK-4632: - User 'prabeesh' has created a pull request for

[jira] [Commented] (SPARK-4632) Upgrade MQTT dependency to use latest mqtt-client

2014-11-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227325#comment-14227325 ] Apache Spark commented on SPARK-4632: - User 'prabeesh' has created a pull request for

[jira] [Commented] (SPARK-4632) Upgrade MQTT dependency to use latest mqtt-client

2014-11-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227324#comment-14227324 ] Apache Spark commented on SPARK-4632: - User 'prabeesh' has created a pull request for

[jira] [Comment Edited] (SPARK-4631) Add real unit test for MQTT

2014-11-26 Thread Prabeesh K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227309#comment-14227309 ] Prabeesh K edited comment on SPARK-4631 at 11/27/14 6:54 AM: -

[jira] [Commented] (SPARK-4631) Add real unit test for MQTT

2014-11-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227337#comment-14227337 ] Reynold Xin commented on SPARK-4631: How popular is MQTT? Do we really need this

[jira] [Comment Edited] (SPARK-4631) Add real unit test for MQTT

2014-11-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227337#comment-14227337 ] Reynold Xin edited comment on SPARK-4631 at 11/27/14 7:11 AM: --

[jira] [Closed] (SPARK-4599) add hive profile to root pom

2014-11-26 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang closed SPARK-4599. -- Resolution: Won't Fix add hive profile to root pom

[jira] [Commented] (SPARK-4628) Put all external projects behind a build flag

2014-11-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227339#comment-14227339 ] Reynold Xin commented on SPARK-4628: Shall we also not depend on artifacts that are

[jira] [Commented] (SPARK-2429) Hierarchical Implementation of KMeans

2014-11-26 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227352#comment-14227352 ] Yu Ishikawa commented on SPARK-2429: Hi [~yangjunpro], {quote} 1. In your

[jira] [Commented] (SPARK-4631) Add real unit test for MQTT

2014-11-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227356#comment-14227356 ] Sean Owen commented on SPARK-4631: -- (FWIW I had never heard of MQTT before Spark. This

  1   2   >