[jira] [Created] (SPARK-11299) SQL Programming Guide's link to DataFrame Function Reference is wrong

2015-10-25 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-11299: -- Summary: SQL Programming Guide's link to DataFrame Function Reference is wrong Key: SPARK-11299 URL: https://issues.apache.org/jira/browse/SPARK-11299 Project: Spark

[jira] [Assigned] (SPARK-9162) Implement code generation for ScalaUDF

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9162: --- Assignee: Apache Spark > Implement code generation for ScalaUDF >

[jira] [Assigned] (SPARK-9162) Implement code generation for ScalaUDF

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9162: --- Assignee: (was: Apache Spark) > Implement code generation for ScalaUDF >

[jira] [Commented] (SPARK-11298) When driver sends message "GetExecutorLossReason" to AM, the AM stops.

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973081#comment-14973081 ] Apache Spark commented on SPARK-11298: -- User 'KaiXinXiaoLei' has created a pull request for this

[jira] [Assigned] (SPARK-11298) When driver sends message "GetExecutorLossReason" to AM, the AM stops.

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11298: Assignee: Apache Spark > When driver sends message "GetExecutorLossReason" to AM, the AM

[jira] [Updated] (SPARK-11298) When driver sends message "GetExecutorLossReason" to AM, the AM stops.

2015-10-25 Thread KaiXinXIaoLei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-11298: -- Component/s: YARN > When driver sends message "GetExecutorLossReason" to AM, the AM stops. >

[jira] [Assigned] (SPARK-11298) When driver sends message "GetExecutorLossReason" to AM, the AM stops.

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11298: Assignee: (was: Apache Spark) > When driver sends message "GetExecutorLossReason" to

[jira] [Commented] (SPARK-11299) SQL Programming Guide's link to DataFrame Function Reference is wrong

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973108#comment-14973108 ] Apache Spark commented on SPARK-11299: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-11299) SQL Programming Guide's link to DataFrame Function Reference is wrong

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11299: Assignee: Apache Spark (was: Josh Rosen) > SQL Programming Guide's link to DataFrame

[jira] [Assigned] (SPARK-11299) SQL Programming Guide's link to DataFrame Function Reference is wrong

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11299: Assignee: Josh Rosen (was: Apache Spark) > SQL Programming Guide's link to DataFrame

[jira] [Commented] (SPARK-9162) Implement code generation for ScalaUDF

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973130#comment-14973130 ] Apache Spark commented on SPARK-9162: - User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-11239) PMML export for ML linear regression

2015-10-25 Thread Kai Sasaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973214#comment-14973214 ] Kai Sasaki commented on SPARK-11239: [~holdenk] Hi, these tickets under SPARK-11171 are blocked by

[jira] [Commented] (SPARK-10386) Model import/export for PrefixSpan

2015-10-25 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973234#comment-14973234 ] Yanbo Liang commented on SPARK-10386: - This is partly depends on SPARK-6724 which we need to figure

[jira] [Comment Edited] (SPARK-6724) Model import/export for FPGrowth

2015-10-25 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973222#comment-14973222 ] Yanbo Liang edited comment on SPARK-6724 at 10/25/15 12:51 PM: --- [~josephkb]

[jira] [Commented] (SPARK-6333) saveAsObjectFile support for compression codec

2015-10-25 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-6333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973232#comment-14973232 ] Maciej Bryński commented on SPARK-6333: --- [~srowen] I'd very like to have this functionality.

[jira] [Commented] (SPARK-6724) Model import/export for FPGrowth

2015-10-25 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973222#comment-14973222 ] Yanbo Liang commented on SPARK-6724: [~josephkb] Now we can save FPGrowthModel with arbitrary types

[jira] [Updated] (SPARK-10562) .partitionBy() creates the metastore partition columns in all lowercase, but persists the data path as MixedCase resulting in an error when the data is later attempted t

2015-10-25 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-10562: --- Description: When using DataFrame.write.partitionBy().saveAsTable() it creates the partiton by

[jira] [Created] (SPARK-11300) Support for string length when writing to JDBC

2015-10-25 Thread JIRA
Maciej Bryński created SPARK-11300: -- Summary: Support for string length when writing to JDBC Key: SPARK-11300 URL: https://issues.apache.org/jira/browse/SPARK-11300 Project: Spark Issue

[jira] [Commented] (SPARK-11234) What's cooking classification

2015-10-25 Thread Kai Sasaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973221#comment-14973221 ] Kai Sasaki commented on SPARK-11234: [~xusen] Thank you so much for very insightful experiments!

[jira] [Comment Edited] (SPARK-10386) Model import/export for PrefixSpan

2015-10-25 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973234#comment-14973234 ] Yanbo Liang edited comment on SPARK-10386 at 10/25/15 1:21 PM: --- This is

[jira] [Resolved] (SPARK-10891) Add MessageHandler to KinesisUtils.createStream similar to Direct Kafka

2015-10-25 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-10891. --- Resolution: Fixed Assignee: Burak Yavuz Fix Version/s: 1.6.0 > Add

[jira] [Assigned] (SPARK-11306) Executor JVM loss can lead to a hang in Standalone mode

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11306: Assignee: Kay Ousterhout (was: Apache Spark) > Executor JVM loss can lead to a hang in

[jira] [Assigned] (SPARK-11306) Executor JVM loss can lead to a hang in Standalone mode

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11306: Assignee: Apache Spark (was: Kay Ousterhout) > Executor JVM loss can lead to a hang in

[jira] [Commented] (SPARK-7106) Support model save/load in Python's FPGrowth

2015-10-25 Thread Kai Jiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973525#comment-14973525 ] Kai Jiang commented on SPARK-7106: -- I would like to do this one after spark-6724 is done. > Support

[jira] [Comment Edited] (SPARK-8890) Reduce memory consumption for dynamic partition insert

2015-10-25 Thread Jerry Lam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973529#comment-14973529 ] Jerry Lam edited comment on SPARK-8890 at 10/26/15 1:02 AM: Hi guys, sorry by

[jira] [Commented] (SPARK-10500) sparkr.zip cannot be created if $SPARK_HOME/R/lib is unwritable

2015-10-25 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973530#comment-14973530 ] Sun Rui commented on SPARK-10500: - yes, I am working on this > sparkr.zip cannot be created if

[jira] [Resolved] (SPARK-11127) Upgrade Kinesis Client Library to the latest stable version

2015-10-25 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-11127. --- Resolution: Fixed Fix Version/s: 1.6.0 > Upgrade Kinesis Client Library to the latest

[jira] [Resolved] (SPARK-11304) SparkR in yarn-client mode fails creating sparkr.zip

2015-10-25 Thread Ram Venkatesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ram Venkatesh resolved SPARK-11304. --- Resolution: Duplicate Same as SPARK-10500 > SparkR in yarn-client mode fails creating

[jira] [Commented] (SPARK-5737) Scanning duplicate columns from parquet table

2015-10-25 Thread Kevin Jung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973523#comment-14973523 ] Kevin Jung commented on SPARK-5737: --- Based on your comment, It must be marked as resolved. Thanks. >

[jira] [Comment Edited] (SPARK-8890) Reduce memory consumption for dynamic partition insert

2015-10-25 Thread Jerry Lam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973529#comment-14973529 ] Jerry Lam edited comment on SPARK-8890 at 10/26/15 12:58 AM: - Hi guys, sorry

[jira] [Assigned] (SPARK-11307) Reduce memory consumption of OutputCommitCoordinator bookkeeping structures

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11307: Assignee: Apache Spark (was: Josh Rosen) > Reduce memory consumption of

[jira] [Commented] (SPARK-11307) Reduce memory consumption of OutputCommitCoordinator bookkeeping structures

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973564#comment-14973564 ] Apache Spark commented on SPARK-11307: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-11307) Reduce memory consumption of OutputCommitCoordinator bookkeeping structures

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11307: Assignee: Josh Rosen (was: Apache Spark) > Reduce memory consumption of

[jira] [Assigned] (SPARK-10286) Add @since annotation to pyspark.ml.param and pyspark.ml.*

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10286: Assignee: (was: Apache Spark) > Add @since annotation to pyspark.ml.param and

[jira] [Commented] (SPARK-10286) Add @since annotation to pyspark.ml.param and pyspark.ml.*

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973651#comment-14973651 ] Apache Spark commented on SPARK-10286: -- User 'lidinghao' has created a pull request for this issue:

[jira] [Created] (SPARK-11308) Change spark streaming's job scheduler logic to ensuer guaranteed order of batch processing

2015-10-25 Thread Renjie Liu (JIRA)
Renjie Liu created SPARK-11308: -- Summary: Change spark streaming's job scheduler logic to ensuer guaranteed order of batch processing Key: SPARK-11308 URL: https://issues.apache.org/jira/browse/SPARK-11308

[jira] [Assigned] (SPARK-10286) Add @since annotation to pyspark.ml.param and pyspark.ml.*

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10286: Assignee: Apache Spark > Add @since annotation to pyspark.ml.param and pyspark.ml.* >

[jira] [Commented] (SPARK-11305) Remove Third-Party Hadoop Distributions Doc Page

2015-10-25 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973493#comment-14973493 ] Patrick Wendell commented on SPARK-11305: - /cc [~srowen] for his thoughts. > Remove Third-Party

[jira] [Created] (SPARK-11305) Remove Third-Party Hadoop Distributions Doc Page

2015-10-25 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-11305: --- Summary: Remove Third-Party Hadoop Distributions Doc Page Key: SPARK-11305 URL: https://issues.apache.org/jira/browse/SPARK-11305 Project: Spark Issue

[jira] [Created] (SPARK-11306) Executor JVM loss can lead to a hang in Standalone mode

2015-10-25 Thread Kay Ousterhout (JIRA)
Kay Ousterhout created SPARK-11306: -- Summary: Executor JVM loss can lead to a hang in Standalone mode Key: SPARK-11306 URL: https://issues.apache.org/jira/browse/SPARK-11306 Project: Spark

[jira] [Resolved] (SPARK-5737) Scanning duplicate columns from parquet table

2015-10-25 Thread Kevin Jung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Jung resolved SPARK-5737. --- Resolution: Fixed Fix Version/s: 1.5.1 > Scanning duplicate columns from parquet table >

[jira] [Created] (SPARK-11307) Reduce memory consumption of OutputCommitCoordinator bookkeeping structures

2015-10-25 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-11307: -- Summary: Reduce memory consumption of OutputCommitCoordinator bookkeeping structures Key: SPARK-11307 URL: https://issues.apache.org/jira/browse/SPARK-11307 Project:

[jira] [Updated] (SPARK-11294) Improve R doc for read.df, write.df, saveAsTable

2015-10-25 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-11294: -- Fix Version/s: (was: 1.5.2) 1.5.3 > Improve R doc for

[jira] [Commented] (SPARK-11300) Support for string length when writing to JDBC

2015-10-25 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973707#comment-14973707 ] Josh Rosen commented on SPARK-11300: I think that this duplicates SPARK-10101 > Support for string

[jira] [Commented] (SPARK-10500) sparkr.zip cannot be created if $SPARK_HOME/R/lib is unwritable

2015-10-25 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973444#comment-14973444 ] Felix Cheung commented on SPARK-10500: -- [~sunrui]suggestion 08/Sept makes sense. Would you like to

[jira] [Commented] (SPARK-11306) Executor JVM loss can lead to a hang in Standalone mode

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973506#comment-14973506 ] Apache Spark commented on SPARK-11306: -- User 'kayousterhout' has created a pull request for this

[jira] [Commented] (SPARK-10971) sparkR: RRunner should allow setting path to Rscript

2015-10-25 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973510#comment-14973510 ] Patrick Wendell commented on SPARK-10971: - Reynold has sent out the vote email based on the

[jira] [Comment Edited] (SPARK-10971) sparkR: RRunner should allow setting path to Rscript

2015-10-25 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973510#comment-14973510 ] Patrick Wendell edited comment on SPARK-10971 at 10/26/15 12:02 AM:

[jira] [Updated] (SPARK-10971) sparkR: RRunner should allow setting path to Rscript

2015-10-25 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-10971: Fix Version/s: (was: 1.5.2) 1.5.3 > sparkR: RRunner should allow

[jira] [Updated] (SPARK-11308) Change spark streaming's job scheduler logic to ensuer guaranteed order of batch processing

2015-10-25 Thread Renjie Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Renjie Liu updated SPARK-11308: --- Description: In current implementation, spark streaming uses a thread pool to run jobs generated in

[jira] [Resolved] (SPARK-10984) Simplify *MemoryManager class structure

2015-10-25 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-10984. Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9127

[jira] [Created] (SPARK-11309) Clean up hacky use of MemoryManager inside of HashedRelation

2015-10-25 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-11309: -- Summary: Clean up hacky use of MemoryManager inside of HashedRelation Key: SPARK-11309 URL: https://issues.apache.org/jira/browse/SPARK-11309 Project: Spark

[jira] [Commented] (SPARK-8890) Reduce memory consumption for dynamic partition insert

2015-10-25 Thread Jerry Lam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973529#comment-14973529 ] Jerry Lam commented on SPARK-8890: -- Hi guys, sorry by injecting comments into the closed jira. I just

[jira] [Commented] (SPARK-8597) DataFrame partitionBy memory pressure scales extremely poorly

2015-10-25 Thread Jerry Lam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973531#comment-14973531 ] Jerry Lam commented on SPARK-8597: -- FYI ... The solution described here solves the problem of memory

[jira] [Commented] (SPARK-11234) What's cooking classification

2015-10-25 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973674#comment-14973674 ] Xusen Yin commented on SPARK-11234: --- The last comment is based on my trial on Avito dataset

[jira] [Updated] (SPARK-11253) reset all accumulators in physical operators before execute an action

2015-10-25 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-11253: - Assignee: Wenchen Fan > reset all accumulators in physical operators before execute an action >

[jira] [Resolved] (SPARK-11253) reset all accumulators in physical operators before execute an action

2015-10-25 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-11253. -- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9215

[jira] [Commented] (SPARK-9861) Join: Determine the number of reducers used by a shuffle join operator at runtime

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973784#comment-14973784 ] Apache Spark commented on SPARK-9861: - User 'yhuai' has created a pull request for this issue:

[jira] [Commented] (SPARK-9858) Introduce an ExchangeCoordinator to estimate the number of post-shuffle partitions.

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973782#comment-14973782 ] Apache Spark commented on SPARK-9858: - User 'yhuai' has created a pull request for this issue:

[jira] [Assigned] (SPARK-9859) Aggregation: Determine the number of reducers at runtime

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9859: --- Assignee: Apache Spark (was: Yin Huai) > Aggregation: Determine the number of reducers at

[jira] [Assigned] (SPARK-9861) Join: Determine the number of reducers used by a shuffle join operator at runtime

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9861: --- Assignee: Yin Huai (was: Apache Spark) > Join: Determine the number of reducers used by a

[jira] [Assigned] (SPARK-9858) Introduce an ExchangeCoordinator to estimate the number of post-shuffle partitions.

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9858: --- Assignee: Yin Huai (was: Apache Spark) > Introduce an ExchangeCoordinator to estimate the

[jira] [Created] (SPARK-11304) SparkR in yarn-client mode fails creating sparkr.zip

2015-10-25 Thread Ram Venkatesh (JIRA)
Ram Venkatesh created SPARK-11304: - Summary: SparkR in yarn-client mode fails creating sparkr.zip Key: SPARK-11304 URL: https://issues.apache.org/jira/browse/SPARK-11304 Project: Spark Issue

[jira] [Updated] (SPARK-11308) Change spark streaming's job scheduler logic to ensuer guaranteed order of batch processing

2015-10-25 Thread Renjie Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Renjie Liu updated SPARK-11308: --- Priority: Major (was: Minor) > Change spark streaming's job scheduler logic to ensuer guaranteed

[jira] [Commented] (SPARK-11206) Support SQL UI on the history server

2015-10-25 Thread Carson Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973700#comment-14973700 ] Carson Wang commented on SPARK-11206: - For the live SQL UI, the SQLContext is responsible for

[jira] [Created] (SPARK-11310) only build spark core,Modify spark pom file:delete graphx

2015-10-25 Thread yindu_asan (JIRA)
yindu_asan created SPARK-11310: -- Summary: only build spark core,Modify spark pom file:delete graphx Key: SPARK-11310 URL: https://issues.apache.org/jira/browse/SPARK-11310 Project: Spark

[jira] [Commented] (SPARK-7146) Should ML sharedParams be a public API?

2015-10-25 Thread Kai Sasaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973745#comment-14973745 ] Kai Sasaki commented on SPARK-7146: --- There are several times when I want to use internal resources(e.g.

[jira] [Commented] (SPARK-9858) Introduce a AdaptiveExchange operator and add it in the query planner.

2015-10-25 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973772#comment-14973772 ] Yin Huai commented on SPARK-9858: - Instead of having an {{AdaptiveExchange}}, we will have an

[jira] [Updated] (SPARK-9858) Introduce an ExchangeCoordinator to estimate the number of post-shuffle partitions.

2015-10-25 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-9858: Summary: Introduce an ExchangeCoordinator to estimate the number of post-shuffle partitions. (was:

[jira] [Created] (SPARK-11302) Multivariate Gaussian Model with Covariance matrix return zero always

2015-10-25 Thread eyal sharon (JIRA)
eyal sharon created SPARK-11302: --- Summary: Multivariate Gaussian Model with Covariance matrix return zero always Key: SPARK-11302 URL: https://issues.apache.org/jira/browse/SPARK-11302 Project:

[jira] [Commented] (SPARK-10181) HiveContext is not used with keytab principal but with user principal/unix username

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973423#comment-14973423 ] Apache Spark commented on SPARK-10181: -- User 'yolandagao' has created a pull request for this issue:

[jira] [Commented] (SPARK-11154) make specificaition spark.yarn.executor.memoryOverhead consistent with typical JVM options

2015-10-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973322#comment-14973322 ] Sean Owen commented on SPARK-11154: --- I think that if this is done at all, it would have to be with a

[jira] [Commented] (SPARK-11302) Multivariate Gaussian Model with Covariance matrix return zero always

2015-10-25 Thread eyal sharon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973325#comment-14973325 ] eyal sharon commented on SPARK-11302: - Hi Sean, Thanks for your reply. I will try to add more info

[jira] [Commented] (SPARK-6333) saveAsObjectFile support for compression codec

2015-10-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973252#comment-14973252 ] Sean Owen commented on SPARK-6333: -- See the pull request. There are some decent reasons that this

[jira] [Created] (SPARK-11301) filter on partitioned column is case sensitive even the context is case insensitive

2015-10-25 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-11301: --- Summary: filter on partitioned column is case sensitive even the context is case insensitive Key: SPARK-11301 URL: https://issues.apache.org/jira/browse/SPARK-11301

[jira] [Assigned] (SPARK-11301) filter on partitioned column is case sensitive even the context is case insensitive

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11301: Assignee: (was: Apache Spark) > filter on partitioned column is case sensitive even

[jira] [Commented] (SPARK-11301) filter on partitioned column is case sensitive even the context is case insensitive

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973297#comment-14973297 ] Apache Spark commented on SPARK-11301: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-11301) filter on partitioned column is case sensitive even the context is case insensitive

2015-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11301: Assignee: Apache Spark > filter on partitioned column is case sensitive even the context

[jira] [Commented] (SPARK-11302) Multivariate Gaussian Model with Covariance matrix return zero always

2015-10-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973321#comment-14973321 ] Sean Owen commented on SPARK-11302: --- It's not clear what you're trying to report. What code are you

[jira] [Resolved] (SPARK-10994) Clustering coefficient computation in GraphX

2015-10-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-10994. --- Resolution: Won't Fix > Clustering coefficient computation in GraphX >

[jira] [Resolved] (SPARK-11287) Executing deploy.client TestClient fails with bad class name

2015-10-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11287. --- Resolution: Fixed Fix Version/s: 1.6.0 1.5.3 Issue resolved by pull

[jira] [Updated] (SPARK-11287) Executing deploy.client TestClient fails with bad class name

2015-10-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11287: -- Assignee: Bryan Cutler > Executing deploy.client TestClient fails with bad class name >

[jira] [Created] (SPARK-11303) sample (without replacement) + filter returns wrong results in DataFrame

2015-10-25 Thread Yuval Tanny (JIRA)
Yuval Tanny created SPARK-11303: --- Summary: sample (without replacement) + filter returns wrong results in DataFrame Key: SPARK-11303 URL: https://issues.apache.org/jira/browse/SPARK-11303 Project: