[jira] [Commented] (SPARK-13331) Spark network encryption optimization

2016-02-17 Thread Dong Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151892#comment-15151892 ] Dong Chen commented on SPARK-13331: --- Sorry for the confusion, below is the change would entail in

[jira] [Commented] (SPARK-12449) Pushing down arbitrary logical plans to data sources

2016-02-17 Thread Max Seiden (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151880#comment-15151880 ] Max Seiden commented on SPARK-12449: Yea, that seems to be the case. There's code in the

[jira] [Created] (SPARK-13373) Generate code for sort merge join

2016-02-17 Thread Davies Liu (JIRA)
Davies Liu created SPARK-13373: -- Summary: Generate code for sort merge join Key: SPARK-13373 URL: https://issues.apache.org/jira/browse/SPARK-13373 Project: Spark Issue Type: New Feature

[jira] [Closed] (SPARK-13354) Push filter throughout outer join when the condition can filter out empty row

2016-02-17 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-13354. -- Resolution: Duplicate > Push filter throughout outer join when the condition can filter out empty row

[jira] [Comment Edited] (SPARK-13370) Lexer not handling whitespaces properly

2016-02-17 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151822#comment-15151822 ] Herman van Hovell edited comment on SPARK-13370 at 2/18/16 7:35 AM:

[jira] [Commented] (SPARK-13370) Lexer not handling whitespaces properly

2016-02-17 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151870#comment-15151870 ] Herman van Hovell commented on SPARK-13370: --- Thought about this a bit more, and realized we can

[jira] [Updated] (SPARK-13371) TaskSetManager.dequeueSpeculativeTask compares Option[String] and String directly.

2016-02-17 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-13371: Summary: TaskSetManager.dequeueSpeculativeTask compares Option[String] and String directly. (was:

[jira] [Updated] (SPARK-13371) Compare Option[String] and String directly in

2016-02-17 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-13371: Summary: Compare Option[String] and String directly in (was: Compare Option[String] and String

[jira] [Updated] (SPARK-13371) Compare Option[String] and String directly

2016-02-17 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-13371: Description: {noformat} TaskSetManager.dequeueSpeculativeTask compares Option[String] and String

[jira] [Updated] (SPARK-13331) Spark network encryption optimization

2016-02-17 Thread Dong Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Chen updated SPARK-13331: -- Description: In network/common, SASL with DIGEST­-MD5 authentication is used for negotiating a secure

[jira] [Commented] (SPARK-12449) Pushing down arbitrary logical plans to data sources

2016-02-17 Thread Evan Chan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151823#comment-15151823 ] Evan Chan commented on SPARK-12449: --- I think in the case of sources.Expressions, by the time they are

[jira] [Commented] (SPARK-13372) ML LogisticRegression behaves incorrectly when standardization = false && regParam = 0.0

2016-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151824#comment-15151824 ] Apache Spark commented on SPARK-13372: -- User 'yanboliang' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13372) ML LogisticRegression behaves incorrectly when standardization = false && regParam = 0.0

2016-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13372: Assignee: (was: Apache Spark) > ML LogisticRegression behaves incorrectly when

[jira] [Assigned] (SPARK-13372) ML LogisticRegression behaves incorrectly when standardization = false && regParam = 0.0

2016-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13372: Assignee: Apache Spark > ML LogisticRegression behaves incorrectly when standardization =

[jira] [Comment Edited] (SPARK-13370) Lexer not handling whitespaces properly

2016-02-17 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151822#comment-15151822 ] Herman van Hovell edited comment on SPARK-13370 at 2/18/16 6:58 AM:

[jira] [Commented] (SPARK-13370) Lexer not handling whitespaces properly

2016-02-17 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151822#comment-15151822 ] Herman van Hovell commented on SPARK-13370: --- Whitespace is optional. This may sound funny, but

[jira] [Comment Edited] (SPARK-2090) spark-shell input text entry not showing on REPL

2016-02-17 Thread Lantao Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151802#comment-15151802 ] Lantao Jin edited comment on SPARK-2090 at 2/18/16 6:36 AM: Richard is right,

[jira] [Comment Edited] (SPARK-2090) spark-shell input text entry not showing on REPL

2016-02-17 Thread Lantao Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151802#comment-15151802 ] Lantao Jin edited comment on SPARK-2090 at 2/18/16 6:35 AM: Richard is right,

[jira] [Created] (SPARK-13372) ML LogisticRegression behaves incorrectly when standardization = false && regParam = 0.0

2016-02-17 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-13372: --- Summary: ML LogisticRegression behaves incorrectly when standardization = false && regParam = 0.0 Key: SPARK-13372 URL: https://issues.apache.org/jira/browse/SPARK-13372

[jira] [Commented] (SPARK-2090) spark-shell input text entry not showing on REPL

2016-02-17 Thread Lantao Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151802#comment-15151802 ] Lantao Jin commented on SPARK-2090: --- Richard is right, this is the permissions problem on home

[jira] [Created] (SPARK-13371) Compare Option[String] and String directly

2016-02-17 Thread Guoqiang Li (JIRA)
Guoqiang Li created SPARK-13371: --- Summary: Compare Option[String] and String directly Key: SPARK-13371 URL: https://issues.apache.org/jira/browse/SPARK-13371 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-13333) DataFrame filter + randn + unionAll has bad interaction

2016-02-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151746#comment-15151746 ] Xiao Li edited comment on SPARK-1 at 2/18/16 5:15 AM: -- Yeah, you are right.

[jira] [Commented] (SPARK-13333) DataFrame filter + randn + unionAll has bad interaction

2016-02-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151746#comment-15151746 ] Xiao Li commented on SPARK-1: - Yeah, you are right. This part is an issue. That is why I did not

[jira] [Created] (SPARK-13370) Lexer not handling whitespaces properly

2016-02-17 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-13370: -- Summary: Lexer not handling whitespaces properly Key: SPARK-13370 URL: https://issues.apache.org/jira/browse/SPARK-13370 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-13369) Number of consecutive fetch failures for a stage before the job is aborted should be configurable

2016-02-17 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-13369: --- Summary: Number of consecutive fetch failures for a stage before the job is aborted should be configurable Key: SPARK-13369 URL: https://issues.apache.org/jira/browse/SPARK-13369

[jira] [Comment Edited] (SPARK-13333) DataFrame filter + randn + unionAll has bad interaction

2016-02-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151720#comment-15151720 ] Liang-Chi Hsieh edited comment on SPARK-1 at 2/18/16 4:13 AM: -- The

[jira] [Commented] (SPARK-13333) DataFrame filter + randn + unionAll has bad interaction

2016-02-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151720#comment-15151720 ] Liang-Chi Hsieh commented on SPARK-1: - Yes. I agree that when user provides a specific seed

[jira] [Commented] (SPARK-13364) history server application column not sorting properly

2016-02-17 Thread Zhuo Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151721#comment-15151721 ] Zhuo Liu commented on SPARK-13364: -- It is not sorting by , but a lexicographical sorting according to

[jira] [Commented] (SPARK-13368) PySpark JavaModel fails to extract params from Spark side automatically

2016-02-17 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151710#comment-15151710 ] Xusen Yin commented on SPARK-13368: --- FYI [~mengxr] [~josephkb] > PySpark JavaModel fails to extract

[jira] [Updated] (SPARK-13368) PySpark JavaModel fails to extract params from Spark side automatically

2016-02-17 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xusen Yin updated SPARK-13368: -- Description: JavaModel fails to extract params from Spark side automatically that causes

[jira] [Updated] (SPARK-13368) PySpark JavaModel fails to extract params from Spark side automatically

2016-02-17 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xusen Yin updated SPARK-13368: -- Description: JavaModel fails to extract params from Spark side automatically that causes

[jira] [Created] (SPARK-13368) PySpark JavaModel fails to extract params from Spark side automatically

2016-02-17 Thread Xusen Yin (JIRA)
Xusen Yin created SPARK-13368: - Summary: PySpark JavaModel fails to extract params from Spark side automatically Key: SPARK-13368 URL: https://issues.apache.org/jira/browse/SPARK-13368 Project: Spark

[jira] [Updated] (SPARK-13368) PySpark JavaModel fails to extract params from Spark side automatically

2016-02-17 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xusen Yin updated SPARK-13368: -- Priority: Minor (was: Major) > PySpark JavaModel fails to extract params from Spark side

[jira] [Resolved] (SPARK-13324) Update plugin, test, example dependencies for 2.x

2016-02-17 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-13324. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11206

[jira] [Updated] (SPARK-13360) pyspark related enviroment variable is not propagated to driver in yarn-cluster mode

2016-02-17 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated SPARK-13360: --- Description: Such as PYSPARK_DRIVER_PYTHON, PYSPARK_PYTHON, PYTHONHASHSEED. > pyspark related

[jira] [Updated] (SPARK-13360) pyspark related enviroment variable is not propagated to driver in yarn-cluster mode

2016-02-17 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated SPARK-13360: --- Summary: pyspark related enviroment variable is not propagated to driver in yarn-cluster mode (was:

[jira] [Updated] (SPARK-13363) Aggregator not working with DataFrame

2016-02-17 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-13363: - Priority: Blocker (was: Minor) > Aggregator not working with DataFrame >

[jira] [Updated] (SPARK-13363) Aggregator not working with DataFrame

2016-02-17 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-13363: - Affects Version/s: (was: 2.0.0) 1.6.0 Target Version/s:

[jira] [Comment Edited] (SPARK-13183) Bytebuffers occupy a large amount of heap memory

2016-02-17 Thread dylanzhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151604#comment-15151604 ] dylanzhou edited comment on SPARK-13183 at 2/18/16 2:33 AM: [~srowen] i donot

[jira] [Issue Comment Deleted] (SPARK-13183) Bytebuffers occupy a large amount of heap memory

2016-02-17 Thread dylanzhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dylanzhou updated SPARK-13183: -- Comment: was deleted (was: @Sean Owen maybe is a memory leak problem, and finally will run out of

[jira] [Commented] (SPARK-13183) Bytebuffers occupy a large amount of heap memory

2016-02-17 Thread dylanzhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151604#comment-15151604 ] dylanzhou commented on SPARK-13183: --- @Sean Owen maybe is a memory leak problem, and finally will run

[jira] [Commented] (SPARK-10001) Allow Ctrl-C in spark-shell to kill running job

2016-02-17 Thread Jon Maurer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151591#comment-15151591 ] Jon Maurer commented on SPARK-10001: I have a number of users who would find this feature to be

[jira] [Commented] (SPARK-6263) Python MLlib API missing items: Utils

2016-02-17 Thread Bruno Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151508#comment-15151508 ] Bruno Wu commented on SPARK-6263: - kFold function is still not available in util.py (as far as I can see).

[jira] [Commented] (SPARK-13367) Refactor KinesisUtils to specify more KCL options

2016-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151485#comment-15151485 ] Apache Spark commented on SPARK-13367: -- User 'addisonj' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13367) Refactor KinesisUtils to specify more KCL options

2016-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13367: Assignee: Apache Spark > Refactor KinesisUtils to specify more KCL options >

[jira] [Assigned] (SPARK-13367) Refactor KinesisUtils to specify more KCL options

2016-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13367: Assignee: (was: Apache Spark) > Refactor KinesisUtils to specify more KCL options >

[jira] [Resolved] (SPARK-13344) Tests have many "accumulator not found" exceptions

2016-02-17 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-13344. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11222

[jira] [Created] (SPARK-13367) Refactor KinesisUtils to specify more KCL options

2016-02-17 Thread Addison Higham (JIRA)
Addison Higham created SPARK-13367: -- Summary: Refactor KinesisUtils to specify more KCL options Key: SPARK-13367 URL: https://issues.apache.org/jira/browse/SPARK-13367 Project: Spark Issue

[jira] [Commented] (SPARK-2541) Standalone mode can't access secure HDFS anymore

2016-02-17 Thread Henry Saputra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151406#comment-15151406 ] Henry Saputra commented on SPARK-2541: -- Based on discussion on

[jira] [Resolved] (SPARK-12953) RDDRelation write set mode will be better to avoid error "pair.parquet already exists"

2016-02-17 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-12953. Resolution: Fixed Fix Version/s: 2.0.0 Fixed by PR for 2.0.0. > RDDRelation write set mode

[jira] [Updated] (SPARK-12953) RDDRelation write set mode will be better to avoid error "pair.parquet already exists"

2016-02-17 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-12953: --- Assignee: shijinkui > RDDRelation write set mode will be better to avoid error "pair.parquet >

[jira] [Reopened] (SPARK-12953) RDDRelation write set mode will be better to avoid error "pair.parquet already exists"

2016-02-17 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reopened SPARK-12953: > RDDRelation write set mode will be better to avoid error "pair.parquet > already exists" >

[jira] [Resolved] (SPARK-13109) SBT publishLocal failed to publish to local ivy repo

2016-02-17 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-13109. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11001

[jira] [Updated] (SPARK-13109) SBT publishLocal failed to publish to local ivy repo

2016-02-17 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-13109: --- Assignee: Saisai Shao > SBT publishLocal failed to publish to local ivy repo >

[jira] [Commented] (SPARK-12449) Pushing down arbitrary logical plans to data sources

2016-02-17 Thread Max Seiden (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151368#comment-15151368 ] Max Seiden commented on SPARK-12449: Very interested in checking out that PR! It would be prudent to

[jira] [Commented] (SPARK-12449) Pushing down arbitrary logical plans to data sources

2016-02-17 Thread Evan Chan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151345#comment-15151345 ] Evan Chan commented on SPARK-12449: --- [~stephank85] would you have any code to share? :D > Pushing

[jira] [Commented] (SPARK-12449) Pushing down arbitrary logical plans to data sources

2016-02-17 Thread Stephan Kessler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151339#comment-15151339 ] Stephan Kessler commented on SPARK-12449: - [~maxseiden] good idea! In order to simplify things

[jira] [Updated] (SPARK-13366) Support Cartesian join for Datasets

2016-02-17 Thread Xiu (Joe) Guo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiu (Joe) Guo updated SPARK-13366: -- Description: Saw a comment from [~marmbrus] regarding Cartesian join for Datasets: "You will

[jira] [Commented] (SPARK-13366) Support Cartesian join for Datasets

2016-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151331#comment-15151331 ] Apache Spark commented on SPARK-13366: -- User 'xguo27' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13366) Support Cartesian join for Datasets

2016-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13366: Assignee: Apache Spark > Support Cartesian join for Datasets >

[jira] [Assigned] (SPARK-13366) Support Cartesian join for Datasets

2016-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13366: Assignee: (was: Apache Spark) > Support Cartesian join for Datasets >

[jira] [Created] (SPARK-13366) Support Cartesian join for Datasets

2016-02-17 Thread Xiu (Joe) Guo (JIRA)
Xiu (Joe) Guo created SPARK-13366: - Summary: Support Cartesian join for Datasets Key: SPARK-13366 URL: https://issues.apache.org/jira/browse/SPARK-13366 Project: Spark Issue Type:

[jira] [Commented] (SPARK-12224) R support for JDBC source

2016-02-17 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151298#comment-15151298 ] Felix Cheung commented on SPARK-12224: -- [~shivaram] could you please review the PR comment

[jira] [Commented] (SPARK-13242) Moderately complex `when` expression causes code generation failure

2016-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151291#comment-15151291 ] Apache Spark commented on SPARK-13242: -- User 'joehalliwell' has created a pull request for this

[jira] [Updated] (SPARK-13344) Tests have many "accumulator not found" exceptions

2016-02-17 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13344: -- Summary: Tests have many "accumulator not found" exceptions (was: SaveLoadSuite has many accumulator

[jira] [Updated] (SPARK-13344) Tests have many "accumulator not found" exceptions

2016-02-17 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13344: -- Description: This is because SparkFunSuite clears all accumulators after every single test. This

[jira] [Updated] (SPARK-13279) Scheduler does O(N^2) operation when adding a new task set (making it prohibitively slow for scheduling 200K tasks)

2016-02-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13279: -- Fix Version/s: (was: 1.7) 2.0.0 > Scheduler does O(N^2) operation when adding a

[jira] [Commented] (SPARK-12449) Pushing down arbitrary logical plans to data sources

2016-02-17 Thread Evan Chan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151268#comment-15151268 ] Evan Chan commented on SPARK-12449: --- I agree with [~maxseiden] on a gradual approach to push more down

[jira] [Commented] (SPARK-13333) DataFrame filter + randn + unionAll has bad interaction

2016-02-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151265#comment-15151265 ] Xiao Li commented on SPARK-1: - Another example is MS SQL Server Rand()

[jira] [Created] (SPARK-13365) should coalesce do anything if coalescing to same number of partitions without shuffle

2016-02-17 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-13365: - Summary: should coalesce do anything if coalescing to same number of partitions without shuffle Key: SPARK-13365 URL: https://issues.apache.org/jira/browse/SPARK-13365

[jira] [Resolved] (SPARK-13279) Scheduler does O(N^2) operation when adding a new task set (making it prohibitively slow for scheduling 200K tasks)

2016-02-17 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout resolved SPARK-13279. Resolution: Fixed Fix Version/s: 1.6.1 1.7 > Scheduler does

[jira] [Comment Edited] (SPARK-13275) With dynamic allocation, executors appear to be added before job starts

2016-02-17 Thread Stephanie Bodoff (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151052#comment-15151052 ] Stephanie Bodoff edited comment on SPARK-13275 at 2/17/16 9:24 PM: ---

[jira] [Commented] (SPARK-9926) Parallelize file listing for partitioned Hive table

2016-02-17 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151211#comment-15151211 ] Ryan Blue commented on SPARK-9926: -- I've just posted [PR

[jira] [Commented] (SPARK-9926) Parallelize file listing for partitioned Hive table

2016-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151208#comment-15151208 ] Apache Spark commented on SPARK-9926: - User 'rdblue' has created a pull request for this issue:

[jira] [Created] (SPARK-13364) history server application column not sorting properly

2016-02-17 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-13364: - Summary: history server application column not sorting properly Key: SPARK-13364 URL: https://issues.apache.org/jira/browse/SPARK-13364 Project: Spark

[jira] [Commented] (SPARK-12449) Pushing down arbitrary logical plans to data sources

2016-02-17 Thread Max Seiden (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151116#comment-15151116 ] Max Seiden commented on SPARK-12449: [~rxin] Given that predicate pushdown via `sources.Filter` is

[jira] [Created] (SPARK-13363) Aggregator not working with DataFrame

2016-02-17 Thread koert kuipers (JIRA)
koert kuipers created SPARK-13363: - Summary: Aggregator not working with DataFrame Key: SPARK-13363 URL: https://issues.apache.org/jira/browse/SPARK-13363 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-13350) Configuration documentation incorrectly states that PYSPARK_PYTHON's default is "python"

2016-02-17 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-13350. Resolution: Fixed Fix Version/s: 1.6.1 2.0.0 Issue resolved by pull

[jira] [Updated] (SPARK-13350) Configuration documentation incorrectly states that PYSPARK_PYTHON's default is "python"

2016-02-17 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-13350: --- Assignee: Christopher Aycock > Configuration documentation incorrectly states that PYSPARK_PYTHON's

[jira] [Commented] (SPARK-9273) Add Convolutional Neural network to Spark MLlib

2016-02-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151015#comment-15151015 ] Sean Owen commented on SPARK-9273: -- No, I mean that I expect it will start life as an external package.

[jira] [Commented] (SPARK-9273) Add Convolutional Neural network to Spark MLlib

2016-02-17 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15150960#comment-15150960 ] Alexander Ulanov commented on SPARK-9273: - [~srowen] Do you mean that CNN will never be merged

[jira] [Commented] (SPARK-9844) File appender race condition during SparkWorker shutdown

2016-02-17 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15150913#comment-15150913 ] Bryan Cutler commented on SPARK-9844: - This error is benign for the most part, once it gets here, the

[jira] [Commented] (SPARK-13349) adding a split and union to a streaming application cause big performance hit

2016-02-17 Thread krishna ramachandran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15150893#comment-15150893 ] krishna ramachandran commented on SPARK-13349: -- i have simple synthetic example below.

[jira] [Reopened] (SPARK-12675) Executor dies because of ClassCastException and causes timeout

2016-02-17 Thread Alexandru Rosianu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexandru Rosianu reopened SPARK-12675: --- Reopening because other users are still reporting this. > Executor dies because of

[jira] [Commented] (SPARK-12675) Executor dies because of ClassCastException and causes timeout

2016-02-17 Thread Sven Krasser (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15150872#comment-15150872 ] Sven Krasser commented on SPARK-12675: -- More findings (Spark 1.6.0): For our initial 200 partition

[jira] [Commented] (SPARK-13328) Possible poor read performance for broadcast variables with dynamic resource allocation

2016-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15150863#comment-15150863 ] Apache Spark commented on SPARK-13328: -- User 'nezihyigitbasi' has created a pull request for this

[jira] [Assigned] (SPARK-13328) Possible poor read performance for broadcast variables with dynamic resource allocation

2016-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13328: Assignee: Apache Spark > Possible poor read performance for broadcast variables with

[jira] [Assigned] (SPARK-13328) Possible poor read performance for broadcast variables with dynamic resource allocation

2016-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13328: Assignee: (was: Apache Spark) > Possible poor read performance for broadcast

[jira] [Commented] (SPARK-9844) File appender race condition during SparkWorker shutdown

2016-02-17 Thread Marcelo Balloni Gomes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15150862#comment-15150862 ] Marcelo Balloni Gomes commented on SPARK-9844: -- Is there any way of avoiding this error in

[jira] [Commented] (SPARK-10340) Use S3 bulk listing for S3-backed Hive tables

2016-02-17 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15150799#comment-15150799 ] Ryan Blue commented on SPARK-10340: --- >From discussion on the pull request, it looks like the solution

[jira] [Updated] (SPARK-13322) AFTSurvivalRegression should support feature standardization

2016-02-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-13322: -- Target Version/s: 2.0.0 > AFTSurvivalRegression should support feature standardization >

[jira] [Updated] (SPARK-13322) AFTSurvivalRegression should support feature standardization

2016-02-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-13322: -- Assignee: Yanbo Liang > AFTSurvivalRegression should support feature standardization >

[jira] [Commented] (SPARK-13362) Build Error: java.lang.OutOfMemoryError: PermGen space

2016-02-17 Thread Mohit Garg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15150683#comment-15150683 ] Mohit Garg commented on SPARK-13362: thanks. > Build Error: java.lang.OutOfMemoryError: PermGen

[jira] [Updated] (SPARK-13362) Build Error: java.lang.OutOfMemoryError: PermGen space

2016-02-17 Thread Mohit Garg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohit Garg updated SPARK-13362: --- Attachment: Error.png VisualVM snapshot > Build Error: java.lang.OutOfMemoryError: PermGen space >

[jira] [Resolved] (SPARK-13362) Build Error: java.lang.OutOfMemoryError: PermGen space

2016-02-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13362. --- Resolution: Not A Problem Fix Version/s: (was: 1.5.2) Please read the build docs. You

[jira] [Updated] (SPARK-13362) Build Error: java.lang.OutOfMemoryError: PermGen space

2016-02-17 Thread Mohit Garg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohit Garg updated SPARK-13362: --- Issue Type: Bug (was: Improvement) > Build Error: java.lang.OutOfMemoryError: PermGen space >

[jira] [Created] (SPARK-13362) Build Error: java.lang.OutOfMemoryError: PermGen space

2016-02-17 Thread Mohit Garg (JIRA)
Mohit Garg created SPARK-13362: -- Summary: Build Error: java.lang.OutOfMemoryError: PermGen space Key: SPARK-13362 URL: https://issues.apache.org/jira/browse/SPARK-13362 Project: Spark Issue

[jira] [Commented] (SPARK-10759) Missing Python code example in ML Programming guide

2016-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15150654#comment-15150654 ] Apache Spark commented on SPARK-10759: -- User 'JeremyNixon' has created a pull request for this

[jira] [Resolved] (SPARK-9273) Add Convolutional Neural network to Spark MLlib

2016-02-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-9273. -- Resolution: Duplicate [~asimjalis] it's not going to happen (directly) in Spark anyway, but this is

[jira] [Closed] (SPARK-9273) Add Convolutional Neural network to Spark MLlib

2016-02-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen closed SPARK-9273. > Add Convolutional Neural network to Spark MLlib > --- > >

  1   2   >