[jira] [Commented] (SPARK-13281) Switch broadcast of RDD to exception from warning

2016-02-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15142453#comment-15142453 ] Sean Owen commented on SPARK-13281: --- I think that's fine. Better to fail fast. Any other places where

[jira] [Updated] (SPARK-13281) Switch broadcast of RDD to exception from warning

2016-02-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13281: -- Component/s: Spark Core > Switch broadcast of RDD to exception from warning >

[jira] [Updated] (SPARK-13249) Filter null keys for inner join

2016-02-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13249: -- Component/s: SQL > Filter null keys for inner join > --- > >

[jira] [Updated] (SPARK-13250) Make vectorized parquet reader work as the build side of a broadcast join

2016-02-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13250: -- Component/s: SQL > Make vectorized parquet reader work as the build side of a broadcast join >

[jira] [Assigned] (SPARK-12915) SQL metrics for generated operators

2016-02-11 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-12915: -- Assignee: Davies Liu > SQL metrics for generated operators >

[jira] [Resolved] (SPARK-11102) Uninformative exception when specifing non-exist input for JSON data source

2016-02-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11102. --- Resolution: Duplicate > Uninformative exception when specifing non-exist input for JSON data source

[jira] [Commented] (SPARK-13260) count(*) does not work with CSV data source

2016-02-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15142427#comment-15142427 ] Hyukjin Kwon commented on SPARK-13260: -- [~falaki] Since this is a quicky fix, I will submit a PR

[jira] [Updated] (SPARK-13279) Spark driver stuck holding a global lock when there are 200k tasks submitted in a stage

2016-02-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13279: -- Target Version/s: (was: 1.6.1) Fix Version/s: (was: 1.6.0) Please read

[jira] [Closed] (SPARK-9498) Some statistical information missed when the driver is out of the cluster

2016-02-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen closed SPARK-9498. > Some statistical information missed when the driver is out of the cluster >

[jira] [Resolved] (SPARK-9498) Some statistical information missed when the driver is out of the cluster

2016-02-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-9498. -- Resolution: Not A Problem Problem description still doesn't make sense > Some statistical information

[jira] [Assigned] (SPARK-12915) SQL metrics for generated operators

2016-02-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12915: Assignee: Apache Spark (was: Davies Liu) > SQL metrics for generated operators >

[jira] [Assigned] (SPARK-12915) SQL metrics for generated operators

2016-02-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12915: Assignee: Davies Liu (was: Apache Spark) > SQL metrics for generated operators >

[jira] [Commented] (SPARK-12915) SQL metrics for generated operators

2016-02-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15142450#comment-15142450 ] Apache Spark commented on SPARK-12915: -- User 'davies' has created a pull request for this issue:

[jira] [Resolved] (SPARK-13074) Add getPersistentRDDs() API to JavaSparkContext

2016-02-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13074. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10978

[jira] [Updated] (SPARK-13074) Add getPersistentRDDs() API to JavaSparkContext

2016-02-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13074: -- Assignee: Junyang Shen > Add getPersistentRDDs() API to JavaSparkContext >

[jira] [Commented] (SPARK-13260) count(*) does not work with CSV data source

2016-02-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15142421#comment-15142421 ] Hyukjin Kwon commented on SPARK-13260: -- [~falaki] Could I work on this if you are not? > count(*)

[jira] [Commented] (SPARK-13260) count(*) does not work with CSV data source

2016-02-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15142420#comment-15142420 ] Hyukjin Kwon commented on SPARK-13260: -- It is 0 and it just does not work. This was because of the

[jira] [Assigned] (SPARK-13260) count(*) does not work with CSV data source

2016-02-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13260: Assignee: (was: Apache Spark) > count(*) does not work with CSV data source >

[jira] [Commented] (SPARK-13260) count(*) does not work with CSV data source

2016-02-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15142436#comment-15142436 ] Apache Spark commented on SPARK-13260: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-13260) count(*) does not work with CSV data source

2016-02-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13260: Assignee: Apache Spark > count(*) does not work with CSV data source >

[jira] [Resolved] (SPARK-13264) Remove multi-byte character in spark-env.sh.template

2016-02-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13264. --- Resolution: Fixed Assignee: Sasaki Toru Fix Version/s: 2.0.0 Resolved by

[jira] [Updated] (SPARK-13163) Column width on new History Server DataTables not getting set correctly

2016-02-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13163: -- Assignee: Alex Bozarth > Column width on new History Server DataTables not getting set correctly >

[jira] [Updated] (SPARK-12950) Improve performance of BytesToBytesMap

2016-02-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12950: -- Assignee: Davies Liu > Improve performance of BytesToBytesMap > --

[jira] [Updated] (SPARK-13234) Remove duplicated SQL metrics

2016-02-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13234: -- Assignee: Davies Liu > Remove duplicated SQL metrics > - > >

[jira] [Updated] (SPARK-13013) Replace example code in mllib-clustering.md using include_example

2016-02-11 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Ren updated SPARK-13013: Description: The example code in the user guide is embedded in the markdown and hence it is not easy to

[jira] [Updated] (SPARK-13018) Replace example code in mllib-pmml-model-export.md using include_example

2016-02-11 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Ren updated SPARK-13018: Description: The example code in the user guide is embedded in the markdown and hence it is not easy to

[jira] [Resolved] (SPARK-6166) Add config to limit number of concurrent outbound connections for shuffle fetch

2016-02-11 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-6166. - Resolution: Fixed Fix Version/s: 2.0.0 > Add config to limit number of concurrent outbound

[jira] [Commented] (SPARK-6166) Add config to limit number of concurrent outbound connections for shuffle fetch

2016-02-11 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144146#comment-15144146 ] Reynold Xin commented on SPARK-6166: [~zsxwing] can you update the title of this ticket to something

[jira] [Updated] (SPARK-6166) Limit number of in flight outbound requests for shuffle fetch

2016-02-11 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-6166: Summary: Limit number of in flight outbound requests for shuffle fetch (was: Add config to limit

[jira] [Commented] (SPARK-6166) Limit number of in flight outbound requests for shuffle fetch

2016-02-11 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144147#comment-15144147 ] Reynold Xin commented on SPARK-6166: Thanks! > Limit number of in flight outbound requests for

[jira] [Commented] (SPARK-6166) Limit number of in flight outbound requests for shuffle fetch

2016-02-11 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144148#comment-15144148 ] Shixiong Zhu commented on SPARK-6166: - Sure. Done > Limit number of in flight outbound requests for

[jira] [Updated] (SPARK-6166) Limit number of in flight outbound requests for shuffle fetch

2016-02-11 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-6166: Description: spark.reducer.maxMbInFlight puts a bound on the in flight data in terms of size. But

[jira] [Created] (SPARK-13295) ML/MLLIB: AFTSurvivalRegression: Improve AFTAggregator - Avoid creating new instances of arrays/vectors for each record

2016-02-11 Thread Narine Kokhlikyan (JIRA)
Narine Kokhlikyan created SPARK-13295: - Summary: ML/MLLIB: AFTSurvivalRegression: Improve AFTAggregator - Avoid creating new instances of arrays/vectors for each record Key: SPARK-13295 URL:

[jira] [Updated] (SPARK-13015) Replace example code in mllib-data-types.md using include_example

2016-02-11 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Ren updated SPARK-13015: Description: The example code in the user guide is embedded in the markdown and hence it is not easy to

[jira] [Updated] (SPARK-13017) Replace example code in mllib-feature-extraction.md using include_example

2016-02-11 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Ren updated SPARK-13017: Description: The example code in the user guide is embedded in the markdown and hence it is not easy to

[jira] [Updated] (SPARK-13013) Replace example code in mllib-clustering.md using include_example

2016-02-11 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Ren updated SPARK-13013: Description: The example code in the user guide is embedded in the markdown and hence it is not easy to

[jira] [Updated] (SPARK-13019) Replace example code in mllib-statistics.md using include_example

2016-02-11 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Ren updated SPARK-13019: Description: The example code in the user guide is embedded in the markdown and hence it is not easy to

[jira] [Updated] (SPARK-13012) Replace example code in ml-guide.md using include_example

2016-02-11 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-13012: -- Shepherd: Xusen Yin > Replace example code in ml-guide.md using include_example >

[jira] [Updated] (SPARK-13012) Replace example code in ml-guide.md using include_example

2016-02-11 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-13012: -- Assignee: Devaraj K > Replace example code in ml-guide.md using include_example >

[jira] [Assigned] (SPARK-13295) ML/MLLIB: AFTSurvivalRegression: Improve AFTAggregator - Avoid creating new instances of arrays/vectors for each record

2016-02-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13295: Assignee: (was: Apache Spark) > ML/MLLIB: AFTSurvivalRegression: Improve

[jira] [Commented] (SPARK-13295) ML/MLLIB: AFTSurvivalRegression: Improve AFTAggregator - Avoid creating new instances of arrays/vectors for each record

2016-02-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144067#comment-15144067 ] Apache Spark commented on SPARK-13295: -- User 'NarineK' has created a pull request for this issue:

[jira] [Updated] (SPARK-13295) ML/MLLIB: AFTSurvivalRegression: Improve AFTAggregator - Avoid creating new instances of arrays/vectors for each record

2016-02-11 Thread Narine Kokhlikyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Narine Kokhlikyan updated SPARK-13295: -- Description: As also mentioned/marked by TODO in AFTAggregator.AFTAggregator.add(data:

[jira] [Comment Edited] (SPARK-13262) cannot coerce type 'environment' to vector of type 'list'

2016-02-11 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15142664#comment-15142664 ] Sun Rui edited comment on SPARK-13262 at 2/11/16 12:36 PM: --- [~samalexg] Thanks

[jira] [Commented] (SPARK-8000) SQLContext.read.load() should be able to auto-detect input data

2016-02-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15142412#comment-15142412 ] Hyukjin Kwon commented on SPARK-8000: - [~yanboliang] Are you working on this or is this already fixed?

[jira] [Created] (SPARK-13283) Spark doesn't escape column names when creating table on JDBC

2016-02-11 Thread JIRA
Maciej Bryński created SPARK-13283: -- Summary: Spark doesn't escape column names when creating table on JDBC Key: SPARK-13283 URL: https://issues.apache.org/jira/browse/SPARK-13283 Project: Spark

[jira] [Updated] (SPARK-13124) Adding JQuery DataTables messed up the Web UI css and js

2016-02-11 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-13124: -- Assignee: Alex Bozarth > Adding JQuery DataTables messed up the Web UI css and js >

[jira] [Resolved] (SPARK-13124) Adding JQuery DataTables messed up the Web UI css and js

2016-02-11 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-13124. --- Resolution: Fixed Fix Version/s: 2.0.0 > Adding JQuery DataTables messed up the Web

[jira] [Updated] (SPARK-13283) Spark doesn't escape column names when creating table on JDBC

2016-02-11 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-13283: --- Description: Hi, I have following problem. I have DF where one of the columns has 'from'

[jira] [Assigned] (SPARK-13295) ML/MLLIB: AFTSurvivalRegression: Improve AFTAggregator - Avoid creating new instances of arrays/vectors for each record

2016-02-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13295: Assignee: Apache Spark > ML/MLLIB: AFTSurvivalRegression: Improve AFTAggregator - Avoid

[jira] [Updated] (SPARK-13295) ML/MLLIB: AFTSurvivalRegression: Improve AFTAggregator - Avoid creating new instances of arrays/vectors for each record

2016-02-11 Thread Narine Kokhlikyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Narine Kokhlikyan updated SPARK-13295: -- Description: As also mentioned/marked by TODO in AFTAggregator.AFTAggregator.add(data:

[jira] [Updated] (SPARK-13014) Replace example code in mllib-collaborative-filtering.md using include_example

2016-02-11 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Ren updated SPARK-13014: Description: The example code in the user guide is embedded in the markdown and hence it is not easy to

[jira] [Commented] (SPARK-13285) Master isn't assigning the worker to process the application

2016-02-11 Thread Lokesh Ranga (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15143258#comment-15143258 ] Lokesh Ranga commented on SPARK-13285: -- Nevermind, Figured it out. > Master isn't assigning the

[jira] [Updated] (SPARK-13279) Spark driver is very slow (due to N^2 check) when there are 200k tasks submitted in a stage

2016-02-11 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-13279: --- Priority: Major (was: Minor) > Spark driver is very slow (due to N^2 check) when there are

[jira] [Commented] (SPARK-8592) CoarseGrainedExecutorBackend: Cannot register with driver => NPE

2016-02-11 Thread Nezih Yigitbasi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15143328#comment-15143328 ] Nezih Yigitbasi commented on SPARK-8592: We still see this problem with 1.5.2 {code} 16/02/11

[jira] [Updated] (SPARK-13279) Spark driver is very slow (due to N^2 check) when there are 200k tasks submitted in a stage

2016-02-11 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-13279: --- Component/s: Scheduler > Spark driver is very slow (due to N^2 check) when there are 200k

[jira] [Commented] (SPARK-8592) CoarseGrainedExecutorBackend: Cannot register with driver => NPE

2016-02-11 Thread Nezih Yigitbasi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15143330#comment-15143330 ] Nezih Yigitbasi commented on SPARK-8592: Any ideas [~joshrosen]? > CoarseGrainedExecutorBackend:

[jira] [Updated] (SPARK-13279) Spark driver stuck holding a global lock when there are 200k tasks submitted in a stage

2016-02-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13279: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) > Spark driver stuck holding a

[jira] [Updated] (SPARK-13279) Scheduler does O(N^2) operation when adding a new task set (making it prohibitively slow for scheduling 200K tasks)

2016-02-11 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-13279: --- Summary: Scheduler does O(N^2) operation when adding a new task set (making it prohibitively

[jira] [Created] (SPARK-13286) JDBC driver doesn't report full exception

2016-02-11 Thread Adrian Bridgett (JIRA)
Adrian Bridgett created SPARK-13286: --- Summary: JDBC driver doesn't report full exception Key: SPARK-13286 URL: https://issues.apache.org/jira/browse/SPARK-13286 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-13277) ANTLR ignores other rule using the USING keyword

2016-02-11 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-13277. --- Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 2.0.0 >

[jira] [Resolved] (SPARK-12982) SQLContext: temporary table registration does not accept valid identifier

2016-02-11 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-12982. --- Resolution: Resolved Assignee: Jayadevan M Fix Version/s: 2.0.0 >

[jira] [Commented] (SPARK-13279) Spark driver stuck holding a global lock when there are 200k tasks submitted in a stage

2016-02-11 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15143259#comment-15143259 ] Sital Kedia commented on SPARK-13279: - As you can see from the jstack of the driver

[jira] [Updated] (SPARK-13279) Spark driver is very slow (due to N^2 check) when there are 200k tasks submitted in a stage

2016-02-11 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-13279: --- Summary: Spark driver is very slow (due to N^2 check) when there are 200k tasks submitted in

[jira] [Updated] (SPARK-13279) Spark driver is very slow (due to N^2 check) when there are 200k tasks submitted in a stage

2016-02-11 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-13279: --- Description: For each task that the TaskSetManager adds, it iterates through the entire list

[jira] [Updated] (SPARK-13286) JDBC driver doesn't report full exception

2016-02-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13286: -- Issue Type: Improvement (was: Bug) > JDBC driver doesn't report full exception >

[jira] [Assigned] (SPARK-13282) LogicalPlan toSql should just return a String rather than Option[String]

2016-02-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13282: Assignee: Apache Spark (was: Reynold Xin) > LogicalPlan toSql should just return a

[jira] [Assigned] (SPARK-13282) LogicalPlan toSql should just return a String rather than Option[String]

2016-02-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13282: Assignee: Reynold Xin (was: Apache Spark) > LogicalPlan toSql should just return a

[jira] [Commented] (SPARK-13282) LogicalPlan toSql should just return a String rather than Option[String]

2016-02-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15142505#comment-15142505 ] Apache Spark commented on SPARK-13282: -- User 'rxin' has created a pull request for this issue:

[jira] [Created] (SPARK-13282) LogicalPlan toSql should just return a String rather than Option[String]

2016-02-11 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-13282: --- Summary: LogicalPlan toSql should just return a String rather than Option[String] Key: SPARK-13282 URL: https://issues.apache.org/jira/browse/SPARK-13282 Project:

[jira] [Updated] (SPARK-13148) support zero-keytab Oozie application launch via HADOOP_TOKEN_FILE_LOCATION env var

2016-02-11 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-13148: --- Summary: support zero-keytab Oozie application launch via HADOOP_TOKEN_FILE_LOCATION env var

[jira] [Commented] (SPARK-13262) cannot coerce type 'environment' to vector of type 'list'

2016-02-11 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15142664#comment-15142664 ] Sun Rui commented on SPARK-13262: - [~samalexg] Thanks for reporting this. This is unlikely to happen.

[jira] [Created] (SPARK-13284) Cannot submit app from Windows java.io.FileNotFoundException: /C:

2016-02-11 Thread DK (JIRA)
DK created SPARK-13284: -- Summary: Cannot submit app from Windows java.io.FileNotFoundException: /C: Key: SPARK-13284 URL: https://issues.apache.org/jira/browse/SPARK-13284 Project: Spark Issue Type:

[jira] [Updated] (SPARK-13284) Cannot submit app from Windows java.io.FileNotFoundException: /C:

2016-02-11 Thread DK (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DK updated SPARK-13284: --- Description: I've tried multiple different ways to submit my spark java application from my Windows machine to a

[jira] [Updated] (SPARK-13284) Cannot submit app from Windows java.io.FileNotFoundException: /C:

2016-02-11 Thread DK (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DK updated SPARK-13284: --- Description: I've tried multiple different ways to submit my spark java application from my Windows machine to a

[jira] [Updated] (SPARK-13284) Cannot submit app from Windows java.io.FileNotFoundException: /C:

2016-02-11 Thread DK (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DK updated SPARK-13284: --- Description: I've tried multiple different ways to submit my spark java application from my Windows machine to a

[jira] [Created] (SPARK-13289) Word2Vec generate infinite distances when numIterations>5

2016-02-11 Thread Qi Dai (JIRA)
Qi Dai created SPARK-13289: -- Summary: Word2Vec generate infinite distances when numIterations>5 Key: SPARK-13289 URL: https://issues.apache.org/jira/browse/SPARK-13289 Project: Spark Issue Type:

[jira] [Created] (SPARK-13290) wholeTextFile and binaryFiles are really slow

2016-02-11 Thread mathieu longtin (JIRA)
mathieu longtin created SPARK-13290: --- Summary: wholeTextFile and binaryFiles are really slow Key: SPARK-13290 URL: https://issues.apache.org/jira/browse/SPARK-13290 Project: Spark Issue

[jira] [Commented] (SPARK-13279) Scheduler does O(N^2) operation when adding a new task set (making it prohibitively slow for scheduling 200K tasks)

2016-02-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15143629#comment-15143629 ] Apache Spark commented on SPARK-13279: -- User 'sitalkedia' has created a pull request for this issue:

[jira] [Resolved] (SPARK-13035) PySpark ml.clustering support export/import

2016-02-11 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-13035. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10999

[jira] [Updated] (SPARK-13035) PySpark ml.clustering support export/import

2016-02-11 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-13035: -- Assignee: Yanbo Liang > PySpark ml.clustering support export/import >

[jira] [Updated] (SPARK-7483) [MLLib] Using Kryo with FPGrowth fails with an exception

2016-02-11 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-7483: - Shepherd: Sean Owen > [MLLib] Using Kryo with FPGrowth fails with an exception >

[jira] [Commented] (SPARK-13069) ActorHelper is not throttled by rate limiter

2016-02-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15143760#comment-15143760 ] Apache Spark commented on SPARK-13069: -- User 'lin-zhao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13069) ActorHelper is not throttled by rate limiter

2016-02-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13069: Assignee: Apache Spark > ActorHelper is not throttled by rate limiter >

[jira] [Assigned] (SPARK-13069) ActorHelper is not throttled by rate limiter

2016-02-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13069: Assignee: (was: Apache Spark) > ActorHelper is not throttled by rate limiter >

[jira] [Commented] (SPARK-12949) Support common expression elimination

2016-02-11 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15143787#comment-15143787 ] Davies Liu commented on SPARK-12949: After some prototype, enable common expression elimination could

[jira] [Commented] (SPARK-13293) Generate code for Expand

2016-02-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15143802#comment-15143802 ] Apache Spark commented on SPARK-13293: -- User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13293) Generate code for Expand

2016-02-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13293: Assignee: Apache Spark (was: Davies Liu) > Generate code for Expand >

[jira] [Assigned] (SPARK-13293) Generate code for Expand

2016-02-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13293: Assignee: Davies Liu (was: Apache Spark) > Generate code for Expand >

[jira] [Updated] (SPARK-13011) K-means wrapper in SparkR

2016-02-11 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-13011: -- Shepherd: Xiangrui Meng > K-means wrapper in SparkR > - > >

[jira] [Assigned] (SPARK-13294) Don't build assembly in dev/run-tests

2016-02-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13294: Assignee: Apache Spark > Don't build assembly in dev/run-tests >

[jira] [Created] (SPARK-13294) Don't build assembly in dev/run-tests

2016-02-11 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-13294: -- Summary: Don't build assembly in dev/run-tests Key: SPARK-13294 URL: https://issues.apache.org/jira/browse/SPARK-13294 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-13294) Don't build assembly in dev/run-tests

2016-02-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13294: Assignee: (was: Apache Spark) > Don't build assembly in dev/run-tests >

[jira] [Commented] (SPARK-13294) Don't build assembly in dev/run-tests

2016-02-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15143846#comment-15143846 ] Apache Spark commented on SPARK-13294: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Resolved] (SPARK-12765) CountVectorizerModel.transform lost the transformSchema

2016-02-11 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-12765. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10720

[jira] [Commented] (SPARK-13069) ActorHelper is not throttled by rate limiter

2016-02-11 Thread Lin Zhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15143768#comment-15143768 ] Lin Zhao commented on SPARK-13069: -- [~zsxwing] I created the PR, please review at your convenience. We

[jira] [Created] (SPARK-13293) Generate code for Expand

2016-02-11 Thread Davies Liu (JIRA)
Davies Liu created SPARK-13293: -- Summary: Generate code for Expand Key: SPARK-13293 URL: https://issues.apache.org/jira/browse/SPARK-13293 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-13047) Pyspark Params.hasParam should not throw an error

2016-02-11 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-13047. --- Resolution: Fixed Fix Version/s: 1.6.1 2.0.0 Issue resolved by

[jira] [Updated] (SPARK-13033) PySpark ml.regression support export/import

2016-02-11 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-13033: -- Assignee: Tommy Yu > PySpark ml.regression support export/import >

[jira] [Updated] (SPARK-13033) PySpark ml.regression support export/import

2016-02-11 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-13033: -- Shepherd: Yanbo Liang Target Version/s: 2.0.0 > PySpark ml.regression support

[jira] [Updated] (SPARK-13153) PySpark ML persistence failed when handle no default value parameter

2016-02-11 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-13153: -- Shepherd: Yanbo Liang Target Version/s: 1.6.1, 2.0.0 > PySpark ML persistence

  1   2   >