[jira] [Commented] (SPARK-5095) Support launching multiple mesos executors in coarse grained mesos mode

[ https://issues.apache.org/jira/browse/SPARK-5095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276607#comment-14276607 ] Josh Devins commented on SPARK-5095: Nice one, gonna try and test it this week. >

[jira] [Commented] (SPARK-5236) java.lang.ClassCastException: org.apache.spark.sql.catalyst.expressions.MutableAny cannot be cast to org.apache.spark.sql.catalyst.expressions.MutableInt

[ https://issues.apache.org/jira/browse/SPARK-5236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276594#comment-14276594 ] Apache Spark commented on SPARK-5236: - User 'alexbaretta' has created a pull request f

[jira] [Commented] (SPARK-5242) "ec2/spark_ec2.py lauch" does not work with VPC if no public DNS or IP is available

[ https://issues.apache.org/jira/browse/SPARK-5242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276591#comment-14276591 ] Vladimir Grigor commented on SPARK-5242: This bug is fixed in https://github.com/a

[jira] [Commented] (SPARK-5242) "ec2/spark_ec2.py lauch" does not work with VPC if no public DNS or IP is available

[ https://issues.apache.org/jira/browse/SPARK-5242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276590#comment-14276590 ] Apache Spark commented on SPARK-5242: - User 'voukka' has created a pull request for th

[jira] [Created] (SPARK-5243) Spark will hang if (driver memory + executor memory) exceeds limit on a 1-worker cluster

yuhao yang created SPARK-5243: - Summary: Spark will hang if (driver memory + executor memory) exceeds limit on a 1-worker cluster Key: SPARK-5243 URL: https://issues.apache.org/jira/browse/SPARK-5243 Proj

[jira] [Commented] (SPARK-5147) write ahead logs from streaming receiver are not purged because cleanupOldBlocks in WriteAheadLogBasedBlockHandler is never called

[ https://issues.apache.org/jira/browse/SPARK-5147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276581#comment-14276581 ] Apache Spark commented on SPARK-5147: - User 'jerryshao' has created a pull request for

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276572#comment-14276572 ] Florian Verhein commented on SPARK-3821: Thanks [~nchammas], that makes sense. Cr

[jira] [Created] (SPARK-5242) "ec2/spark_ec2.py lauch" does not work with VPC if no public DNS or IP is available

Vladimir Grigor created SPARK-5242: -- Summary: "ec2/spark_ec2.py lauch" does not work with VPC if no public DNS or IP is available Key: SPARK-5242 URL: https://issues.apache.org/jira/browse/SPARK-5242

[jira] [Created] (SPARK-5241) spark-ec2 spark init scripts do not handle all hadoop (or tachyon?) dependencies correctly

Florian Verhein created SPARK-5241: -- Summary: spark-ec2 spark init scripts do not handle all hadoop (or tachyon?) dependencies correctly Key: SPARK-5241 URL: https://issues.apache.org/jira/browse/SPARK-5241

[jira] [Commented] (SPARK-5240) Adding `createDataSourceTable` interface to Catalog

[ https://issues.apache.org/jira/browse/SPARK-5240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276566#comment-14276566 ] Apache Spark commented on SPARK-5240: - User 'scwf' has created a pull request for this

[jira] [Commented] (SPARK-4923) Maven build should keep publishing spark-repl

[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276562#comment-14276562 ] Chip Senkbeil commented on SPARK-4923: -- As the nice bot has stated, I created a pull

[jira] [Created] (SPARK-5240) Adding `createDataSourceTable` interface to Catalog

wangfei created SPARK-5240: -- Summary: Adding `createDataSourceTable` interface to Catalog Key: SPARK-5240 URL: https://issues.apache.org/jira/browse/SPARK-5240 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-5236) java.lang.ClassCastException: org.apache.spark.sql.catalyst.expressions.MutableAny cannot be cast to org.apache.spark.sql.catalyst.expressions.MutableInt

[ https://issues.apache.org/jira/browse/SPARK-5236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Baretta updated SPARK-5236: Summary: java.lang.ClassCastException: org.apache.spark.sql.catalyst.expressions.MutableAny cannot b

[jira] [Commented] (SPARK-4923) Maven build should keep publishing spark-repl

[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276558#comment-14276558 ] Apache Spark commented on SPARK-4923: - User 'rcsenkbeil' has created a pull request fo

[jira] [Commented] (SPARK-5239) JdbcRDD throws "java.lang.AbstractMethodError: oracle.jdbc.driver.xxxxxx.isClosed()Z"

[ https://issues.apache.org/jira/browse/SPARK-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276540#comment-14276540 ] Apache Spark commented on SPARK-5239: - User 'luogankun' has created a pull request for

[jira] [Created] (SPARK-5239) JdbcRDD throws "java.lang.AbstractMethodError: oracle.jdbc.driver.xxxxxx.isClosed()Z"

Gankun Luo created SPARK-5239: - Summary: JdbcRDD throws "java.lang.AbstractMethodError: oracle.jdbc.driver.xx.isClosed()Z" Key: SPARK-5239 URL: https://issues.apache.org/jira/browse/SPARK-5239 Project

[jira] [Updated] (SPARK-5142) Possibly data may be ruined in Spark Streaming's WAL mechanism.

[ https://issues.apache.org/jira/browse/SPARK-5142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-5142: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-5238 > Possibly data may be ruined in Spark Str

[jira] [Updated] (SPARK-5147) write ahead logs from streaming receiver are not purged because cleanupOldBlocks in WriteAheadLogBasedBlockHandler is never called

[ https://issues.apache.org/jira/browse/SPARK-5147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-5147: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-5238 > write ahead logs from streaming receiver

[jira] [Updated] (SPARK-5233) Error replay of WAL when recovered from driver failue

[ https://issues.apache.org/jira/browse/SPARK-5233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-5233: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-5238 > Error replay of WAL when recovered from

[jira] [Updated] (SPARK-5238) Improve the robustness of Spark Streaming WAL mechanism

[ https://issues.apache.org/jira/browse/SPARK-5238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-5238: --- Description: Several issues identified in Spark Streaming's WAL mechanism, this is a cap of all the r

[jira] [Updated] (SPARK-5237) UDTF don't work on SparK SQL

[ https://issues.apache.org/jira/browse/SPARK-5237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-5237: --- Description: Hive query with UDTF don't work on Spark SQL 15/01/14 13:23:50 INFO ParseDriver: Parse Completed

[jira] [Created] (SPARK-5238) Improve the robustness of Spark Streaming WAL mechanism

Saisai Shao created SPARK-5238: -- Summary: Improve the robustness of Spark Streaming WAL mechanism Key: SPARK-5238 URL: https://issues.apache.org/jira/browse/SPARK-5238 Project: Spark Issue Type:

[jira] [Created] (SPARK-5237) UDTF don't work on SparK SQL

Yi Zhou created SPARK-5237: -- Summary: UDTF don't work on SparK SQL Key: SPARK-5237 URL: https://issues.apache.org/jira/browse/SPARK-5237 Project: Spark Issue Type: Bug Components: SQL

[jira] [Commented] (SPARK-5233) Error replay of WAL when recovered from driver failue

[ https://issues.apache.org/jira/browse/SPARK-5233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276513#comment-14276513 ] Apache Spark commented on SPARK-5233: - User 'jerryshao' has created a pull request for

[jira] [Created] (SPARK-5236) parquet.io.ParquetDecodingException: Can not read value at 0 in block 0

Alex Baretta created SPARK-5236: --- Summary: parquet.io.ParquetDecodingException: Can not read value at 0 in block 0 Key: SPARK-5236 URL: https://issues.apache.org/jira/browse/SPARK-5236 Project: Spark

[jira] [Updated] (SPARK-5233) Error replay of WAL when recovered from driver failue

[ https://issues.apache.org/jira/browse/SPARK-5233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-5233: --- Description: Spark Streaming will write all the event into WAL for driver recovery, the sequence in t

[jira] [Commented] (SPARK-5235) java.io.NotSerializableException: org.apache.spark.sql.SQLConf

[ https://issues.apache.org/jira/browse/SPARK-5235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276507#comment-14276507 ] Apache Spark commented on SPARK-5235: - User 'alexbaretta' has created a pull request f

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276505#comment-14276505 ] Shivaram Venkataraman commented on SPARK-3821: -- [~nchammas] Yes -- That sound

[jira] [Created] (SPARK-5235) java.io.NotSerializableException: org.apache.spark.sql.SQLConf

Alex Baretta created SPARK-5235: --- Summary: java.io.NotSerializableException: org.apache.spark.sql.SQLConf Key: SPARK-5235 URL: https://issues.apache.org/jira/browse/SPARK-5235 Project: Spark I

[jira] [Updated] (SPARK-1805) Error launching cluster when master and slave machines are of different virtualization types

[ https://issues.apache.org/jira/browse/SPARK-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-1805: Description: In the current EC2 script, the AMI image object is loaded only once. This

[jira] [Updated] (SPARK-1805) Error launching cluster when master and slave machines are of different virtualization types

[ https://issues.apache.org/jira/browse/SPARK-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-1805: Affects Version/s: 1.1.1 1.2.0 > Error launching cluster when master

[jira] [Updated] (SPARK-1805) Error launching cluster when master and slaves machines are of different visualization types

[ https://issues.apache.org/jira/browse/SPARK-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-1805: Issue Type: Bug (was: Improvement) > Error launching cluster when master and slaves machine

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276471#comment-14276471 ] Nicholas Chammas commented on SPARK-3821: - [~shivaram] Are we ready to open a PR a

[jira] [Created] (SPARK-5234) examples for ml don't have sparkContext.stop

yuhao yang created SPARK-5234: - Summary: examples for ml don't have sparkContext.stop Key: SPARK-5234 URL: https://issues.apache.org/jira/browse/SPARK-5234 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-3185) SPARK launch on Hadoop 2 in EC2 throws Tachyon exception when Formatting JOURNAL_FOLDER

[ https://issues.apache.org/jira/browse/SPARK-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276436#comment-14276436 ] Florian Verhein commented on SPARK-3185: I'm also getting this, though with "Serve

[jira] [Commented] (SPARK-3678) Yarn app name reported in RM is different between cluster and client mode

[ https://issues.apache.org/jira/browse/SPARK-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276429#comment-14276429 ] WangTaoTheTonic commented on SPARK-3678: In SparkHdfsLR there has {quote}val spark

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276411#comment-14276411 ] Nicholas Chammas commented on SPARK-3821: - Hi [~florianverhein] and thanks for chi

[jira] [Updated] (SPARK-3185) SPARK launch on Hadoop 2 in EC2 throws Tachyon exception when Formatting JOURNAL_FOLDER

[ https://issues.apache.org/jira/browse/SPARK-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3185: Description: {code} org.apache.hadoop.ipc.RemoteException: Server IPC version 7 cannot commu

[jira] [Comment Edited] (SPARK-4894) Add Bernoulli-variant of Naive Bayes

[ https://issues.apache.org/jira/browse/SPARK-4894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276380#comment-14276380 ] RJ Nowling edited comment on SPARK-4894 at 1/14/15 2:06 AM: Hi

[jira] [Commented] (SPARK-4894) Add Bernoulli-variant of Naive Bayes

[ https://issues.apache.org/jira/browse/SPARK-4894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276380#comment-14276380 ] RJ Nowling commented on SPARK-4894: --- Hi @lmcguire, Always happy to have more help! :)

[jira] [Commented] (SPARK-5220) keepPushingBlocks in BlockGenerator terminated when an exception occurs, which causes the block pushing thread to terminate and blocks receiver

[ https://issues.apache.org/jira/browse/SPARK-5220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276350#comment-14276350 ] Saisai Shao commented on SPARK-5220: Hi Max, as I said in the mail, this is an expecte

[jira] [Created] (SPARK-5233) Error replay of WAL when recovered from driver failue

Saisai Shao created SPARK-5233: -- Summary: Error replay of WAL when recovered from driver failue Key: SPARK-5233 URL: https://issues.apache.org/jira/browse/SPARK-5233 Project: Spark Issue Type: B

[jira] [Commented] (SPARK-5167) Move Row into sql package and make it usable for Java

[ https://issues.apache.org/jira/browse/SPARK-5167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276327#comment-14276327 ] Apache Spark commented on SPARK-5167: - User 'rxin' has created a pull request for this

[jira] [Resolved] (SPARK-5123) Stabilize Spark SQL data type API

[ https://issues.apache.org/jira/browse/SPARK-5123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-5123. Resolution: Fixed Fix Version/s: 1.3.0 > Stabilize Spark SQL data type API >

[jira] [Commented] (SPARK-4296) Throw "Expression not in GROUP BY" when using same expression in group by clause and select clause

[ https://issues.apache.org/jira/browse/SPARK-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276278#comment-14276278 ] Cheng Lian commented on SPARK-4296: --- Yeah, I think whenever we use expressions that are

[jira] [Closed] (SPARK-5232) CombineFileInputFormatShim#getDirIndices is expensive

[ https://issues.apache.org/jira/browse/SPARK-5232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang closed SPARK-5232. -- Resolution: Invalid Wrong project. > CombineFileInputFormatShim#getDirIndices is expensive > --

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276263#comment-14276263 ] Florian Verhein commented on SPARK-3821: This is great stuff! It'll also help serv

[jira] [Created] (SPARK-5232) CombineFileInputFormatShim#getDirIndices is expensive

Jimmy Xiang created SPARK-5232: -- Summary: CombineFileInputFormatShim#getDirIndices is expensive Key: SPARK-5232 URL: https://issues.apache.org/jira/browse/SPARK-5232 Project: Spark Issue Type: I

[jira] [Commented] (SPARK-5231) History Server shows wrong job submission time.

[ https://issues.apache.org/jira/browse/SPARK-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276246#comment-14276246 ] Apache Spark commented on SPARK-5231: - User 'sarutak' has created a pull request for t

[jira] [Created] (SPARK-5231) History Server shows wrong job submission time.

Kousuke Saruta created SPARK-5231: - Summary: History Server shows wrong job submission time. Key: SPARK-5231 URL: https://issues.apache.org/jira/browse/SPARK-5231 Project: Spark Issue Type: B

[jira] [Created] (SPARK-5230) Print usage for spark-submit and spark-class in Windows

Andrew Or created SPARK-5230: Summary: Print usage for spark-submit and spark-class in Windows Key: SPARK-5230 URL: https://issues.apache.org/jira/browse/SPARK-5230 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-5088) Use spark-class for running executors directly on mesos

[ https://issues.apache.org/jira/browse/SPARK-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-5088: - Target Version/s: 1.3.0 (was: 1.3.0, 1.2.1) > Use spark-class for running executors directly on m

[jira] [Updated] (SPARK-5088) Use spark-class for running executors directly on mesos

[ https://issues.apache.org/jira/browse/SPARK-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-5088: - Fix Version/s: (was: 1.2.1) > Use spark-class for running executors directly on mesos > --

[jira] [Commented] (SPARK-4520) SparkSQL exception when reading certain columns from a parquet file

[ https://issues.apache.org/jira/browse/SPARK-4520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276202#comment-14276202 ] Tyler commented on SPARK-4520: -- Schema: requestedSchema: message root { required group key

[jira] [Commented] (SPARK-4520) SparkSQL exception when reading certain columns from a parquet file

[ https://issues.apache.org/jira/browse/SPARK-4520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276196#comment-14276196 ] Tyler commented on SPARK-4520: -- I'm also interested in the solution to this. I'm having a sim

[jira] [Commented] (SPARK-4894) Add Bernoulli-variant of Naive Bayes

[ https://issues.apache.org/jira/browse/SPARK-4894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276161#comment-14276161 ] Leah McGuire commented on SPARK-4894: - Are you guys working on this? I would like to c

[jira] [Commented] (SPARK-5228) Hide tables for "Active Jobs/Completed Jobs/Failed Jobs" when they are empty

[ https://issues.apache.org/jira/browse/SPARK-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276109#comment-14276109 ] Apache Spark commented on SPARK-5228: - User 'sarutak' has created a pull request for t

[jira] [Commented] (SPARK-5095) Support launching multiple mesos executors in coarse grained mesos mode

[ https://issues.apache.org/jira/browse/SPARK-5095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276093#comment-14276093 ] Apache Spark commented on SPARK-5095: - User 'tnachen' has created a pull request for t

[jira] [Commented] (SPARK-5095) Support launching multiple mesos executors in coarse grained mesos mode

[ https://issues.apache.org/jira/browse/SPARK-5095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276094#comment-14276094 ] Timothy Chen commented on SPARK-5095: - [~joshdevins][~maasg] I have a PR out now, I wo

[jira] [Created] (SPARK-5229) Use tableIdentifier as the reference of a table

Yin Huai created SPARK-5229: --- Summary: Use tableIdentifier as the reference of a table Key: SPARK-5229 URL: https://issues.apache.org/jira/browse/SPARK-5229 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-5228) Hide tables for "Active Jobs/Completed Jobs/Failed Jobs" when they are empty

Kousuke Saruta created SPARK-5228: - Summary: Hide tables for "Active Jobs/Completed Jobs/Failed Jobs" when they are empty Key: SPARK-5228 URL: https://issues.apache.org/jira/browse/SPARK-5228 Project:

[jira] [Created] (SPARK-5227) InputOutputMetricsSuite "input metrics when reading text file with multiple splits" test fails in branch-1.2 SBT Jenkins build w/hadoop1.0 and hadoop2.0 profiles

Josh Rosen created SPARK-5227: - Summary: InputOutputMetricsSuite "input metrics when reading text file with multiple splits" test fails in branch-1.2 SBT Jenkins build w/hadoop1.0 and hadoop2.0 profiles Key: SPARK-5227

[jira] [Resolved] (SPARK-5168) Make SQLConf a field rather than mixin in SQLContext

[ https://issues.apache.org/jira/browse/SPARK-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5168. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 3965 [https:/

[jira] [Updated] (SPARK-5226) Add DBSCAN Clustering Algorithm to MLlib

[ https://issues.apache.org/jira/browse/SPARK-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Muhammad-Ali A'rabi updated SPARK-5226: --- Labels: DBSCAN (was: ) > Add DBSCAN Clustering Algorithm to MLlib > -

[jira] [Updated] (SPARK-5226) Add DBSCAN Clustering Algorithm to MLlib

[ https://issues.apache.org/jira/browse/SPARK-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Muhammad-Ali A'rabi updated SPARK-5226: --- Description: MLlib is all k-means now, and I think we should add some new clustering a

[jira] [Commented] (SPARK-5226) Add DBSCAN Clustering Algorithm to MLlib

[ https://issues.apache.org/jira/browse/SPARK-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14275981#comment-14275981 ] Muhammad-Ali A'rabi commented on SPARK-5226: Although I can't assign this task

[jira] [Updated] (SPARK-5179) Spark UI history job duration is wrong

[ https://issues.apache.org/jira/browse/SPARK-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olivier Toupin updated SPARK-5179: -- Target Version/s: 1.2.1 > Spark UI history job duration is wrong > -

[jira] [Created] (SPARK-5226) Add DBSCAN Clustering Algorithm to MLlib

Muhammad-Ali A'rabi created SPARK-5226: -- Summary: Add DBSCAN Clustering Algorithm to MLlib Key: SPARK-5226 URL: https://issues.apache.org/jira/browse/SPARK-5226 Project: Spark Issue Type

[jira] [Commented] (SPARK-1405) parallel Latent Dirichlet Allocation (LDA) atop of spark in MLlib

[ https://issues.apache.org/jira/browse/SPARK-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14275970#comment-14275970 ] Joseph K. Bradley commented on SPARK-1405: -- [~pedrorodriguez] Thanks for the tes

[jira] [Commented] (SPARK-5097) Adding data frame APIs to SchemaRDD

[ https://issues.apache.org/jira/browse/SPARK-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14275960#comment-14275960 ] Mohit Jaggi commented on SPARK-5097: minor comment: mutate existing can do df("x") =

[jira] [Resolved] (SPARK-4912) Persistent data source tables

[ https://issues.apache.org/jira/browse/SPARK-4912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4912. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 3960 [https:/

[jira] [Resolved] (SPARK-5223) Use pickle instead of MapConvert and ListConvert in MLlib Python API

[ https://issues.apache.org/jira/browse/SPARK-5223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-5223. -- Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 Issue resolved by pull re

[jira] [Updated] (SPARK-5223) Use pickle instead of MapConvert and ListConvert in MLlib Python API

[ https://issues.apache.org/jira/browse/SPARK-5223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-5223: - Assignee: Davies Liu > Use pickle instead of MapConvert and ListConvert in MLlib Python API >

[jira] [Updated] (SPARK-5223) Use pickle instead of MapConvert and ListConvert in MLlib Python API

[ https://issues.apache.org/jira/browse/SPARK-5223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-5223: -- Description: It will introduce problems if the object in dict/list/tuple can not support by py4j, such

[jira] [Commented] (SPARK-5211) Restore HiveMetastoreTypes.toDataType

[ https://issues.apache.org/jira/browse/SPARK-5211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14275846#comment-14275846 ] Apache Spark commented on SPARK-5211: - User 'yhuai' has created a pull request for thi

[jira] [Updated] (SPARK-5225) Support coalesed Input Metrics from different sources

[ https://issues.apache.org/jira/browse/SPARK-5225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kostas Sakellis updated SPARK-5225: --- Description: Currently, If task reads data from more than one block and it is from different

[jira] [Created] (SPARK-5225) Support coalesed Input Metrics from different sources

Kostas Sakellis created SPARK-5225: -- Summary: Support coalesed Input Metrics from different sources Key: SPARK-5225 URL: https://issues.apache.org/jira/browse/SPARK-5225 Project: Spark Issue

[jira] [Comment Edited] (SPARK-4879) Missing output partitions after job completes with speculative execution

[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272291#comment-14272291 ] Zach Fry edited comment on SPARK-4879 at 1/13/15 7:53 PM: -- Hey Jo

[jira] [Comment Edited] (SPARK-4879) Missing output partitions after job completes with speculative execution

[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272291#comment-14272291 ] Zach Fry edited comment on SPARK-4879 at 1/13/15 7:51 PM: -- Hey Jo

[jira] [Commented] (SPARK-2909) Indexing for SparseVector in pyspark

[ https://issues.apache.org/jira/browse/SPARK-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14275741#comment-14275741 ] Apache Spark commented on SPARK-2909: - User 'MechCoder' has created a pull request for

[jira] [Commented] (SPARK-5224) parallelize list/ndarray is really slow

[ https://issues.apache.org/jira/browse/SPARK-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14275711#comment-14275711 ] Apache Spark commented on SPARK-5224: - User 'davies' has created a pull request for th

[jira] [Created] (SPARK-5224) parallelize list/ndarray is really slow

Davies Liu created SPARK-5224: - Summary: parallelize list/ndarray is really slow Key: SPARK-5224 URL: https://issues.apache.org/jira/browse/SPARK-5224 Project: Spark Issue Type: Bug Com

[jira] [Commented] (SPARK-5223) Use pickle instead of MapConvert and ListConvert in MLlib Python API

[ https://issues.apache.org/jira/browse/SPARK-5223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14275671#comment-14275671 ] Apache Spark commented on SPARK-5223: - User 'davies' has created a pull request for th

[jira] [Updated] (SPARK-4955) Dynamic allocation doesn't work in YARN cluster mode

[ https://issues.apache.org/jira/browse/SPARK-4955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4955: - Assignee: Lianhui Wang > Dynamic allocation doesn't work in YARN cluster mode > --

[jira] [Updated] (SPARK-4955) Dynamic allocation doesn't work in YARN cluster mode

[ https://issues.apache.org/jira/browse/SPARK-4955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4955: - Priority: Critical (was: Major) > Dynamic allocation doesn't work in YARN cluster mode >

[jira] [Updated] (SPARK-4955) Dynamic allocation doesn't work in YARN cluster mode

[ https://issues.apache.org/jira/browse/SPARK-4955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4955: - Summary: Dynamic allocation doesn't work in YARN cluster mode (was: Executor does not get killed after co

[jira] [Commented] (SPARK-5008) Persistent HDFS does not recognize EBS Volumes

[ https://issues.apache.org/jira/browse/SPARK-5008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14275656#comment-14275656 ] Nicholas Chammas commented on SPARK-5008: - [~brdwrd] - Thank you for documenting t

[jira] [Created] (SPARK-5223) Use pickle instead of MapConvert and ListConvert in MLlib Python API

Davies Liu created SPARK-5223: - Summary: Use pickle instead of MapConvert and ListConvert in MLlib Python API Key: SPARK-5223 URL: https://issues.apache.org/jira/browse/SPARK-5223 Project: Spark

[jira] [Updated] (SPARK-5219) Race condition in TaskSchedulerImpl and TaskSetManager

[ https://issues.apache.org/jira/browse/SPARK-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5219: - Affects Version/s: 1.2.0 > Race condition in TaskSchedulerImpl and TaskSetManager > --

[jira] [Updated] (SPARK-5219) Race condition in TaskSchedulerImpl and TaskSetManager

[ https://issues.apache.org/jira/browse/SPARK-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5219: - Assignee: Shixiong Zhu > Race condition in TaskSchedulerImpl and TaskSetManager >

[jira] [Commented] (SPARK-733) Add documentation on use of accumulators in lazy transformation

[ https://issues.apache.org/jira/browse/SPARK-733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14275608#comment-14275608 ] Apache Spark commented on SPARK-733: User 'ilganeli' has created a pull request for thi

[jira] [Commented] (SPARK-3885) Provide mechanism to remove accumulators once they are no longer used

[ https://issues.apache.org/jira/browse/SPARK-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14275607#comment-14275607 ] Apache Spark commented on SPARK-3885: - User 'ilganeli' has created a pull request for

[jira] [Commented] (SPARK-3288) All fields in TaskMetrics should be private and use getters/setters

[ https://issues.apache.org/jira/browse/SPARK-3288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14275600#comment-14275600 ] Apache Spark commented on SPARK-3288: - User 'ilganeli' has created a pull request for

[jira] [Updated] (SPARK-5222) YARN client and cluster modes have different app name behaviors

[ https://issues.apache.org/jira/browse/SPARK-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5222: - Description: The behavior is summarized in a table produced by [~WangTaoTheTonic] here: https://github.co

[jira] [Updated] (SPARK-5222) YARN client and cluster modes have different app name behaviors

[ https://issues.apache.org/jira/browse/SPARK-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5222: - Component/s: YARN > YARN client and cluster modes have different app name behaviors >

[jira] [Updated] (SPARK-5222) YARN client and cluster modes have different app name behaviors

[ https://issues.apache.org/jira/browse/SPARK-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5222: - Affects Version/s: 1.0.0 > YARN client and cluster modes have different app name behaviors > -

[jira] [Commented] (SPARK-5008) Persistent HDFS does not recognize EBS Volumes

[ https://issues.apache.org/jira/browse/SPARK-5008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14275586#comment-14275586 ] Brad Willard commented on SPARK-5008: - [~nchammas] I went ahead and created a cluster

[jira] [Created] (SPARK-5222) YARN client and cluster modes have different app name behaviors

Andrew Or created SPARK-5222: Summary: YARN client and cluster modes have different app name behaviors Key: SPARK-5222 URL: https://issues.apache.org/jira/browse/SPARK-5222 Project: Spark Issue

[jira] [Updated] (SPARK-4697) System properties should override environment variables

[ https://issues.apache.org/jira/browse/SPARK-4697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4697: - Assignee: WangTaoTheTonic > System properties should override environment variables >

[jira] [Updated] (SPARK-4697) System properties should override environment variables

[ https://issues.apache.org/jira/browse/SPARK-4697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4697: - Affects Version/s: 1.0.0 > System properties should override environment variables > -

  1   2   >