[jira] [Commented] (SPARK-6397) Override QueryPlan.missingInput when necessary and rely on CheckAnalysis

[ https://issues.apache.org/jira/browse/SPARK-6397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14375477#comment-14375477 ] Apache Spark commented on SPARK-6397: - User 'watermen' has created a pull request for

[jira] [Commented] (SPARK-6461) spark.executorEnv.PATH in spark-defaults.conf is not pass to mesos

-1.3.0-bin-2.4.0.tar.gz' '/home/mesos/work_dir/slaves/20150323-100710-1214949568-5050-3453-S3/frameworks/20150323-133400-1214949568-5050-15440-0007/executors/20150323-100710-1214949568-5050-3453-S3/runs/915b40d8-f7c4-428a-9df8-ac9804c6cd21/spark-1.3.0-bin-2.4.0.tar.gz' sh: hadoop: command not found

[jira] [Commented] (SPARK-6213) sql.catalyst.expressions.Expression is not friendly to java

[ https://issues.apache.org/jira/browse/SPARK-6213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14375612#comment-14375612 ] Littlestar commented on SPARK-6213: --- may be change protected[sql] def

[jira] [Issue Comment Deleted] (SPARK-6461) spark.executorEnv.PATH in spark-defaults.conf is not pass to mesos

spark.executorEnv.HADOOP_HOME spark.executorEnv.JAVA_HOME E0323 14:24:36.400635 11355 fetcher.cpp:109] HDFS copyToLocal failed: hadoop fs -copyToLocal 'hdfs://192.168.1.9:54310/home/test/spark-1.3.0-bin-2.4.0.tar.gz' '/home/mesos/work_dir/slaves/20150323-100710-1214949568-5050-3453-S3/frameworks

[jira] [Commented] (SPARK-6474) Replace image.run with connection.run_instances in spark_ec2.py

[ https://issues.apache.org/jira/browse/SPARK-6474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376579#comment-14376579 ] Andrew Drozdov commented on SPARK-6474: --- Great, and thanks. Taking a look now.

[jira] [Updated] (SPARK-6474) Replace image.run with connection.run_instances in spark_ec2.py

[ https://issues.apache.org/jira/browse/SPARK-6474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-6474: Issue Type: Improvement (was: Bug) Replace image.run with connection.run_instances in

[jira] [Updated] (SPARK-6477) Run MIMA tests before the Spark test suite

[ https://issues.apache.org/jira/browse/SPARK-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brennon York updated SPARK-6477: Issue Type: Improvement (was: Bug) Run MIMA tests before the Spark test suite

[jira] [Commented] (SPARK-6474) Replace image.run with connection.run_instances in spark_ec2.py

[ https://issues.apache.org/jira/browse/SPARK-6474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376584#comment-14376584 ] Nicholas Chammas commented on SPARK-6474: - This change also fits the pattern of

[jira] [Created] (SPARK-6476) Spark fileserver not started on same IP as using spark.driver.host

Rares Vernica created SPARK-6476: Summary: Spark fileserver not started on same IP as using spark.driver.host Key: SPARK-6476 URL: https://issues.apache.org/jira/browse/SPARK-6476 Project: Spark

[jira] [Created] (SPARK-6477) Run MIMA tests before the Spark test suite

Brennon York created SPARK-6477: --- Summary: Run MIMA tests before the Spark test suite Key: SPARK-6477 URL: https://issues.apache.org/jira/browse/SPARK-6477 Project: Spark Issue Type: Bug

[jira] [Reopened] (SPARK-6122) Upgrade Tachyon dependency to 0.6.0

[ https://issues.apache.org/jira/browse/SPARK-6122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell reopened SPARK-6122: I reverted this because it looks like it was responsible for some testing failures due to the

[jira] [Resolved] (SPARK-6308) VectorUDT is displayed as `vecto` in dtypes

[ https://issues.apache.org/jira/browse/SPARK-6308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-6308. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5118

[jira] [Updated] (SPARK-6474) Replace image.run with connection.run_instances in spark_ec2.py

[ https://issues.apache.org/jira/browse/SPARK-6474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Drozdov updated SPARK-6474: -- Summary: Replace image.run with connection.run_instances in spark_ec2.py (was: Replace

[jira] [Created] (SPARK-6474) Replace image.run with connection.run_instances

Andrew Drozdov created SPARK-6474: - Summary: Replace image.run with connection.run_instances Key: SPARK-6474 URL: https://issues.apache.org/jira/browse/SPARK-6474 Project: Spark Issue Type:

[jira] [Updated] (SPARK-6474) Replace image.run with connection.run_instances in spark_ec2.py

[ https://issues.apache.org/jira/browse/SPARK-6474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-6474: Priority: Minor (was: Major) Replace image.run with connection.run_instances in

[jira] [Comment Edited] (SPARK-6474) Replace image.run with connection.run_instances in spark_ec2.py

[ https://issues.apache.org/jira/browse/SPARK-6474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376572#comment-14376572 ] Nicholas Chammas edited comment on SPARK-6474 at 3/23/15 8:29 PM:

[jira] [Commented] (SPARK-6473) Launcher lib shouldn't try to figure out Scala version when not in dev mode

[ https://issues.apache.org/jira/browse/SPARK-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376569#comment-14376569 ] Apache Spark commented on SPARK-6473: - User 'vanzin' has created a pull request for

[jira] [Commented] (SPARK-6474) Replace image.run with connection.run_instances in spark_ec2.py

[ https://issues.apache.org/jira/browse/SPARK-6474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376572#comment-14376572 ] Nicholas Chammas commented on SPARK-6474: - LGTM. Replace image.run with

[jira] [Created] (SPARK-6475) DataFrame should support array types when creating DFs from JavaBeans.

Xiangrui Meng created SPARK-6475: Summary: DataFrame should support array types when creating DFs from JavaBeans. Key: SPARK-6475 URL: https://issues.apache.org/jira/browse/SPARK-6475 Project: Spark

[jira] [Assigned] (SPARK-6475) DataFrame should support array types when creating DFs from JavaBeans.

[ https://issues.apache.org/jira/browse/SPARK-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-6475: Assignee: Xiangrui Meng DataFrame should support array types when creating DFs from

[jira] [Commented] (SPARK-6475) DataFrame should support array types when creating DFs from JavaBeans.

[ https://issues.apache.org/jira/browse/SPARK-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376780#comment-14376780 ] Apache Spark commented on SPARK-6475: - User 'mengxr' has created a pull request for

[jira] [Commented] (SPARK-6369) InsertIntoHiveTable should use logic from SparkHadoopWriter

[ https://issues.apache.org/jira/browse/SPARK-6369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376234#comment-14376234 ] Apache Spark commented on SPARK-6369: - User 'liancheng' has created a pull request for

[jira] [Commented] (SPARK-4848) On a stand-alone cluster, several worker-specific variables are read only on the master

[ https://issues.apache.org/jira/browse/SPARK-4848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376239#comment-14376239 ] Apache Spark commented on SPARK-4848: - User 'nkronenfeld' has created a pull request

[jira] [Commented] (SPARK-5954) Add topByKey to pair RDDs

[ https://issues.apache.org/jira/browse/SPARK-5954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376215#comment-14376215 ] Xiangrui Meng commented on SPARK-5954: -- Note: We added topByKey in

[jira] [Assigned] (SPARK-6470) Allow Spark apps to put YARN node labels in their requests

[ https://issues.apache.org/jira/browse/SPARK-6470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned SPARK-6470: - Assignee: Sandy Ryza Allow Spark apps to put YARN node labels in their requests

[jira] [Commented] (SPARK-6471) Metastore schema should only be a subset of parquet schema to support dropping of columns using replace columns

[ https://issues.apache.org/jira/browse/SPARK-6471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376311#comment-14376311 ] Apache Spark commented on SPARK-6471: - User 'saucam' has created a pull request for

[jira] [Commented] (SPARK-3720) support ORC in spark sql

[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376361#comment-14376361 ] Zhan Zhang commented on SPARK-3720: --- [~iward] Since this jiar is duplicated to

[jira] [Updated] (SPARK-2331) SparkContext.emptyRDD should return RDD[T] not EmptyRDD[T]

[ https://issues.apache.org/jira/browse/SPARK-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-2331: - Priority: Minor (was: Major) Target Version/s: 2+ SparkContext.emptyRDD should return

[jira] [Reopened] (SPARK-2331) SparkContext.emptyRDD should return RDD[T] not EmptyRDD[T]

[ https://issues.apache.org/jira/browse/SPARK-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-2331: -- Reopening to catch this for version 2.x SparkContext.emptyRDD should return RDD[T] not EmptyRDD[T]

[jira] [Commented] (SPARK-4830) Spark Streaming Java Application : java.lang.ClassNotFoundException

[ https://issues.apache.org/jira/browse/SPARK-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376335#comment-14376335 ] sam commented on SPARK-4830: I've been seeing a similar problem but just for a regular Spark

[jira] [Updated] (SPARK-6471) Metastore schema should only be a subset of parquet schema to support dropping of columns using replace columns

[ https://issues.apache.org/jira/browse/SPARK-6471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Datta updated SPARK-6471: -- Summary: Metastore schema should only be a subset of parquet schema to support dropping of columns

[jira] [Created] (SPARK-6471) Metastoreschema should only be a subset of parquetSchema to support dropping of columns using replace columns

Yash Datta created SPARK-6471: - Summary: Metastoreschema should only be a subset of parquetSchema to support dropping of columns using replace columns Key: SPARK-6471 URL:

[jira] [Commented] (SPARK-4086) Fold-style aggregation for VertexRDD

[ https://issues.apache.org/jira/browse/SPARK-4086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376341#comment-14376341 ] Apache Spark commented on SPARK-4086: - User 'brennonyork' has created a pull request

[jira] [Commented] (SPARK-6469) The YARN driver in yarn-client mode will not use the local directories configured for YARN

[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376409#comment-14376409 ] Sean Owen commented on SPARK-6469: -- [~preaudc] would you like to open a small PR to add a

[jira] [Commented] (SPARK-6471) Metastore schema should only be a subset of parquet schema to support dropping of columns using replace columns

[ https://issues.apache.org/jira/browse/SPARK-6471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376286#comment-14376286 ] Yash Datta commented on SPARK-6471: --- https://github.com/apache/spark/pull/5141

[jira] [Issue Comment Deleted] (SPARK-6471) Metastore schema should only be a subset of parquet schema to support dropping of columns using replace columns

[ https://issues.apache.org/jira/browse/SPARK-6471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Datta updated SPARK-6471: -- Comment: was deleted (was: https://github.com/apache/spark/pull/5141) Metastore schema should only be

[jira] [Commented] (SPARK-6192) Enhance MLlib's Python API (GSoC 2015)

[ https://issues.apache.org/jira/browse/SPARK-6192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376329#comment-14376329 ] Manoj Kumar commented on SPARK-6192: [~mengxr] Sorry for spamming, but do you have

[jira] [Created] (SPARK-6480) histogram() bucket function is wrong in some simple edge cases

Sean Owen created SPARK-6480: Summary: histogram() bucket function is wrong in some simple edge cases Key: SPARK-6480 URL: https://issues.apache.org/jira/browse/SPARK-6480 Project: Spark Issue

[jira] [Updated] (SPARK-5508) Arrays and Maps stored with Hive Parquet Serde may not be able to read by the Parquet support in the Data Souce API

[ https://issues.apache.org/jira/browse/SPARK-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-5508: Priority: Critical (was: Major) Arrays and Maps stored with Hive Parquet Serde may not be

[jira] [Updated] (SPARK-5508) Arrays and Maps stored with Hive Parquet Serde may not be able to read by the Parquet support in the Data Souce API

[ https://issues.apache.org/jira/browse/SPARK-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-5508: Assignee: Cheng Lian Arrays and Maps stored with Hive Parquet Serde may not be able to

[jira] [Updated] (SPARK-6437) SQL ExternalSort should use CompletionIterator to clean up temp files

[ https://issues.apache.org/jira/browse/SPARK-6437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6437: Assignee: Michael Armbrust SQL ExternalSort should use CompletionIterator to clean up temp

[jira] [Resolved] (SPARK-6124) Support jdbc connection properties in OPTIONS part of the query

[ https://issues.apache.org/jira/browse/SPARK-6124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-6124. - Resolution: Fixed Fix Version/s: 1.4.0 1.3.1 Issue resolved by

[jira] [Updated] (SPARK-6112) Provide OffHeap support through HDFS RAM_DISK

[ https://issues.apache.org/jira/browse/SPARK-6112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-6112: -- Attachment: SparkOffheapsupportbyHDFS.pdf Design doc for hdfs offheap support Provide OffHeap support

[jira] [Updated] (SPARK-6479) Create off-heap block storage API (internal)

[ https://issues.apache.org/jira/browse/SPARK-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-6479: -- Attachment: SparkOffheapsupportbyHDFS.pdf The design doc also includes stuff from SPARK-6112 Create

[jira] [Assigned] (SPARK-6054) SQL UDF returning object of case class; regression from 1.2.0

[ https://issues.apache.org/jira/browse/SPARK-6054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reassigned SPARK-6054: --- Assignee: Michael Armbrust SQL UDF returning object of case class; regression from

[jira] [Commented] (SPARK-6480) histogram() bucket function is wrong in some simple edge cases

[ https://issues.apache.org/jira/browse/SPARK-6480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376894#comment-14376894 ] Apache Spark commented on SPARK-6480: - User 'srowen' has created a pull request for

[jira] [Updated] (SPARK-4925) Publish Spark SQL hive-thriftserver maven artifact

[ https://issues.apache.org/jira/browse/SPARK-4925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-4925: Assignee: Patrick Wendell Publish Spark SQL hive-thriftserver maven artifact

[jira] [Commented] (SPARK-6373) Add SSL/TLS for the Netty based BlockTransferService

[ https://issues.apache.org/jira/browse/SPARK-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376907#comment-14376907 ] Jeffrey Turpin commented on SPARK-6373: --- Hey Aaron, Thanks for the feedback! I

[jira] [Created] (SPARK-6481) Set In Progress when a PR is opened for an issue

Michael Armbrust created SPARK-6481: --- Summary: Set In Progress when a PR is opened for an issue Key: SPARK-6481 URL: https://issues.apache.org/jira/browse/SPARK-6481 Project: Spark Issue

[jira] [Created] (SPARK-6483) Spark SQL udf(ScalaUdf) is very slow

zzc created SPARK-6483: -- Summary: Spark SQL udf(ScalaUdf) is very slow Key: SPARK-6483 URL: https://issues.apache.org/jira/browse/SPARK-6483 Project: Spark Issue Type: Improvement Components:

[jira] [Commented] (SPARK-6112) Provide OffHeap support through HDFS RAM_DISK

[ https://issues.apache.org/jira/browse/SPARK-6112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376849#comment-14376849 ] Reynold Xin commented on SPARK-6112: [~zhanzhang] I created

[jira] [Updated] (SPARK-2973) Use LocalRelation for all ExecutedCommands, avoid job for take/collect()

[ https://issues.apache.org/jira/browse/SPARK-2973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2973: Priority: Blocker (was: Critical) Use LocalRelation for all ExecutedCommands, avoid job

[jira] [Updated] (SPARK-2973) Use LocalRelation for all ExecutedCommands, avoid job for take/collect()

[ https://issues.apache.org/jira/browse/SPARK-2973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2973: Summary: Use LocalRelation for all ExecutedCommands, avoid job for take/collect() (was:

[jira] [Created] (SPARK-6482) Remove synchronization of Hive Native commands

David Ross created SPARK-6482: - Summary: Remove synchronization of Hive Native commands Key: SPARK-6482 URL: https://issues.apache.org/jira/browse/SPARK-6482 Project: Spark Issue Type:

[jira] [Created] (SPARK-6478) new RDD.pipeWithPartition method

Maxim Ivanov created SPARK-6478: --- Summary: new RDD.pipeWithPartition method Key: SPARK-6478 URL: https://issues.apache.org/jira/browse/SPARK-6478 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-6478) new RDD.pipeWithPartition method

[ https://issues.apache.org/jira/browse/SPARK-6478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376827#comment-14376827 ] Apache Spark commented on SPARK-6478: - User 'redbaron' has created a pull request for

[jira] [Comment Edited] (SPARK-5928) Remote Shuffle Blocks cannot be more than 2 GB

[ https://issues.apache.org/jira/browse/SPARK-5928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376833#comment-14376833 ] Allan Douglas R. de Oliveira edited comment on SPARK-5928 at 3/23/15 10:54 PM:

[jira] [Created] (SPARK-6479) Create off-heap block storage API

Reynold Xin created SPARK-6479: -- Summary: Create off-heap block storage API Key: SPARK-6479 URL: https://issues.apache.org/jira/browse/SPARK-6479 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-6450) Native Parquet reader does not assign table name as qualifier

[ https://issues.apache.org/jira/browse/SPARK-6450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6450: Summary: Native Parquet reader does not assign table name as qualifier (was: Self joining

[jira] [Commented] (SPARK-6479) Create off-heap block storage API (internal)

[ https://issues.apache.org/jira/browse/SPARK-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376978#comment-14376978 ] Zhan Zhang commented on SPARK-6479: --- The current API may not be good enough as it has

[jira] [Commented] (SPARK-6481) Set In Progress when a PR is opened for an issue

[ https://issues.apache.org/jira/browse/SPARK-6481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377034#comment-14377034 ] Nicholas Chammas commented on SPARK-6481: - I'm guessing this will be done via

[jira] [Commented] (SPARK-6122) Upgrade Tachyon dependency to 0.6.0

[ https://issues.apache.org/jira/browse/SPARK-6122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376813#comment-14376813 ] Calvin Jia commented on SPARK-6122: --- [~pwendell] Are you referring to the issues here:

[jira] [Commented] (SPARK-5928) Remote Shuffle Blocks cannot be more than 2 GB

[ https://issues.apache.org/jira/browse/SPARK-5928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376833#comment-14376833 ] Allan Douglas R. de Oliveira commented on SPARK-5928: - I will answer

[jira] [Updated] (SPARK-6479) Create off-heap block storage API (internal)

[ https://issues.apache.org/jira/browse/SPARK-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6479: --- Summary: Create off-heap block storage API (internal) (was: Create off-heap block storage API)

[jira] [Updated] (SPARK-6369) InsertIntoHiveTable should use logic from SparkHadoopWriter

[ https://issues.apache.org/jira/browse/SPARK-6369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6369: Assignee: Cheng Lian InsertIntoHiveTable should use logic from SparkHadoopWriter

[jira] [Commented] (SPARK-1684) Merge script should standardize SPARK-XXX prefix

[ https://issues.apache.org/jira/browse/SPARK-1684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377108#comment-14377108 ] Apache Spark commented on SPARK-1684: - User 'texasmichelle' has created a pull request

[jira] [Commented] (SPARK-6100) Distributed linear algebra in PySpark/MLlib

[ https://issues.apache.org/jira/browse/SPARK-6100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377208#comment-14377208 ] Xiangrui Meng commented on SPARK-6100: -- We don't have APIs for distributed matrices

[jira] [Created] (SPARK-6488) Support addition/multiplication in PySpark's BlockMatrix

Xiangrui Meng created SPARK-6488: Summary: Support addition/multiplication in PySpark's BlockMatrix Key: SPARK-6488 URL: https://issues.apache.org/jira/browse/SPARK-6488 Project: Spark Issue

[jira] [Commented] (SPARK-6449) Driver OOM results in reported application result SUCCESS

[ https://issues.apache.org/jira/browse/SPARK-6449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377249#comment-14377249 ] Ryan Williams commented on SPARK-6449: -- It doesn't look like it; [here is a

[jira] [Commented] (SPARK-6449) Driver OOM results in reported application result SUCCESS

[ https://issues.apache.org/jira/browse/SPARK-6449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377251#comment-14377251 ] Ryan Williams commented on SPARK-6449: -- Seems like this was fixed as of

[jira] [Updated] (SPARK-6475) DataFrame should support array types when creating DFs from JavaBeans.

[ https://issues.apache.org/jira/browse/SPARK-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6475: Component/s: (was: DataFrame) DataFrame should support array types when creating DFs

[jira] [Updated] (SPARK-5941) `def table` is not using the unresolved logical plan `UnresolvedRelation`

[ https://issues.apache.org/jira/browse/SPARK-5941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-5941: Component/s: (was: DataFrame) SQL `def table` is not using the

[jira] [Commented] (SPARK-6484) Ganglia metrics xml reporter doesn't escape correctly

[ https://issues.apache.org/jira/browse/SPARK-6484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377080#comment-14377080 ] Josh Rosen commented on SPARK-6484: --- To provide some extra context for this JIRA, I

[jira] [Comment Edited] (SPARK-5368) Spark should support NAT (via akka improvements)

[ https://issues.apache.org/jira/browse/SPARK-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14375163#comment-14375163 ] jay vyas edited comment on SPARK-5368 at 3/24/15 1:59 AM: -- Okay,

[jira] [Commented] (SPARK-5368) Spark should support NAT (via akka improvements)

[ https://issues.apache.org/jira/browse/SPARK-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377096#comment-14377096 ] Matthew Farrellee commented on SPARK-5368: -- [~jayunit100] the relevant config is

[jira] [Commented] (SPARK-6361) Support adding a column with metadata in DataFrames

[ https://issues.apache.org/jira/browse/SPARK-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377187#comment-14377187 ] Apache Spark commented on SPARK-6361: - User 'mengxr' has created a pull request for

[jira] [Created] (SPARK-6485) Add CoordinateMatrix/RowMatrix/IndexedRowMatrix in PySpark

Xiangrui Meng created SPARK-6485: Summary: Add CoordinateMatrix/RowMatrix/IndexedRowMatrix in PySpark Key: SPARK-6485 URL: https://issues.apache.org/jira/browse/SPARK-6485 Project: Spark

[jira] [Updated] (SPARK-6486) Add BlockMatrix in PySpark

[ https://issues.apache.org/jira/browse/SPARK-6486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6486: - Description: We should add BlockMatrix to PySpark. Internally, we can use DataFrames and

[jira] [Created] (SPARK-6487) Add sequential pattern mining algorithm to Spark MLlib

Zhang JiaJin created SPARK-6487: --- Summary: Add sequential pattern mining algorithm to Spark MLlib Key: SPARK-6487 URL: https://issues.apache.org/jira/browse/SPARK-6487 Project: Spark Issue

[jira] [Created] (SPARK-6489) Optimize lateral view with explode to not read unnecessary columns

Konstantin Shaposhnikov created SPARK-6489: -- Summary: Optimize lateral view with explode to not read unnecessary columns Key: SPARK-6489 URL: https://issues.apache.org/jira/browse/SPARK-6489

[jira] [Commented] (SPARK-6487) Add sequential pattern mining algorithm to Spark MLlib

[ https://issues.apache.org/jira/browse/SPARK-6487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377234#comment-14377234 ] Xiangrui Meng commented on SPARK-6487: -- [~Zhang JiaJin] I'm not very familiar with

[jira] [Updated] (SPARK-6489) Optimize lateral view with explode to not read unnecessary columns

[ https://issues.apache.org/jira/browse/SPARK-6489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shaposhnikov updated SPARK-6489: --- Description: Currently a query with lateral view explode(...) results in an

[jira] [Commented] (SPARK-6352) Supporting non-default OutputCommitter when using saveAsParquetFile

[ https://issues.apache.org/jira/browse/SPARK-6352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377264#comment-14377264 ] Pei-Lun Lee commented on SPARK-6352: The above PR adds a new hadoop config value

[jira] [Created] (SPARK-6484) Ganglia metrics xml reporter doesn't escape correctly

Michael Armbrust created SPARK-6484: --- Summary: Ganglia metrics xml reporter doesn't escape correctly Key: SPARK-6484 URL: https://issues.apache.org/jira/browse/SPARK-6484 Project: Spark

[jira] [Commented] (SPARK-5368) Spark should support NAT (via akka improvements)

[ https://issues.apache.org/jira/browse/SPARK-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377106#comment-14377106 ] jay vyas commented on SPARK-5368: - looks like this is subsumed maybe by the work going on

[jira] [Commented] (SPARK-6430) Cannot resolve column correctlly when using left semi join

[ https://issues.apache.org/jira/browse/SPARK-6430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377139#comment-14377139 ] zzc commented on SPARK-6430: what's wrong with this? Cannot resolve column correctlly when

[jira] [Commented] (SPARK-3720) support ORC in spark sql

[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377155#comment-14377155 ] iward commented on SPARK-3720: -- [~zhanzhang], I see. since the patch is delayed, so we can't

[jira] [Commented] (SPARK-3735) Sending the factor directly or AtA based on the cost in ALS

[ https://issues.apache.org/jira/browse/SPARK-3735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377200#comment-14377200 ] Xiangrui Meng commented on SPARK-3735: -- The proposal is actually something different.

[jira] [Commented] (SPARK-3278) Isotonic regression

[ https://issues.apache.org/jira/browse/SPARK-3278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377205#comment-14377205 ] Xiangrui Meng commented on SPARK-3278: -- Did you try truncating the digits of x to

[jira] [Commented] (SPARK-6464) Add a new transformation of rdd named processCoalesce which was particularly to deal with the small and cached rdd

[ https://issues.apache.org/jira/browse/SPARK-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377206#comment-14377206 ] Apache Spark commented on SPARK-6464: - User 'SaintBacchus' has created a pull request

[jira] [Updated] (SPARK-6487) Add sequential pattern mining algorithm to Spark MLlib

[ https://issues.apache.org/jira/browse/SPARK-6487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhang JiaJin updated SPARK-6487: Description: [~mengxr] [~zhangyouhua] Sequential pattern mining is an important branch in the

[jira] [Resolved] (SPARK-6449) Driver OOM results in reported application result SUCCESS

[ https://issues.apache.org/jira/browse/SPARK-6449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Williams resolved SPARK-6449. -- Resolution: Implemented Fix Version/s: 1.3.0 Driver OOM results in reported application

[jira] [Updated] (SPARK-5692) Model import/export for Word2Vec

[ https://issues.apache.org/jira/browse/SPARK-5692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-5692: - Assignee: Manoj Kumar (was: ANUPAM MEDIRATTA) Model import/export for Word2Vec

[jira] [Commented] (SPARK-5692) Model import/export for Word2Vec

[ https://issues.apache.org/jira/browse/SPARK-5692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377266#comment-14377266 ] Xiangrui Meng commented on SPARK-5692: -- [~anupamme] You should get familiar with

[jira] [Updated] (SPARK-6189) Pandas to DataFrame conversion should check field names for periods

[ https://issues.apache.org/jira/browse/SPARK-6189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6189: Component/s: (was: DataFrame) Pandas to DataFrame conversion should check field names

[jira] [Updated] (SPARK-5919) Enable broadcast joins for Parquet files

[ https://issues.apache.org/jira/browse/SPARK-5919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-5919: Component/s: (was: DataFrame) SQL Enable broadcast joins for Parquet

[jira] [Commented] (SPARK-6229) Support SASL encryption in network/common module

[ https://issues.apache.org/jira/browse/SPARK-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377102#comment-14377102 ] Marcelo Vanzin commented on SPARK-6229: --- Hi, me again. So I finally got back to

[jira] [Updated] (SPARK-1684) Merge script should standardize SPARK-XXX prefix

[ https://issues.apache.org/jira/browse/SPARK-1684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michelle Casbon updated SPARK-1684: --- Attachment: spark_pulls_before_after.txt Test data (spark_pulls_before_after.txt): titles

[jira] [Commented] (SPARK-6192) Enhance MLlib's Python API (GSoC 2015)

[ https://issues.apache.org/jira/browse/SPARK-6192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377197#comment-14377197 ] Xiangrui Meng commented on SPARK-6192: -- Thanks for the update! The current version

[jira] [Resolved] (SPARK-6334) spark-local dir not getting cleared during ALS

[ https://issues.apache.org/jira/browse/SPARK-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-6334. -- Resolution: Duplicate SPARK-5955 was merged. So if you can use the latest master, you can set

<    1   2   3   >