[jira] [Resolved] (SPARK-5706) Support inference schema from a single json string

2015-02-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5706. -- Resolution: Duplicate > Support inference schema from a single json string > ---

[jira] [Resolved] (SPARK-4336) auto detect type from json string

2015-02-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4336. -- Resolution: Won't Fix Per PR discussion, WontFix due to concerns over speed > auto detect type from jso

[jira] [Updated] (SPARK-5735) Replace uses of EasyMock with Mockito

2015-02-10 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5735: --- Assignee: Josh Rosen > Replace uses of EasyMock with Mockito > ---

[jira] [Created] (SPARK-5735) Replace uses of EasyMock with Mockito

2015-02-10 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-5735: -- Summary: Replace uses of EasyMock with Mockito Key: SPARK-5735 URL: https://issues.apache.org/jira/browse/SPARK-5735 Project: Spark Issue Type: Improveme

[jira] [Updated] (SPARK-4382) Add locations parameter to Twitter Stream

2015-02-10 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4382: --- Component/s: Streaming > Add locations parameter to Twitter Stream > -

[jira] [Commented] (SPARK-5016) GaussianMixtureEM should distribute matrix inverse for large numFeatures, k

2015-02-10 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315695#comment-14315695 ] Manoj Kumar commented on SPARK-5016: [~tgaloppo] How about a method setParallelGaussia

[jira] [Updated] (SPARK-5677) Python DataFrame API remaining tasks

2015-02-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-5677: --- Description: - DataFrame.renameColumn - DataFrame.show (also we should override __repr__ or __str__) -

[jira] [Commented] (SPARK-5522) Accelerate the Histroty Server start

2015-02-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315686#comment-14315686 ] Apache Spark commented on SPARK-5522: - User 'marsishandsome' has created a pull reques

[jira] [Commented] (SPARK-5522) Accelerate the Histroty Server start

2015-02-10 Thread Mars Gu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315683#comment-14315683 ] Mars Gu commented on SPARK-5522: https://github.com/apache/spark/pull/4525 > Accelerate t

[jira] [Updated] (SPARK-5522) Accelerate the Histroty Server start

2015-02-10 Thread Mars Gu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mars Gu updated SPARK-5522: --- Description: When starting the history server, all the log files will be fetched and parsed in order to get t

[jira] [Resolved] (SPARK-5568) Python API for the write support of the data source API

2015-02-10 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-5568. - Resolution: Fixed Fix Version/s: 1.3.0 It has been resolved by https://github.com/apache/spark/pull

[jira] [Updated] (SPARK-5183) Document data source API

2015-02-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-5183: --- Description: We need to document the data types the caller needs to support. > Document data source AP

[jira] [Commented] (SPARK-3688) LogicalPlan can't resolve column correctlly

2015-02-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315625#comment-14315625 ] Apache Spark commented on SPARK-3688: - User 'tianyi' has created a pull request for th

[jira] [Commented] (SPARK-5733) Error Link in Pagination of HistroyPage when showing Incomplete Applications

2015-02-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315614#comment-14315614 ] Apache Spark commented on SPARK-5733: - User 'marsishandsome' has created a pull reques

[jira] [Commented] (SPARK-5733) Error Link in Pagination of HistroyPage when showing Incomplete Applications

2015-02-10 Thread Mars Gu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315611#comment-14315611 ] Mars Gu commented on SPARK-5733: https://github.com/apache/spark/pull/4523 > Error Link i

[jira] [Closed] (SPARK-4945) Add overwrite option support for SchemaRDD.saveAsParquetFile

2015-02-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-4945. -- Resolution: Implemented > Add overwrite option support for SchemaRDD.saveAsParquetFile > ---

[jira] [Resolved] (SPARK-5714) Refactor initial step of LDA to remove redundant operations

2015-02-10 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-5714. -- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4501 [https://githu

[jira] [Updated] (SPARK-5714) Refactor initial step of LDA to remove redundant operations

2015-02-10 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-5714: - Assignee: Liang-Chi Hsieh > Refactor initial step of LDA to remove redundant operations >

[jira] [Updated] (SPARK-3688) LogicalPlan can't resolve column correctlly

2015-02-10 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Tian updated SPARK-3688: --- Description: How to reproduce this problem: {code} CREATE TABLE t1(x INT); CREATE TABLE t2(a STRUCT, k INT); S

[jira] [Created] (SPARK-5734) Allow creating a DataFrame from local Python data

2015-02-10 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-5734: -- Summary: Allow creating a DataFrame from local Python data Key: SPARK-5734 URL: https://issues.apache.org/jira/browse/SPARK-5734 Project: Spark Issue Type: Sub-t

[jira] [Updated] (SPARK-5677) Python DataFrame API remaining tasks

2015-02-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-5677: --- Description: - DataFrame.renameColumn - DataFrame.show (also we should override __repr__ or __str__)

[jira] [Resolved] (SPARK-5702) Allow short names for built-in data sources

2015-02-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-5702. Resolution: Fixed Fix Version/s: 1.3.0 > Allow short names for built-in data sources > --

[jira] [Commented] (SPARK-5654) Integrate SparkR into Apache Spark

2015-02-10 Thread Jason Dai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315605#comment-14315605 ] Jason Dai commented on SPARK-5654: -- I agree with this proposal. Given all ongoing efforts

[jira] [Commented] (SPARK-5732) Add an option to print the spark version in spark script

2015-02-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315585#comment-14315585 ] Apache Spark commented on SPARK-5732: - User 'uncleGen' has created a pull request for

[jira] [Created] (SPARK-5733) Error Link in Pagination of HistroyPage when showing Incomplete Applications

2015-02-10 Thread Mars Gu (JIRA)
Mars Gu created SPARK-5733: -- Summary: Error Link in Pagination of HistroyPage when showing Incomplete Applications Key: SPARK-5733 URL: https://issues.apache.org/jira/browse/SPARK-5733 Project: Spark

[jira] [Updated] (SPARK-5732) Add an option to print the spark version in spark script

2015-02-10 Thread uncleGen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] uncleGen updated SPARK-5732: Summary: Add an option to print the spark version in spark script (was: Add a option to print the spark ver

[jira] [Updated] (SPARK-5732) Add a option to print the spark version in spark script

2015-02-10 Thread uncleGen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] uncleGen updated SPARK-5732: Description: Naturally, we may need to add a option to print the spark version in spark script. It is pretty

[jira] [Updated] (SPARK-5732) Add a option to print the spark version in spark script

2015-02-10 Thread uncleGen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] uncleGen updated SPARK-5732: Description: Naturally, we may need to add an option to print the spark version in spark script. It is prett

[jira] [Updated] (SPARK-5732) Add a option to print the spark version in spark script

2015-02-10 Thread uncleGen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] uncleGen updated SPARK-5732: Description: Naturally, we may need to add a option to > Add a option to print the spark version in spark s

[jira] [Updated] (SPARK-5732) Add a option to print the spark version in spark script

2015-02-10 Thread uncleGen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] uncleGen updated SPARK-5732: Description: Naturally, we may need to add a option to print the spark version in spark script. It (was: Na

[jira] [Created] (SPARK-5732) Add a option to print the spark version in spark script

2015-02-10 Thread uncleGen (JIRA)
uncleGen created SPARK-5732: --- Summary: Add a option to print the spark version in spark script Key: SPARK-5732 URL: https://issues.apache.org/jira/browse/SPARK-5732 Project: Spark Issue Type: Impro

[jira] [Commented] (SPARK-5722) Infer_schema_type incorrect for Integers in pyspark

2015-02-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315569#comment-14315569 ] Apache Spark commented on SPARK-5722: - User 'dondrake' has created a pull request for

[jira] [Closed] (SPARK-5729) Potential NPE in StandaloneRestServer if user specifies bad path

2015-02-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-5729. Resolution: Fixed Fix Version/s: 1.3.0 > Potential NPE in StandaloneRestServer if user specifies bad

[jira] [Updated] (SPARK-4879) Missing output partitions after job completes with speculative execution

2015-02-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4879: - Labels: backport-needed (was: ) > Missing output partitions after job completes with speculative executio

[jira] [Updated] (SPARK-4879) Missing output partitions after job completes with speculative execution

2015-02-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4879: - Fix Version/s: 1.3.0 > Missing output partitions after job completes with speculative execution >

[jira] [Commented] (SPARK-5613) YarnClientSchedulerBackend fails to get application report when yarn restarts

2015-02-10 Thread Kashish Jain (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315545#comment-14315545 ] Kashish Jain commented on SPARK-5613: - Thanks Patrick and Andrew > YarnClientSchedule

[jira] [Created] (SPARK-5731) Flaky Test: org.apache.spark.streaming.kafka.DirectKafkaStreamSuite.basic stream receiving with multiple topics and smallest starting offset

2015-02-10 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-5731: -- Summary: Flaky Test: org.apache.spark.streaming.kafka.DirectKafkaStreamSuite.basic stream receiving with multiple topics and smallest starting offset Key: SPARK-5731 URL: htt

[jira] [Updated] (SPARK-5731) Flaky Test: org.apache.spark.streaming.kafka.DirectKafkaStreamSuite.basic stream receiving with multiple topics and smallest starting offset

2015-02-10 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5731: --- Affects Version/s: 1.3.0 > Flaky Test: org.apache.spark.streaming.kafka.DirectKafkaStreamSuite

[jira] [Updated] (SPARK-5731) Flaky Test: org.apache.spark.streaming.kafka.DirectKafkaStreamSuite.basic stream receiving with multiple topics and smallest starting offset

2015-02-10 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5731: --- Component/s: Tests > Flaky Test: org.apache.spark.streaming.kafka.DirectKafkaStreamSuite.basic

[jira] [Resolved] (SPARK-5709) Add "EXPLAIN" support for DataFrame API for debugging purpose

2015-02-10 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5709. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4496 [https:/

[jira] [Updated] (SPARK-5679) Flaky tests in InputOutputMetricsSuite: input metrics with interleaved reads and input metrics with mixed read method

2015-02-10 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5679: --- Priority: Major (was: Blocker) > Flaky tests in InputOutputMetricsSuite: input metrics with i

[jira] [Commented] (SPARK-5443) jsonRDD with schema should ignore sub-objects that are omitted in schema

2015-02-10 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315530#comment-14315530 ] Yin Huai commented on SPARK-5443: - Yeah, I think we can improve performance by only constr

[jira] [Resolved] (SPARK-5704) createDataFrame replace applySchema/inferSchema

2015-02-10 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5704. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4498 [https:/

[jira] [Commented] (SPARK-5706) Support inference schema from a single json string

2015-02-10 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315524#comment-14315524 ] Yin Huai commented on SPARK-5706: - Is https://issues.apache.org/jira/browse/SPARK-4336 sam

[jira] [Resolved] (SPARK-5576) saveAsTable into Hive fails due to duplicate columns

2015-02-10 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-5576. - Resolution: Won't Fix I am resolving it per discussions in the PR (https://github.com/apache/spark/pull/4

[jira] [Commented] (SPARK-5454) [SQL] Self join with ArrayType columns problems

2015-02-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315515#comment-14315515 ] Apache Spark commented on SPARK-5454: - User 'marmbrus' has created a pull request for

[jira] [Created] (SPARK-5730) Group methods in the generated doc for spark.ml algorithms.

2015-02-10 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-5730: Summary: Group methods in the generated doc for spark.ml algorithms. Key: SPARK-5730 URL: https://issues.apache.org/jira/browse/SPARK-5730 Project: Spark Is

[jira] [Commented] (SPARK-1302) httpd doesn't start in spark-ec2 (cc2.8xlarge)

2015-02-10 Thread Greg Temchenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315447#comment-14315447 ] Greg Temchenko commented on SPARK-1302: --- I'm getting this httpd error on t2.medium i

[jira] [Resolved] (SPARK-5683) Improve the json serialization for DataFrame API

2015-02-10 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5683. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4468 [https:/

[jira] [Commented] (SPARK-5729) Potential NPE in StandaloneRestServer if user specifies bad path

2015-02-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315428#comment-14315428 ] Apache Spark commented on SPARK-5729: - User 'andrewor14' has created a pull request fo

[jira] [Created] (SPARK-5729) Potential NPE in StandaloneRestServer if user specifies bad path

2015-02-10 Thread Andrew Or (JIRA)
Andrew Or created SPARK-5729: Summary: Potential NPE in StandaloneRestServer if user specifies bad path Key: SPARK-5729 URL: https://issues.apache.org/jira/browse/SPARK-5729 Project: Spark Issue

[jira] [Updated] (SPARK-5155) Python API for MQTT streaming

2015-02-10 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-5155: - Assignee: Prabeesh K > Python API for MQTT streaming > - > >

[jira] [Commented] (SPARK-5728) MQTTStreamSuite leaves behind ActiveMQ database files

2015-02-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315360#comment-14315360 ] Apache Spark commented on SPARK-5728: - User 'srowen' has created a pull request for th

[jira] [Resolved] (SPARK-5658) Finalize DDL and write support APIs

2015-02-10 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5658. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4446 [https:/

[jira] [Created] (SPARK-5728) MQTTStreamSuite leaves behind ActiveMQ database files

2015-02-10 Thread Sean Owen (JIRA)
Sean Owen created SPARK-5728: Summary: MQTTStreamSuite leaves behind ActiveMQ database files Key: SPARK-5728 URL: https://issues.apache.org/jira/browse/SPARK-5728 Project: Spark Issue Type: Bug

[jira] [Closed] (SPARK-5717) add sc.stop to LDA examples

2015-02-10 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuhao yang closed SPARK-5717. - merged. Thanks > add sc.stop to LDA examples > --- > > Key: SPARK-571

[jira] [Resolved] (SPARK-5687) in TaskResultGetter need to catch OutOfMemoryError.

2015-02-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5687. -- Resolution: Won't Fix Resolving WontFix per PR discussion. > in TaskResultGetter need to catch OutOfMem

[jira] [Resolved] (SPARK-5493) Support proxy users under kerberos

2015-02-10 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5493. Resolution: Fixed Fix Version/s: 1.3.0 Target Version/s: 1.3.0 > Support pr

[jira] [Updated] (SPARK-5493) Support proxy users under kerberos

2015-02-10 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5493: --- Assignee: Marcelo Vanzin > Support proxy users under kerberos > --

[jira] [Commented] (SPARK-5081) Shuffle write increases

2015-02-10 Thread Kevin Jung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315341#comment-14315341 ] Kevin Jung commented on SPARK-5081: --- Xuefeng Wu mentioned about one difference of snappy

[jira] [Resolved] (SPARK-5725) ParquetRelation2.equals throws when compared with non-Parquet relations

2015-02-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-5725. --- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4513 [https://github.com/

[jira] [Updated] (SPARK-5243) Spark will hang if (driver memory + executor memory) exceeds limit on a 1-worker cluster

2015-02-10 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuhao yang updated SPARK-5243: -- Description: Spark will hang if calling spark-submit under the conditions: 1. the cluster has only one

[jira] [Issue Comment Deleted] (SPARK-5682) Reuse hadoop encrypted shuffle algorithm to enable spark encrypted shuffle

2015-02-10 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated SPARK-5682: Comment: was deleted (was: encrypted_shuffle.patch.4 is how to reuse hadoop encrypted class

[jira] [Updated] (SPARK-5243) Spark will hang if (driver memory + executor memory) exceeds limit on a 1-worker cluster

2015-02-10 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuhao yang updated SPARK-5243: -- Description: Spark will hang if calling spark-submit under the conditions: 1. the cluster has only one

[jira] [Comment Edited] (SPARK-5081) Shuffle write increases

2015-02-10 Thread Kevin Jung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14308620#comment-14308620 ] Kevin Jung edited comment on SPARK-5081 at 2/11/15 12:56 AM: -

[jira] [Commented] (SPARK-5727) Deprecate, remove Debian packaging

2015-02-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315294#comment-14315294 ] Apache Spark commented on SPARK-5727: - User 'srowen' has created a pull request for th

[jira] [Created] (SPARK-5727) Deprecate, remove Debian packaging

2015-02-10 Thread Sean Owen (JIRA)
Sean Owen created SPARK-5727: Summary: Deprecate, remove Debian packaging Key: SPARK-5727 URL: https://issues.apache.org/jira/browse/SPARK-5727 Project: Spark Issue Type: Task Component

[jira] [Created] (SPARK-5726) Hadamard Vector Product Transformer

2015-02-10 Thread Octavian Geagla (JIRA)
Octavian Geagla created SPARK-5726: -- Summary: Hadamard Vector Product Transformer Key: SPARK-5726 URL: https://issues.apache.org/jira/browse/SPARK-5726 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-4682) Consolidate various 'Clock' classes

2015-02-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315162#comment-14315162 ] Apache Spark commented on SPARK-4682: - User 'srowen' has created a pull request for th

[jira] [Resolved] (SPARK-5644) Delete tmp dir when sc is stop

2015-02-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5644. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 4412 [https://github.com/ap

[jira] [Commented] (SPARK-5725) ParquetRelation2.equals throws when compared with non-Parquet relations

2015-02-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315155#comment-14315155 ] Apache Spark commented on SPARK-5725: - User 'liancheng' has created a pull request for

[jira] [Created] (SPARK-5725) ParquetRelation2.equals throws when compared with non-Parquet relations

2015-02-10 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-5725: - Summary: ParquetRelation2.equals throws when compared with non-Parquet relations Key: SPARK-5725 URL: https://issues.apache.org/jira/browse/SPARK-5725 Project: Spark

[jira] [Resolved] (SPARK-5343) ShortestPaths traverses backwards

2015-02-10 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-5343. --- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4478 https://github.com/a

[jira] [Commented] (SPARK-5724) misconfiguration in Akka system

2015-02-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315135#comment-14315135 ] Apache Spark commented on SPARK-5724: - User 'CodingCat' has created a pull request for

[jira] [Created] (SPARK-5724) misconfiguration in Akka system

2015-02-10 Thread Nan Zhu (JIRA)
Nan Zhu created SPARK-5724: -- Summary: misconfiguration in Akka system Key: SPARK-5724 URL: https://issues.apache.org/jira/browse/SPARK-5724 Project: Spark Issue Type: Bug Components: Spark

[jira] [Commented] (SPARK-4879) Missing output partitions after job completes with speculative execution

2015-02-10 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315087#comment-14315087 ] Andrew Ash commented on SPARK-4879: --- This is really great work [~joshrosen]! I really a

[jira] [Commented] (SPARK-4879) Missing output partitions after job completes with speculative execution

2015-02-10 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315077#comment-14315077 ] Josh Rosen commented on SPARK-4879: --- This issue is _really_ hard to reproduce, but I man

[jira] [Resolved] (SPARK-5021) GaussianMixtureEM should be faster for SparseVector input

2015-02-10 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-5021. -- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4459 [https://githu

[jira] [Commented] (SPARK-4705) Driver retries in yarn-cluster mode always fail if event logging is enabled

2015-02-10 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315024#comment-14315024 ] Marcelo Vanzin commented on SPARK-4705: --- Hi [~twinkle], I think the UI on the lates

[jira] [Updated] (SPARK-4879) Missing output partitions after job completes with speculative execution

2015-02-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4879: - Target Version/s: 1.0.3, 1.3.0, 1.1.2, 1.2.2 (was: 1.0.3, 1.3.0, 1.1.2, 1.2.1) > Missing output partitio

[jira] [Closed] (SPARK-4136) Under dynamic allocation, cancel outstanding executor requests when no longer needed

2015-02-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-4136. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Sandy Ryza > Under dynamic allocation, canc

[jira] [Resolved] (SPARK-5686) Support `show current roles`

2015-02-10 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5686. - Resolution: Fixed Issue resolved by pull request 4471 [https://github.com/apache/spark/pul

[jira] [Updated] (SPARK-5722) Infer_schema_type incorrect for Integers in pyspark

2015-02-10 Thread Don Drake (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Don Drake updated SPARK-5722: - Description: The Integers datatype in Python does not match what a Scala/Java integer is defined as. Th

[jira] [Updated] (SPARK-5722) Infer_schema_type incorrect for Integers in pyspark

2015-02-10 Thread Don Drake (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Don Drake updated SPARK-5722: - Summary: Infer_schema_type incorrect for Integers in pyspark (was: Infer_schma_type incorrect for Integer

[jira] [Commented] (SPARK-5613) YarnClientSchedulerBackend fails to get application report when yarn restarts

2015-02-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14314922#comment-14314922 ] Andrew Or commented on SPARK-5613: -- Thanks Patrick. I just verified that it was merged in

[jira] [Closed] (SPARK-5613) YarnClientSchedulerBackend fails to get application report when yarn restarts

2015-02-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-5613. Resolution: Fixed > YarnClientSchedulerBackend fails to get application report when yarn restarts >

[jira] [Created] (SPARK-5723) Change the default file format to Parquet for CTAS statements.

2015-02-10 Thread Yin Huai (JIRA)
Yin Huai created SPARK-5723: --- Summary: Change the default file format to Parquet for CTAS statements. Key: SPARK-5723 URL: https://issues.apache.org/jira/browse/SPARK-5723 Project: Spark Issue Typ

[jira] [Commented] (SPARK-4964) Exactly-once + WAL-free Kafka Support in Spark Streaming

2015-02-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14314903#comment-14314903 ] Apache Spark commented on SPARK-4964: - User 'koeninger' has created a pull request for

[jira] [Closed] (SPARK-3754) Spark Streaming fileSystem API is not callable from Java

2015-02-10 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das closed SPARK-3754. Resolution: Duplicate > Spark Streaming fileSystem API is not callable from Java > -

[jira] [Commented] (SPARK-3754) Spark Streaming fileSystem API is not callable from Java

2015-02-10 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14314899#comment-14314899 ] Tathagata Das commented on SPARK-3754: -- Yes, it is. This has already been fixed. >

[jira] [Created] (SPARK-5722) Infer_schma_type incorrect for Integers in pyspark

2015-02-10 Thread Don Drake (JIRA)
Don Drake created SPARK-5722: Summary: Infer_schma_type incorrect for Integers in pyspark Key: SPARK-5722 URL: https://issues.apache.org/jira/browse/SPARK-5722 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-5613) YarnClientSchedulerBackend fails to get application report when yarn restarts

2015-02-10 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14314821#comment-14314821 ] Patrick Wendell commented on SPARK-5613: I have cherry picked it into the 1.3 bran

[jira] [Commented] (SPARK-5645) Track local bytes read for shuffles - update UI

2015-02-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14314811#comment-14314811 ] Apache Spark commented on SPARK-5645: - User 'kayousterhout' has created a pull request

[jira] [Closed] (SPARK-5668) spark_ec2.py region parameter could be either mandatory or its value displayed

2015-02-10 Thread Miguel Peralvo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miguel Peralvo closed SPARK-5668. - > spark_ec2.py region parameter could be either mandatory or its value displayed > ---

[jira] [Updated] (SPARK-5592) java.net.URISyntaxException when insert data to a partitioned table

2015-02-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-5592: -- Description: {code} create table sc as select * from (select '2011-01-11', '2011-01-11+14:18:26' from s

[jira] [Resolved] (SPARK-5592) java.net.URISyntaxException when insert data to a partitioned table

2015-02-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-5592. --- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4368 [https://github.com/

[jira] [Updated] (SPARK-5592) java.net.URISyntaxException when insert data to a partitioned table

2015-02-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-5592: -- Assignee: wangfei > java.net.URISyntaxException when insert data to a partitioned table >

[jira] [Resolved] (SPARK-5668) spark_ec2.py region parameter could be either mandatory or its value displayed

2015-02-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5668. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 4457 [https://github.com/ap

[jira] [Created] (SPARK-5721) Propagate missing external shuffle service errors to client

2015-02-10 Thread Kostas Sakellis (JIRA)
Kostas Sakellis created SPARK-5721: -- Summary: Propagate missing external shuffle service errors to client Key: SPARK-5721 URL: https://issues.apache.org/jira/browse/SPARK-5721 Project: Spark

[jira] [Resolved] (SPARK-5716) Support TOK_CHARSETLITERAL in HiveQl

2015-02-10 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5716. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4502 [https:/

  1   2   >