[jira] [Commented] (SPARK-6765) Turn scalastyle on for test code

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14484785#comment-14484785 ] Apache Spark commented on SPARK-6765: - User 'rxin' has created a pull request for this

[jira] [Commented] (SPARK-6764) Add wheel package support for PySpark

2015-04-08 Thread Takao Magoori (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14484786#comment-14484786 ] Takao Magoori commented on SPARK-6764: -- Note:

[jira] [Commented] (SPARK-6765) Turn scalastyle on for test code

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14484800#comment-14484800 ] Apache Spark commented on SPARK-6765: - User 'rxin' has created a pull request for this

[jira] [Commented] (SPARK-6677) pyspark.sql nondeterministic issue with row fields

2015-04-08 Thread Stefano Parmesan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14484835#comment-14484835 ] Stefano Parmesan commented on SPARK-6677: - An update: on a more complex script I'm

[jira] [Commented] (SPARK-6733) Suppression of usage of Scala existential code should be done

2015-04-08 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14484841#comment-14484841 ] Vinod KC commented on SPARK-6733: - Can you please assign this to me? Suppression of

[jira] [Commented] (SPARK-6765) Turn scalastyle on for test code

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14484852#comment-14484852 ] Apache Spark commented on SPARK-6765: - User 'rxin' has created a pull request for this

[jira] [Assigned] (SPARK-6765) Turn scalastyle on for test code

2015-04-08 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-6765: -- Assignee: Reynold Xin Turn scalastyle on for test code

[jira] [Issue Comment Deleted] (SPARK-1499) Workers continuously produce failing executors

2015-04-08 Thread XeonZhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XeonZhao updated SPARK-1499: Comment: was deleted (was: How to solve this problem ?I have encountered this issue.) Workers

[jira] [Assigned] (SPARK-6773) check -license will passed in next time when rat jar download failed.

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6773: --- Assignee: (was: Apache Spark) check -license will passed in next time when rat jar

[jira] [Commented] (SPARK-6773) check -license will passed in next time when rat jar download failed.

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485156#comment-14485156 ] Apache Spark commented on SPARK-6773: - User 'sisihj' has created a pull request for

[jira] [Commented] (SPARK-5960) Allow AWS credentials to be passed to KinesisUtils.createStream()

2015-04-08 Thread Cihan Emre (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485168#comment-14485168 ] Cihan Emre commented on SPARK-5960: --- [~cfregly] Are you working on this? I might be able

[jira] [Commented] (SPARK-6762) Fix potential resource leaks in CheckPoint CheckpointWriter and CheckpointReader

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14484900#comment-14484900 ] Apache Spark commented on SPARK-6762: - User 'zhichao-li' has created a pull request

[jira] [Commented] (SPARK-6767) Documentation error in Spark SQL Readme file

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14484946#comment-14484946 ] Apache Spark commented on SPARK-6767: - User 'tijoparacka' has created a pull request

[jira] [Commented] (SPARK-1499) Workers continuously produce failing executors

2015-04-08 Thread XeonZhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14484963#comment-14484963 ] XeonZhao commented on SPARK-1499: - How to solve this problem ?I have encountered this

[jira] [Updated] (SPARK-6768) Do not support float/double union decimal and decimal(a ,b) union decimal(c, d)

2015-04-08 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongshuai Pei updated SPARK-6768: -- Description: Do not support sql like that : ``` select cast(12.2056999 as float) from testData

[jira] [Updated] (SPARK-6768) Do not support float/double union decimal or decimal(a ,b) union decimal(c, d)

2015-04-08 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongshuai Pei updated SPARK-6768: -- Summary: Do not support float/double union decimal or decimal(a ,b) union decimal(c, d) (was:

[jira] [Updated] (SPARK-6768) Do not support float/double union decimal or decimal(a ,b) union decimal(c, d)

2015-04-08 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongshuai Pei updated SPARK-6768: -- Description: Do not support sql like that : select cast(12.2056999 as float) from testData

[jira] [Assigned] (SPARK-6769) Usage of the ListenerBus in YarnClusterSuite is wrong

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6769: --- Assignee: Apache Spark Usage of the ListenerBus in YarnClusterSuite is wrong

[jira] [Updated] (SPARK-6773) check -license will passed in next time when rat jar download failed.

2015-04-08 Thread June (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] June updated SPARK-6773: Description: In dev/check-license, it will download Rat jar if it not exist. if download failed, it will report

[jira] [Created] (SPARK-6773) check -license will passed in next time when rat jar download failed.

2015-04-08 Thread June (JIRA)
June created SPARK-6773: --- Summary: check -license will passed in next time when rat jar download failed. Key: SPARK-6773 URL: https://issues.apache.org/jira/browse/SPARK-6773 Project: Spark Issue

[jira] [Created] (SPARK-6772) spark sql error when running code on large number of records

2015-04-08 Thread Aditya Parmar (JIRA)
Aditya Parmar created SPARK-6772: Summary: spark sql error when running code on large number of records Key: SPARK-6772 URL: https://issues.apache.org/jira/browse/SPARK-6772 Project: Spark

[jira] [Assigned] (SPARK-6773) check -license will passed in next time when rat jar download failed.

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6773: --- Assignee: Apache Spark check -license will passed in next time when rat jar download

[jira] [Created] (SPARK-6771) Table alias in Spark SQL

2015-04-08 Thread Jacky19820629 (JIRA)
Jacky19820629 created SPARK-6771: Summary: Table alias in Spark SQL Key: SPARK-6771 URL: https://issues.apache.org/jira/browse/SPARK-6771 Project: Spark Issue Type: Question

[jira] [Resolved] (SPARK-6681) JAVA_HOME error with upgrade to Spark 1.3.0

2015-04-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6681. -- Resolution: Cannot Reproduce We can reopen this if there is a follow-up with more detail but looks like

[jira] [Commented] (SPARK-6681) JAVA_HOME error with upgrade to Spark 1.3.0

2015-04-08 Thread Ken Williams (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485170#comment-14485170 ] Ken Williams commented on SPARK-6681: - Sounds good - I haven't been able to

[jira] [Commented] (SPARK-6766) StreamingListenerBatchSubmitted isn't sent and StreamingListenerBatchStarted.batchInfo.processingStartTime is a wrong value

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14484898#comment-14484898 ] Apache Spark commented on SPARK-6766: - User 'zsxwing' has created a pull request for

[jira] [Assigned] (SPARK-6766) StreamingListenerBatchSubmitted isn't sent and StreamingListenerBatchStarted.batchInfo.processingStartTime is a wrong value

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6766: --- Assignee: (was: Apache Spark) StreamingListenerBatchSubmitted isn't sent and

[jira] [Assigned] (SPARK-6766) StreamingListenerBatchSubmitted isn't sent and StreamingListenerBatchStarted.batchInfo.processingStartTime is a wrong value

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6766: --- Assignee: Apache Spark StreamingListenerBatchSubmitted isn't sent and

[jira] [Created] (SPARK-6768) Do not support float/double union decimal and decimal(a ,b) union decimal(c, d)

2015-04-08 Thread Zhongshuai Pei (JIRA)
Zhongshuai Pei created SPARK-6768: - Summary: Do not support float/double union decimal and decimal(a ,b) union decimal(c, d) Key: SPARK-6768 URL: https://issues.apache.org/jira/browse/SPARK-6768

[jira] [Created] (SPARK-6769) Usage of the ListenerBus in YarnClusterSuite is wrong

2015-04-08 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-6769: - Summary: Usage of the ListenerBus in YarnClusterSuite is wrong Key: SPARK-6769 URL: https://issues.apache.org/jira/browse/SPARK-6769 Project: Spark Issue

[jira] [Updated] (SPARK-6768) Do not support float/double union decimal and decimal(a ,b) union decimal(c, d)

2015-04-08 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongshuai Pei updated SPARK-6768: -- Description: Do not support sql like that : ``` select cast(12.2056999 as float) from testData

[jira] [Updated] (SPARK-6770) DirectKafkaInputDStream has not been initialized when recovery from checkpoint

2015-04-08 Thread yangping wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yangping wu updated SPARK-6770: --- Description: I am read data from kafka using createDirectStream method and save the received log to

[jira] [Updated] (SPARK-6770) DirectKafkaInputDStream has not been initialized when recovery from checkpoint

2015-04-08 Thread yangping wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yangping wu updated SPARK-6770: --- Description: I am read the data from kafka using createDirectStream method and save the received

[jira] [Created] (SPARK-6770) DirectKafkaInputDStream has not been initialized when recovery from checkpoint

2015-04-08 Thread yangping wu (JIRA)
yangping wu created SPARK-6770: -- Summary: DirectKafkaInputDStream has not been initialized when recovery from checkpoint Key: SPARK-6770 URL: https://issues.apache.org/jira/browse/SPARK-6770 Project:

[jira] [Updated] (SPARK-6770) DirectKafkaInputDStream has not been initialized when recovery from checkpoint

2015-04-08 Thread yangping wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yangping wu updated SPARK-6770: --- Description: I am read the data from kafka using createDirectStream method and save the received

[jira] [Assigned] (SPARK-6769) Usage of the ListenerBus in YarnClusterSuite is wrong

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6769: --- Assignee: (was: Apache Spark) Usage of the ListenerBus in YarnClusterSuite is wrong

[jira] [Commented] (SPARK-1499) Workers continuously produce failing executors

2015-04-08 Thread XeonZhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14484964#comment-14484964 ] XeonZhao commented on SPARK-1499: - How to solve this problem ?I have encountered this

[jira] [Commented] (SPARK-6769) Usage of the ListenerBus in YarnClusterSuite is wrong

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14484966#comment-14484966 ] Apache Spark commented on SPARK-6769: - User 'sarutak' has created a pull request for

[jira] [Commented] (SPARK-1499) Workers continuously produce failing executors

2015-04-08 Thread XeonZhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14484962#comment-14484962 ] XeonZhao commented on SPARK-1499: - How to solve this problem ?I have encountered this

[jira] [Updated] (SPARK-6768) Do not support float/double union decimal and decimal(a ,b) union decimal(c, d)

2015-04-08 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongshuai Pei updated SPARK-6768: -- Description: Do not support sql like that : ... select cast(12.2056999 as float) from testData

[jira] [Updated] (SPARK-6768) Do not support float/double union decimal and decimal(a ,b) union decimal(c, d)

2015-04-08 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongshuai Pei updated SPARK-6768: -- Description: Do not support sql like that : ... select cast(12.2056999 as float) from testData

[jira] [Updated] (SPARK-6733) Suppression of usage of Scala existential code should be done

2015-04-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6733: - Assignee: Vinod KC Suppression of usage of Scala existential code should be done

[jira] [Commented] (SPARK-6762) Fix potential resource leaks in CheckPoint CheckpointWriter and CheckpointReader

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14484875#comment-14484875 ] Apache Spark commented on SPARK-6762: - User 'nareshpr' has created a pull request for

[jira] [Assigned] (SPARK-6762) Fix potential resource leaks in CheckPoint CheckpointWriter and CheckpointReader

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6762: --- Assignee: (was: Apache Spark) Fix potential resource leaks in CheckPoint

[jira] [Assigned] (SPARK-6762) Fix potential resource leaks in CheckPoint CheckpointWriter and CheckpointReader

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6762: --- Assignee: Apache Spark Fix potential resource leaks in CheckPoint CheckpointWriter and

[jira] [Created] (SPARK-6766) StreamingListenerBatchSubmitted isn't sent and StreamingListenerBatchStarted.batchInfo.processingStartTime is a wrong value

2015-04-08 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-6766: --- Summary: StreamingListenerBatchSubmitted isn't sent and StreamingListenerBatchStarted.batchInfo.processingStartTime is a wrong value Key: SPARK-6766 URL:

[jira] [Assigned] (SPARK-6767) Documentation error in Spark SQL Readme file

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6767: --- Assignee: (was: Apache Spark) Documentation error in Spark SQL Readme file

[jira] [Assigned] (SPARK-6767) Documentation error in Spark SQL Readme file

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6767: --- Assignee: Apache Spark Documentation error in Spark SQL Readme file

[jira] [Updated] (SPARK-6769) Usage of the ListenerBus in YarnClusterSuite is wrong

2015-04-08 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-6769: -- Component/s: (was: Spark Core) Usage of the ListenerBus in YarnClusterSuite is wrong

[jira] [Commented] (SPARK-6768) Do not support float/double union decimal or decimal(a ,b) union decimal(c, d)

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14484967#comment-14484967 ] Apache Spark commented on SPARK-6768: - User 'DoingDone9' has created a pull request

[jira] [Assigned] (SPARK-6768) Do not support float/double union decimal or decimal(a ,b) union decimal(c, d)

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6768: --- Assignee: Apache Spark Do not support float/double union decimal or decimal(a ,b) union

[jira] [Assigned] (SPARK-6768) Do not support float/double union decimal or decimal(a ,b) union decimal(c, d)

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6768: --- Assignee: (was: Apache Spark) Do not support float/double union decimal or decimal(a

[jira] [Created] (SPARK-6767) Documentation error in Spark SQL Readme file

2015-04-08 Thread Tijo Thomas (JIRA)
Tijo Thomas created SPARK-6767: -- Summary: Documentation error in Spark SQL Readme file Key: SPARK-6767 URL: https://issues.apache.org/jira/browse/SPARK-6767 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-6699) PySpark Acess Denied error in windows seen only in ver 1.3

2015-04-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6699. -- Resolution: Not A Problem This looks strongly like a local permissions issue and/or lack of numpy, but

[jira] [Assigned] (SPARK-5957) Better handling of default parameter values.

2015-04-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-5957: Assignee: Xiangrui Meng Better handling of default parameter values.

[jira] [Updated] (SPARK-5874) How to improve the current ML pipeline API?

2015-04-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-5874: - Description: I created this JIRA to collect feedbacks about the ML pipeline API we introduced in

[jira] [Updated] (SPARK-5957) Better handling of default parameter values.

2015-04-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-5957: - Description: We store the default value of a parameter in the Param instance. In many cases, the

[jira] [Commented] (SPARK-6682) Deprecate static train and use builder instead for Scala/Java

2015-04-08 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485554#comment-14485554 ] Alexander Ulanov commented on SPARK-6682: - [~yuu.ishik...@gmail.com] They reside

[jira] [Resolved] (SPARK-6506) python support yarn cluster mode requires SPARK_HOME to be set

2015-04-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-6506. --- Resolution: Fixed Fix Version/s: 1.4.0 1.3.2 Issue resolved by pull request

[jira] [Updated] (SPARK-6506) python support yarn cluster mode requires SPARK_HOME to be set

2015-04-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-6506: -- Assignee: Marcelo Vanzin python support yarn cluster mode requires SPARK_HOME to be set

[jira] [Created] (SPARK-6780) Add saveAsTextFileByKey method for PySpark

2015-04-08 Thread Ilya Ganelin (JIRA)
Ilya Ganelin created SPARK-6780: --- Summary: Add saveAsTextFileByKey method for PySpark Key: SPARK-6780 URL: https://issues.apache.org/jira/browse/SPARK-6780 Project: Spark Issue Type:

[jira] [Commented] (SPARK-6780) Add saveAsTextFileByKey method for PySpark

2015-04-08 Thread Ilya Ganelin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485577#comment-14485577 ] Ilya Ganelin commented on SPARK-6780: - SPARK-3533 defines matching methods for Scala

[jira] [Commented] (SPARK-6780) Add saveAsTextFileByKey method for PySpark

2015-04-08 Thread Ilya Ganelin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485582#comment-14485582 ] Ilya Ganelin commented on SPARK-6780: - This code was my attempt to implement this

[jira] [Commented] (SPARK-6780) Add saveAsTextFileByKey method for PySpark

2015-04-08 Thread Ilya Ganelin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485586#comment-14485586 ] Ilya Ganelin commented on SPARK-6780: - Matching test code: {code}

[jira] [Updated] (SPARK-6753) Unit test for SPARK-3426 (in ShuffleSuite) doesn't correctly clone the SparkConf

2015-04-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-6753: -- Fix Version/s: 1.4.0 1.3.2 1.1.2 1.2.3 Unit

[jira] [Resolved] (SPARK-6753) Unit test for SPARK-3426 (in ShuffleSuite) doesn't correctly clone the SparkConf

2015-04-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-6753. --- Resolution: Fixed Fixed by https://github.com/apache/spark/pull/5401 Unit test for SPARK-3426 (in

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485258#comment-14485258 ] Apache Spark commented on SPARK-1537: - User 'steveloughran' has created a pull request

[jira] [Assigned] (SPARK-1537) Add integration with Yarn's Application Timeline Server

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-1537: --- Assignee: Marcelo Vanzin (was: Apache Spark) Add integration with Yarn's Application

[jira] [Updated] (SPARK-6776) Implement backwards-compatibility rules in CatalystConverters

2015-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6776: -- Description: Spark SQL should also be able to read Parquet complex types represented in several

[jira] [Commented] (SPARK-6776) Implement backwards-compatibility rules in CatalystConverters

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485221#comment-14485221 ] Apache Spark commented on SPARK-6776: - User 'liancheng' has created a pull request for

[jira] [Assigned] (SPARK-6775) Simplify CatalystConverter class hierarchy and pass in Parquet schema

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6775: --- Assignee: Apache Spark (was: Cheng Lian) Simplify CatalystConverter class hierarchy and

[jira] [Assigned] (SPARK-6776) Implement backwards-compatibility rules in CatalystConverters

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6776: --- Assignee: Apache Spark (was: Cheng Lian) Implement backwards-compatibility rules in

[jira] [Assigned] (SPARK-6776) Implement backwards-compatibility rules in CatalystConverters

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6776: --- Assignee: Cheng Lian (was: Apache Spark) Implement backwards-compatibility rules in

[jira] [Commented] (SPARK-6775) Simplify CatalystConverter class hierarchy and pass in Parquet schema

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485219#comment-14485219 ] Apache Spark commented on SPARK-6775: - User 'liancheng' has created a pull request for

[jira] [Created] (SPARK-6774) Implement Parquet complex types backwards-compatiblity rules

2015-04-08 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-6774: - Summary: Implement Parquet complex types backwards-compatiblity rules Key: SPARK-6774 URL: https://issues.apache.org/jira/browse/SPARK-6774 Project: Spark Issue

[jira] [Updated] (SPARK-6774) Implement Parquet complex types backwards-compatiblity rules

2015-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6774: -- Description: [Parquet format PR #17|https://github.com/apache/incubator-parquet-format/pull/17]

[jira] [Created] (SPARK-6777) Implement backwards-compatibility rules in Parquet schema converters

2015-04-08 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-6777: - Summary: Implement backwards-compatibility rules in Parquet schema converters Key: SPARK-6777 URL: https://issues.apache.org/jira/browse/SPARK-6777 Project: Spark

[jira] [Assigned] (SPARK-6775) Simplify CatalystConverter class hierarchy and pass in Parquet schema

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6775: --- Assignee: Cheng Lian (was: Apache Spark) Simplify CatalystConverter class hierarchy and

[jira] [Created] (SPARK-6775) Simplify CatalystConverter class hierarchy and pass in Parquet schema

2015-04-08 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-6775: - Summary: Simplify CatalystConverter class hierarchy and pass in Parquet schema Key: SPARK-6775 URL: https://issues.apache.org/jira/browse/SPARK-6775 Project: Spark

[jira] [Created] (SPARK-6776) Implement backwards-compatibility rules in CatalystConverters

2015-04-08 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-6776: - Summary: Implement backwards-compatibility rules in CatalystConverters Key: SPARK-6776 URL: https://issues.apache.org/jira/browse/SPARK-6776 Project: Spark Issue

[jira] [Comment Edited] (SPARK-5114) Should Evaluator be a PipelineStage

2015-04-08 Thread Peter Rudenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14483335#comment-14483335 ] Peter Rudenko edited comment on SPARK-5114 at 4/8/15 2:14 PM: --

[jira] [Commented] (SPARK-6440) ipv6 URI for HttpServer

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485750#comment-14485750 ] Apache Spark commented on SPARK-6440: - User 'nyaapa' has created a pull request for

[jira] [Created] (SPARK-6781) sqlCtx - sqlContext in pyspark shell

2015-04-08 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-6781: --- Summary: sqlCtx - sqlContext in pyspark shell Key: SPARK-6781 URL: https://issues.apache.org/jira/browse/SPARK-6781 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-6781) sqlCtx - sqlContext in pyspark shell

2015-04-08 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485757#comment-14485757 ] Michael Armbrust commented on SPARK-6781: - [~pwendell] suggests that we might

[jira] [Commented] (SPARK-6677) pyspark.sql nondeterministic issue with row fields

2015-04-08 Thread Stefano Parmesan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485772#comment-14485772 ] Stefano Parmesan commented on SPARK-6677: - Hi Davies, Thanks for taking the time

[jira] [Updated] (SPARK-6677) pyspark.sql nondeterministic issue with row fields

2015-04-08 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6677: Assignee: Davies Liu pyspark.sql nondeterministic issue with row fields

[jira] [Commented] (SPARK-5594) SparkException: Failed to get broadcast (TorrentBroadcast)

2015-04-08 Thread Zach Fry (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485780#comment-14485780 ] Zach Fry commented on SPARK-5594: - [~pwendell], We saw this while running a long Spark

[jira] [Commented] (SPARK-6781) sqlCtx - sqlContext in pyspark shell

2015-04-08 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485788#comment-14485788 ] Davies Liu commented on SPARK-6781: --- We use `sqlCtx` a lot in doc tests (will be in API

[jira] [Commented] (SPARK-6781) sqlCtx - sqlContext in pyspark shell

2015-04-08 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485801#comment-14485801 ] Michael Armbrust commented on SPARK-6781: - Yeah, I think it would be better to be

[jira] [Assigned] (SPARK-6781) sqlCtx - sqlContext in pyspark shell

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6781: --- Assignee: Apache Spark (was: Davies Liu) sqlCtx - sqlContext in pyspark shell

[jira] [Assigned] (SPARK-6781) sqlCtx - sqlContext in pyspark shell

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6781: --- Assignee: Davies Liu (was: Apache Spark) sqlCtx - sqlContext in pyspark shell

[jira] [Commented] (SPARK-6781) sqlCtx - sqlContext in pyspark shell

2015-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485803#comment-14485803 ] Apache Spark commented on SPARK-6781: - User 'davies' has created a pull request for

[jira] [Resolved] (SPARK-6757) spark.sql.shuffle.partitions is global, not per connection

2015-04-08 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-6757. - Resolution: Duplicate spark.sql.shuffle.partitions is global, not per connection

[jira] [Updated] (SPARK-5594) SparkException: Failed to get broadcast (TorrentBroadcast)

2015-04-08 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-5594: -- Description: I am uncertain whether this is a bug, however I am getting the error below when running

[jira] [Created] (SPARK-6778) SQL contexts in spark-shell and pyspark should both be called sqlContext

2015-04-08 Thread Matei Zaharia (JIRA)
Matei Zaharia created SPARK-6778: Summary: SQL contexts in spark-shell and pyspark should both be called sqlContext Key: SPARK-6778 URL: https://issues.apache.org/jira/browse/SPARK-6778 Project:

[jira] [Commented] (SPARK-6677) pyspark.sql nondeterministic issue with row fields

2015-04-08 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485521#comment-14485521 ] Davies Liu commented on SPARK-6677: --- What is the expected output? I got this: {code}

[jira] [Created] (SPARK-6779) Move shared params to param.shared and use code gen

2015-04-08 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-6779: Summary: Move shared params to param.shared and use code gen Key: SPARK-6779 URL: https://issues.apache.org/jira/browse/SPARK-6779 Project: Spark Issue

[jira] [Commented] (SPARK-6772) spark sql error when running code on large number of records

2015-04-08 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485708#comment-14485708 ] Michael Armbrust commented on SPARK-6772: - Can you provide your code?

[jira] [Updated] (SPARK-6772) spark sql error when running code on large number of records

2015-04-08 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6772: Description: Hi all , I am getting an Arrayoutboundsindex error when i try to run a simple

[jira] [Created] (SPARK-6782) add sbt-revolver plugin to sbt build

2015-04-08 Thread Imran Rashid (JIRA)
Imran Rashid created SPARK-6782: --- Summary: add sbt-revolver plugin to sbt build Key: SPARK-6782 URL: https://issues.apache.org/jira/browse/SPARK-6782 Project: Spark Issue Type: Improvement

  1   2   3   >