[jira] [Commented] (SPARK-11728) Replace example code in ml-ensembles.md using include_example

2015-11-13 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005225#comment-15005225 ] Xusen Yin commented on SPARK-11728: --- I'll take this. > Replace example code in ml-ense

[jira] [Commented] (SPARK-11704) Optimize the Cartesian Join

2015-11-13 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005217#comment-15005217 ] Takeshi Yamamuro commented on SPARK-11704: -- You're right; they're not automatica

[jira] [Commented] (SPARK-11725) Let UDF to handle null value

2015-11-13 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005208#comment-15005208 ] Jeff Zhang commented on SPARK-11725: Thanks [~hvanhovell] Should we prevent use primi

[jira] [Assigned] (SPARK-11729) Replace example code in ml-linear-methods.md using include_example

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11729: Assignee: Apache Spark > Replace example code in ml-linear-methods.md using include_exampl

[jira] [Assigned] (SPARK-11729) Replace example code in ml-linear-methods.md using include_example

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11729: Assignee: (was: Apache Spark) > Replace example code in ml-linear-methods.md using inc

[jira] [Commented] (SPARK-11729) Replace example code in ml-linear-methods.md using include_example

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005207#comment-15005207 ] Apache Spark commented on SPARK-11729: -- User 'yinxusen' has created a pull request f

[jira] [Assigned] (SPARK-11743) Add UserDefinedType support to RowEncoder

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11743: Assignee: Apache Spark > Add UserDefinedType support to RowEncoder > -

[jira] [Commented] (SPARK-11743) Add UserDefinedType support to RowEncoder

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005197#comment-15005197 ] Apache Spark commented on SPARK-11743: -- User 'viirya' has created a pull request for

[jira] [Assigned] (SPARK-11743) Add UserDefinedType support to RowEncoder

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11743: Assignee: (was: Apache Spark) > Add UserDefinedType support to RowEncoder > --

[jira] [Created] (SPARK-11743) Add UserDefinedType support to RowEncoder

2015-11-13 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-11743: --- Summary: Add UserDefinedType support to RowEncoder Key: SPARK-11743 URL: https://issues.apache.org/jira/browse/SPARK-11743 Project: Spark Issue Type: I

[jira] [Commented] (SPARK-11742) Show batch failures in the Streaming UI landing page

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005175#comment-15005175 ] Apache Spark commented on SPARK-11742: -- User 'zsxwing' has created a pull request fo

[jira] [Assigned] (SPARK-11742) Show batch failures in the Streaming UI landing page

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11742: Assignee: (was: Apache Spark) > Show batch failures in the Streaming UI landing page >

[jira] [Assigned] (SPARK-11742) Show batch failures in the Streaming UI landing page

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11742: Assignee: Apache Spark > Show batch failures in the Streaming UI landing page > --

[jira] [Created] (SPARK-11742) Show batch failures in the Streaming UI landing page

2015-11-13 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-11742: Summary: Show batch failures in the Streaming UI landing page Key: SPARK-11742 URL: https://issues.apache.org/jira/browse/SPARK-11742 Project: Spark Issue Ty

[jira] [Comment Edited] (SPARK-11704) Optimize the Cartesian Join

2015-11-13 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005136#comment-15005136 ] Zhan Zhang edited comment on SPARK-11704 at 11/14/15 5:16 AM: -

[jira] [Updated] (SPARK-11729) Replace example code in ml-linear-methods.md using include_example

2015-11-13 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xusen Yin updated SPARK-11729: -- Description: (was: Process these two markdown files in one JIRA issue because they have fewer code

[jira] [Updated] (SPARK-11729) Replace example code in ml-linear-methods.md using include_example

2015-11-13 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xusen Yin updated SPARK-11729: -- Summary: Replace example code in ml-linear-methods.md using include_example (was: Replace example code

[jira] [Commented] (SPARK-11729) Replace example code in ml-linear-methods.md and ml-ann.md using include_example

2015-11-13 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005138#comment-15005138 ] Xusen Yin commented on SPARK-11729: --- OK I'll take it. > Replace example code in ml-lin

[jira] [Commented] (SPARK-11704) Optimize the Cartesian Join

2015-11-13 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005136#comment-15005136 ] Zhan Zhang commented on SPARK-11704: I think we can add a cleanup hook in SQLContext,

[jira] [Commented] (SPARK-11704) Optimize the Cartesian Join

2015-11-13 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005134#comment-15005134 ] Zhan Zhang commented on SPARK-11704: [~maropu] Maybe I misunderstand. If RDD2 is comi

[jira] [Commented] (SPARK-11704) Optimize the Cartesian Join

2015-11-13 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005111#comment-15005111 ] Takeshi Yamamuro commented on SPARK-11704: -- ISTM that some earlier stages in rdd

[jira] [Resolved] (SPARK-7970) Optimize code for SQL queries fired on Union of RDDs (closure cleaner)

2015-11-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-7970. -- Resolution: Fixed Fix Version/s: 1.6.0 Target Version/s: 1.6.0 > Optimize code for SQL

[jira] [Commented] (SPARK-11702) Guava ClassLoading Issue When Using Different Hive Metastore Version

2015-11-13 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005074#comment-15005074 ] Marcelo Vanzin commented on SPARK-11702: Sean, it's a real bug. He's not trying t

[jira] [Reopened] (SPARK-11702) Guava ClassLoading Issue When Using Different Hive Metastore Version

2015-11-13 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reopened SPARK-11702: > Guava ClassLoading Issue When Using Different Hive Metastore Version > --

[jira] [Commented] (SPARK-11153) Turns off Parquet filter push-down for string and binary columns

2015-11-13 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005023#comment-15005023 ] Cheng Lian commented on SPARK-11153: Good question. We tried, see [PR #9225|https://

[jira] [Comment Edited] (SPARK-10863) Method coltypes() to return the R column types of a DataFrame

2015-11-13 Thread Oscar D. Lara Yejas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005019#comment-15005019 ] Oscar D. Lara Yejas edited comment on SPARK-10863 at 11/14/15 1:19 AM:

[jira] [Commented] (SPARK-10863) Method coltypes() to return the R column types of a DataFrame

2015-11-13 Thread Oscar D. Lara Yejas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005019#comment-15005019 ] Oscar D. Lara Yejas commented on SPARK-10863: - [~felixcheung] I think a solut

[jira] [Commented] (SPARK-11153) Turns off Parquet filter push-down for string and binary columns

2015-11-13 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005005#comment-15005005 ] Cheng Lian commented on SPARK-11153: Yes. > Turns off Parquet filter push-down for s

[jira] [Commented] (SPARK-11153) Turns off Parquet filter push-down for string and binary columns

2015-11-13 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005004#comment-15005004 ] Cheng Lian commented on SPARK-11153: Yes. > Turns off Parquet filter push-down for s

[jira] [Issue Comment Deleted] (SPARK-11153) Turns off Parquet filter push-down for string and binary columns

2015-11-13 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-11153: --- Comment: was deleted (was: Yes.) > Turns off Parquet filter push-down for string and binary columns

[jira] [Commented] (SPARK-11741) Process doctests using TextTestRunner/XMLTestRunner

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004986#comment-15004986 ] Apache Spark commented on SPARK-11741: -- User 'gliptak' has created a pull request fo

[jira] [Assigned] (SPARK-11741) Process doctests using TextTestRunner/XMLTestRunner

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11741: Assignee: (was: Apache Spark) > Process doctests using TextTestRunner/XMLTestRunner >

[jira] [Assigned] (SPARK-11741) Process doctests using TextTestRunner/XMLTestRunner

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11741: Assignee: Apache Spark > Process doctests using TextTestRunner/XMLTestRunner > ---

[jira] [Created] (SPARK-11741) Process doctests using TextTestRunner/XMLTestRunner

2015-11-13 Thread Gabor Liptak (JIRA)
Gabor Liptak created SPARK-11741: Summary: Process doctests using TextTestRunner/XMLTestRunner Key: SPARK-11741 URL: https://issues.apache.org/jira/browse/SPARK-11741 Project: Spark Issue Typ

[jira] [Commented] (SPARK-11648) IllegalReferenceCountException in Spark workloads

2015-11-13 Thread Nishkam Ravi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004976#comment-15004976 ] Nishkam Ravi commented on SPARK-11648: -- The updated patch seems to have resolved thi

[jira] [Resolved] (SPARK-11648) IllegalReferenceCountException in Spark workloads

2015-11-13 Thread Nishkam Ravi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishkam Ravi resolved SPARK-11648. -- Resolution: Fixed > IllegalReferenceCountException in Spark workloads > ---

[jira] [Commented] (SPARK-11153) Turns off Parquet filter push-down for string and binary columns

2015-11-13 Thread Mark Hamstra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004944#comment-15004944 ] Mark Hamstra commented on SPARK-11153: -- Is there a reason why parquet.version hasn't

[jira] [Commented] (SPARK-11583) Make MapStatus use less memory uage

2015-11-13 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004924#comment-15004924 ] Reynold Xin commented on SPARK-11583: - I will get somebody to take a look at this and

[jira] [Assigned] (SPARK-11569) StringIndexer transform fails when column contains nulls

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11569: Assignee: Apache Spark > StringIndexer transform fails when column contains nulls > --

[jira] [Assigned] (SPARK-11569) StringIndexer transform fails when column contains nulls

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11569: Assignee: (was: Apache Spark) > StringIndexer transform fails when column contains nul

[jira] [Commented] (SPARK-11569) StringIndexer transform fails when column contains nulls

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004912#comment-15004912 ] Apache Spark commented on SPARK-11569: -- User 'jliwork' has created a pull request fo

[jira] [Updated] (SPARK-8582) Optimize checkpointing to avoid computing an RDD twice

2015-11-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8582: - Target Version/s: 1.7.0 (was: 1.6.0) > Optimize checkpointing to avoid computing an RDD twice > -

[jira] [Commented] (SPARK-8582) Optimize checkpointing to avoid computing an RDD twice

2015-11-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004874#comment-15004874 ] Andrew Or commented on SPARK-8582: -- Hi everyone, I have bumped this to 1.7.0 because of t

[jira] [Commented] (SPARK-11648) IllegalReferenceCountException in Spark workloads

2015-11-13 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004866#comment-15004866 ] Marcelo Vanzin commented on SPARK-11648: I'm pretty sure the fix for SPARK-11617

[jira] [Commented] (SPARK-11740) Fix DStream checkpointing logic to prevent failures during checkpoint recovery

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004849#comment-15004849 ] Apache Spark commented on SPARK-11740: -- User 'zsxwing' has created a pull request fo

[jira] [Assigned] (SPARK-11740) Fix DStream checkpointing logic to prevent failures during checkpoint recovery

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11740: Assignee: Apache Spark > Fix DStream checkpointing logic to prevent failures during checkp

[jira] [Assigned] (SPARK-11740) Fix DStream checkpointing logic to prevent failures during checkpoint recovery

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11740: Assignee: (was: Apache Spark) > Fix DStream checkpointing logic to prevent failures du

[jira] [Created] (SPARK-11740) Fix DStream checkpointing logic to prevent failures during checkpoint recovery

2015-11-13 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-11740: Summary: Fix DStream checkpointing logic to prevent failures during checkpoint recovery Key: SPARK-11740 URL: https://issues.apache.org/jira/browse/SPARK-11740 Projec

[jira] [Updated] (SPARK-7308) Should there be multiple concurrent attempts for one stage?

2015-11-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-7308: - Assignee: Davies Liu > Should there be multiple concurrent attempts for one stage? > -

[jira] [Assigned] (SPARK-11739) Dead SQLContext may not be cleared

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11739: Assignee: Apache Spark (was: Davies Liu) > Dead SQLContext may not be cleared > -

[jira] [Assigned] (SPARK-11739) Dead SQLContext may not be cleared

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11739: Assignee: Davies Liu (was: Apache Spark) > Dead SQLContext may not be cleared > -

[jira] [Commented] (SPARK-11739) Dead SQLContext may not be cleared

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004807#comment-15004807 ] Apache Spark commented on SPARK-11739: -- User 'davies' has created a pull request for

[jira] [Created] (SPARK-11739) Dead SQLContext may not be cleared

2015-11-13 Thread Davies Liu (JIRA)
Davies Liu created SPARK-11739: -- Summary: Dead SQLContext may not be cleared Key: SPARK-11739 URL: https://issues.apache.org/jira/browse/SPARK-11739 Project: Spark Issue Type: Bug Comp

[jira] [Commented] (SPARK-7308) Should there be multiple concurrent attempts for one stage?

2015-11-13 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004798#comment-15004798 ] Imran Rashid commented on SPARK-7308: - yeah I think you are right, I just marked it as

[jira] [Resolved] (SPARK-7308) Should there be multiple concurrent attempts for one stage?

2015-11-13 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-7308. - Resolution: Fixed Assignee: (was: Imran Rashid) Fix Version/s: 1.6.0

[jira] [Assigned] (SPARK-11720) Return Double.NaN instead of null for Mean and Average when count = 0

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11720: Assignee: Apache Spark > Return Double.NaN instead of null for Mean and Average when count

[jira] [Commented] (SPARK-11720) Return Double.NaN instead of null for Mean and Average when count = 0

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004791#comment-15004791 ] Apache Spark commented on SPARK-11720: -- User 'JihongMA' has created a pull request f

[jira] [Assigned] (SPARK-11720) Return Double.NaN instead of null for Mean and Average when count = 0

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11720: Assignee: (was: Apache Spark) > Return Double.NaN instead of null for Mean and Average

[jira] [Commented] (SPARK-7308) Should there be multiple concurrent attempts for one stage?

2015-11-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004783#comment-15004783 ] Andrew Or commented on SPARK-7308: -- Should this still be open given that all associated J

[jira] [Updated] (SPARK-8029) ShuffleMapTasks must be robust to concurrent attempts on the same executor

2015-11-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8029: - Description: When stages get retried, a task may have more than one attempt running at the same time, on

[jira] [Updated] (SPARK-8029) ShuffleMapTasks must be robust to concurrent attempts on the same executor

2015-11-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8029: - Description: When stages get retried, a task may have more than one attempt running at the same time, on

[jira] [Resolved] (SPARK-7829) SortShuffleWriter writes inconsistent data & index files on stage retry

2015-11-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-7829. -- Resolution: Fixed Assignee: Davies Liu (was: Imran Rashid) Fix Version/s: 1.6.0

[jira] [Commented] (SPARK-7829) SortShuffleWriter writes inconsistent data & index files on stage retry

2015-11-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004780#comment-15004780 ] Andrew Or commented on SPARK-7829: -- I believe this is now fixed due to https://github.com

[jira] [Created] (SPARK-11738) Make array orderable

2015-11-13 Thread Yin Huai (JIRA)
Yin Huai created SPARK-11738: Summary: Make array orderable Key: SPARK-11738 URL: https://issues.apache.org/jira/browse/SPARK-11738 Project: Spark Issue Type: Improvement Components: SQ

[jira] [Commented] (SPARK-11705) Eliminate unnecessary Cartesian Join

2015-11-13 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004744#comment-15004744 ] Zhan Zhang commented on SPARK-11705: simple reproduce step: import sqlContext.implici

[jira] [Resolved] (SPARK-11336) Include a link to the source file in generated example code

2015-11-13 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-11336. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9320 [https://gi

[jira] [Updated] (SPARK-11336) Include path to the source file in generated example code

2015-11-13 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-11336: -- Description: It would be nice to include -a link- the path to the example source file at the bo

[jira] [Updated] (SPARK-11336) Include path to the source file in generated example code

2015-11-13 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-11336: -- Summary: Include path to the source file in generated example code (was: Include a link to the

[jira] [Assigned] (SPARK-11737) String may not be serialized correctly with Kyro

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11737: Assignee: Apache Spark (was: Davies Liu) > String may not be serialized correctly with Ky

[jira] [Commented] (SPARK-11737) String may not be serialized correctly with Kyro

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004734#comment-15004734 ] Apache Spark commented on SPARK-11737: -- User 'davies' has created a pull request for

[jira] [Assigned] (SPARK-11737) String may not be serialized correctly with Kyro

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11737: Assignee: Davies Liu (was: Apache Spark) > String may not be serialized correctly with Ky

[jira] [Resolved] (SPARK-11672) Flaky test: ml.JavaDefaultReadWriteSuite

2015-11-13 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-11672. --- Resolution: Fixed Issue resolved by pull request 9694 [https://github.com/apache/spark/pull/9

[jira] [Updated] (SPARK-8029) ShuffleMapTasks must be robust to concurrent attempts on the same executor

2015-11-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8029: - Target Version/s: 1.5.3, 1.6.0 (was: 1.5.2, 1.6.0) > ShuffleMapTasks must be robust to concurrent attempt

[jira] [Updated] (SPARK-8029) ShuffleMapTasks must be robust to concurrent attempts on the same executor

2015-11-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8029: - Fix Version/s: (was: 1.5.2) 1.5.3 > ShuffleMapTasks must be robust to concurrent at

[jira] [Updated] (SPARK-8029) ShuffleMapTasks must be robust to concurrent attempts on the same executor

2015-11-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8029: - Fix Version/s: 1.5.2 > ShuffleMapTasks must be robust to concurrent attempts on the same executor > --

[jira] [Created] (SPARK-11737) String may not be serialized correctly with Kyro

2015-11-13 Thread Davies Liu (JIRA)
Davies Liu created SPARK-11737: -- Summary: String may not be serialized correctly with Kyro Key: SPARK-11737 URL: https://issues.apache.org/jira/browse/SPARK-11737 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-11736) Add MonotonicallyIncreasingID to function registry

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11736: Assignee: Apache Spark (was: Yin Huai) > Add MonotonicallyIncreasingID to function regist

[jira] [Assigned] (SPARK-11736) Add MonotonicallyIncreasingID to function registry

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11736: Assignee: Yin Huai (was: Apache Spark) > Add MonotonicallyIncreasingID to function regist

[jira] [Commented] (SPARK-11736) Add MonotonicallyIncreasingID to function registry

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004662#comment-15004662 ] Apache Spark commented on SPARK-11736: -- User 'yhuai' has created a pull request for

[jira] [Created] (SPARK-11736) Add MonotonicallyIncreasingID to function registry

2015-11-13 Thread Yin Huai (JIRA)
Yin Huai created SPARK-11736: Summary: Add MonotonicallyIncreasingID to function registry Key: SPARK-11736 URL: https://issues.apache.org/jira/browse/SPARK-11736 Project: Spark Issue Type: Improv

[jira] [Commented] (SPARK-10712) JVM crashes with spark.sql.tungsten.enabled = true

2015-11-13 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004636#comment-15004636 ] Davies Liu commented on SPARK-10712: How is you small table looks like? Does 1.5.2-RC

[jira] [Commented] (SPARK-11735) Add a check in the constructor of SqlContext to make sure the SparkContext is not stopped

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004627#comment-15004627 ] Apache Spark commented on SPARK-11735: -- User 'yhuai' has created a pull request for

[jira] [Assigned] (SPARK-11735) Add a check in the constructor of SqlContext to make sure the SparkContext is not stopped

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11735: Assignee: Apache Spark > Add a check in the constructor of SqlContext to make sure the Spa

[jira] [Commented] (SPARK-9278) DataFrameWriter.insertInto inserts incorrect data

2015-11-13 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004626#comment-15004626 ] Davies Liu commented on SPARK-9278: --- cc [~lian cheng] > DataFrameWriter.insertInto inse

[jira] [Assigned] (SPARK-11735) Add a check in the constructor of SqlContext to make sure the SparkContext is not stopped

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11735: Assignee: (was: Apache Spark) > Add a check in the constructor of SqlContext to make s

[jira] [Assigned] (SPARK-11643) inserting date with leading zero inserts null example '0001-12-10'

2015-11-13 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-11643: -- Assignee: Davies Liu > inserting date with leading zero inserts null example '0001-12-10' > --

[jira] [Commented] (SPARK-11643) inserting date with leading zero inserts null example '0001-12-10'

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004622#comment-15004622 ] Apache Spark commented on SPARK-11643: -- User 'davies' has created a pull request for

[jira] [Assigned] (SPARK-11643) inserting date with leading zero inserts null example '0001-12-10'

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11643: Assignee: Apache Spark (was: Davies Liu) > inserting date with leading zero inserts null

[jira] [Assigned] (SPARK-11643) inserting date with leading zero inserts null example '0001-12-10'

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11643: Assignee: Davies Liu (was: Apache Spark) > inserting date with leading zero inserts null

[jira] [Created] (SPARK-11735) Add a check in the constructor of SqlContext to make sure the SparkContext is not stopped

2015-11-13 Thread Yin Huai (JIRA)
Yin Huai created SPARK-11735: Summary: Add a check in the constructor of SqlContext to make sure the SparkContext is not stopped Key: SPARK-11735 URL: https://issues.apache.org/jira/browse/SPARK-11735 Pro

[jira] [Commented] (SPARK-11734) Move reference sort into test and standardize on TungstenSort

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004597#comment-15004597 ] Apache Spark commented on SPARK-11734: -- User 'rxin' has created a pull request for t

[jira] [Assigned] (SPARK-11734) Move reference sort into test and standardize on TungstenSort

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11734: Assignee: Reynold Xin (was: Apache Spark) > Move reference sort into test and standardize

[jira] [Assigned] (SPARK-11734) Move reference sort into test and standardize on TungstenSort

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11734: Assignee: Apache Spark (was: Reynold Xin) > Move reference sort into test and standardize

[jira] [Created] (SPARK-11734) Move reference sort into test and standardize on TungstenSort

2015-11-13 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-11734: --- Summary: Move reference sort into test and standardize on TungstenSort Key: SPARK-11734 URL: https://issues.apache.org/jira/browse/SPARK-11734 Project: Spark

[jira] [Assigned] (SPARK-2344) Add Fuzzy C-Means algorithm to MLlib

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-2344: --- Assignee: (was: Apache Spark) > Add Fuzzy C-Means algorithm to MLlib > --

[jira] [Commented] (SPARK-2344) Add Fuzzy C-Means algorithm to MLlib

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004589#comment-15004589 ] Apache Spark commented on SPARK-2344: - User 'acflorea' has created a pull request for

[jira] [Assigned] (SPARK-2344) Add Fuzzy C-Means algorithm to MLlib

2015-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-2344: --- Assignee: Apache Spark > Add Fuzzy C-Means algorithm to MLlib > -

[jira] [Resolved] (SPARK-11727) split ExpressionEncoder into FlatEncoder and ProductEncoder

2015-11-13 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-11727. -- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9693 [http

[jira] [Updated] (SPARK-11724) Casting integer types to timestamp has unexpected semantics

2015-11-13 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-11724: Assignee: Nong Li > Casting integer types to timestamp has unexpected semantics > -

[jira] [Updated] (SPARK-11724) Casting integer types to timestamp has unexpected semantics

2015-11-13 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-11724: Labels: releasenotes (was: ) > Casting integer types to timestamp has unexpected semantics > -

  1   2   >