[jira] [Commented] (SPARK-16643) When doing Shuffle, report "java.io.FileNotFoundException"

2016-07-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387272#comment-15387272 ] Sean Owen commented on SPARK-16643: --- Could be. There are others related to FileNotFound

[jira] [Commented] (SPARK-16659) use Maven project to submit spark application via yarn-client

2016-07-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387270#comment-15387270 ] Sean Owen commented on SPARK-16659: --- I think there is probably an earlier error here. T

[jira] [Commented] (SPARK-16658) Add EdgePartition.withVertexAttributes

2016-07-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387268#comment-15387268 ] Sean Owen commented on SPARK-16658: --- As a point of process, we'd need to be careful tha

[jira] [Commented] (SPARK-16652) JVM crash from unsafe memory access for Dataset of class with List[Long]

2016-07-20 Thread Liwei Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387261#comment-15387261 ] Liwei Lin commented on SPARK-16652: --- Not sure about whether this relates to SPARK-16634

[jira] [Comment Edited] (SPARK-16652) JVM crash from unsafe memory access for Dataset of class with List[Long]

2016-07-20 Thread Liwei Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387258#comment-15387258 ] Liwei Lin edited comment on SPARK-16652 at 7/21/16 6:43 AM: H

[jira] [Commented] (SPARK-16652) JVM crash from unsafe memory access for Dataset of class with List[Long]

2016-07-20 Thread Liwei Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387258#comment-15387258 ] Liwei Lin commented on SPARK-16652: --- Hi [~dsbos], thanks for reporting this. This could

[jira] [Assigned] (SPARK-16660) CreateViewCommand should not take CatalogTable

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16660: Assignee: Apache Spark (was: Wenchen Fan) > CreateViewCommand should not take CatalogTabl

[jira] [Assigned] (SPARK-16660) CreateViewCommand should not take CatalogTable

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16660: Assignee: Wenchen Fan (was: Apache Spark) > CreateViewCommand should not take CatalogTabl

[jira] [Created] (SPARK-16660) CreateViewCommand should not take CatalogTable

2016-07-20 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-16660: --- Summary: CreateViewCommand should not take CatalogTable Key: SPARK-16660 URL: https://issues.apache.org/jira/browse/SPARK-16660 Project: Spark Issue Type: Impr

[jira] [Assigned] (SPARK-16639) query fails if having condition contains grouping column

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16639: Assignee: Apache Spark > query fails if having condition contains grouping column > --

[jira] [Assigned] (SPARK-16639) query fails if having condition contains grouping column

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16639: Assignee: (was: Apache Spark) > query fails if having condition contains grouping colu

[jira] [Commented] (SPARK-16639) query fails if having condition contains grouping column

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387176#comment-15387176 ] Apache Spark commented on SPARK-16639: -- User 'viirya' has created a pull request for

[jira] [Commented] (SPARK-16646) LEAST doesn't accept numeric arguments with different data types

2016-07-20 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387172#comment-15387172 ] Cheng Lian commented on SPARK-16646: Thanks for the help! I'm not working on this. >

[jira] [Updated] (SPARK-16646) LEAST doesn't accept numeric arguments with different data types

2016-07-20 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16646: --- Reporter: Cheng Lian (was: liancheng) > LEAST doesn't accept numeric arguments with different data t

[jira] [Updated] (SPARK-16646) LEAST doesn't accept numeric arguments with different data types

2016-07-20 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16646: --- Assignee: Hyukjin Kwon > LEAST doesn't accept numeric arguments with different data types > -

[jira] [Updated] (SPARK-16648) LAST_VALUE(FALSE) OVER () throws IndexOutOfBoundsException

2016-07-20 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16648: --- Reporter: Cheng Lian (was: liancheng) > LAST_VALUE(FALSE) OVER () throws IndexOutOfBoundsException >

[jira] [Updated] (SPARK-16648) LAST_VALUE(FALSE) OVER () throws IndexOutOfBoundsException

2016-07-20 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-16648: Assignee: Cheng Lian > LAST_VALUE(FALSE) OVER () throws IndexOutOfBoundsException > ---

[jira] [Assigned] (SPARK-16648) LAST_VALUE(FALSE) OVER () throws IndexOutOfBoundsException

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16648: Assignee: Apache Spark > LAST_VALUE(FALSE) OVER () throws IndexOutOfBoundsException >

[jira] [Commented] (SPARK-16648) LAST_VALUE(FALSE) OVER () throws IndexOutOfBoundsException

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387162#comment-15387162 ] Apache Spark commented on SPARK-16648: -- User 'liancheng' has created a pull request

[jira] [Assigned] (SPARK-16648) LAST_VALUE(FALSE) OVER () throws IndexOutOfBoundsException

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16648: Assignee: (was: Apache Spark) > LAST_VALUE(FALSE) OVER () throws IndexOutOfBoundsExcep

[jira] [Comment Edited] (SPARK-2666) Always try to cancel running tasks when a stage is marked as zombie

2016-07-20 Thread Lianhui Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387145#comment-15387145 ] Lianhui Wang edited comment on SPARK-2666 at 7/21/16 4:38 AM: --

[jira] [Commented] (SPARK-2666) Always try to cancel running tasks when a stage is marked as zombie

2016-07-20 Thread Lianhui Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387145#comment-15387145 ] Lianhui Wang commented on SPARK-2666: - I think what [~irashid] said is more about non-

[jira] [Commented] (SPARK-16646) LEAST doesn't accept numeric arguments with different data types

2016-07-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387067#comment-15387067 ] Hyukjin Kwon commented on SPARK-16646: -- Maybe I had to ask if you are working on thi

[jira] [Assigned] (SPARK-16646) LEAST doesn't accept numeric arguments with different data types

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16646: Assignee: Apache Spark > LEAST doesn't accept numeric arguments with different data types

[jira] [Commented] (SPARK-16646) LEAST doesn't accept numeric arguments with different data types

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387025#comment-15387025 ] Apache Spark commented on SPARK-16646: -- User 'HyukjinKwon' has created a pull reques

[jira] [Assigned] (SPARK-16646) LEAST doesn't accept numeric arguments with different data types

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16646: Assignee: (was: Apache Spark) > LEAST doesn't accept numeric arguments with different

[jira] [Created] (SPARK-16659) use Maven project to submit spark application via yarn-client

2016-07-20 Thread Jack Jiang (JIRA)
Jack Jiang created SPARK-16659: -- Summary: use Maven project to submit spark application via yarn-client Key: SPARK-16659 URL: https://issues.apache.org/jira/browse/SPARK-16659 Project: Spark Is

[jira] [Commented] (SPARK-16643) When doing Shuffle, report "java.io.FileNotFoundException"

2016-07-20 Thread Deng Changchun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387012#comment-15387012 ] Deng Changchun commented on SPARK-16643: Thank you for response. I think this pr

[jira] [Updated] (SPARK-16644) constraints propagation may fail the query

2016-07-20 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-16644: - Fix Version/s: (was: 2.0.0) 2.1.0 2.0.1 > constraints propagati

[jira] [Updated] (SPARK-16644) constraints propagation may fail the query

2016-07-20 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-16644: - Assignee: Wenchen Fan > constraints propagation may fail the query >

[jira] [Resolved] (SPARK-16644) constraints propagation may fail the query

2016-07-20 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-16644. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 14281 [https://github.com/

[jira] [Resolved] (SPARK-16272) Allow configs to reference other configs, env and system properties

2016-07-20 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-16272. Resolution: Fixed Assignee: Marcelo Vanzin Fix Version/s: 2.1.0 > Allow con

[jira] [Commented] (SPARK-14131) Add a workaround for HADOOP-10622 to fix DataFrameReaderWriterSuite

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386907#comment-15386907 ] Apache Spark commented on SPARK-14131: -- User 'tdas' has created a pull request for t

[jira] [Resolved] (SPARK-16344) Array of struct with a single field name "element" can't be decoded from Parquet files written by Spark 1.6+

2016-07-20 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-16344. -- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14014 [https://github.com/

[jira] [Assigned] (SPARK-16658) Add EdgePartition.withVertexAttributes

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16658: Assignee: (was: Apache Spark) > Add EdgePartition.withVertexAttributes > -

[jira] [Commented] (SPARK-16658) Add EdgePartition.withVertexAttributes

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386796#comment-15386796 ] Apache Spark commented on SPARK-16658: -- User 'benmccann' has created a pull request

[jira] [Assigned] (SPARK-16658) Add EdgePartition.withVertexAttributes

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16658: Assignee: Apache Spark > Add EdgePartition.withVertexAttributes >

[jira] [Created] (SPARK-16658) Add EdgePartition.withVertexAttributes

2016-07-20 Thread Ben McCann (JIRA)
Ben McCann created SPARK-16658: -- Summary: Add EdgePartition.withVertexAttributes Key: SPARK-16658 URL: https://issues.apache.org/jira/browse/SPARK-16658 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-16280) Implement histogram_numeric SQL function

2016-07-20 Thread MIN-FU YANG (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386787#comment-15386787 ] MIN-FU YANG commented on SPARK-16280: - I implemented histogram_numeric according to t

[jira] [Commented] (SPARK-16657) Replace children by innerChildren in InsertIntoHadoopFsRelationCommand and CreateHiveTableAsSelectCommand

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386765#comment-15386765 ] Apache Spark commented on SPARK-16657: -- User 'gatorsmile' has created a pull request

[jira] [Assigned] (SPARK-16657) Replace children by innerChildren in InsertIntoHadoopFsRelationCommand and CreateHiveTableAsSelectCommand

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16657: Assignee: (was: Apache Spark) > Replace children by innerChildren in InsertIntoHadoopF

[jira] [Assigned] (SPARK-16657) Replace children by innerChildren in InsertIntoHadoopFsRelationCommand and CreateHiveTableAsSelectCommand

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16657: Assignee: Apache Spark > Replace children by innerChildren in InsertIntoHadoopFsRelationCo

[jira] [Created] (SPARK-16657) Replace children by innerChildren in InsertIntoHadoopFsRelationCommand and CreateHiveTableAsSelectCommand

2016-07-20 Thread Xiao Li (JIRA)
Xiao Li created SPARK-16657: --- Summary: Replace children by innerChildren in InsertIntoHadoopFsRelationCommand and CreateHiveTableAsSelectCommand Key: SPARK-16657 URL: https://issues.apache.org/jira/browse/SPARK-16657

[jira] [Commented] (SPARK-16644) constraints propagation may fail the query

2016-07-20 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386764#comment-15386764 ] Yin Huai commented on SPARK-16644: -- oh, I think this problem only happens if an aggregat

[jira] [Commented] (SPARK-12675) Executor dies because of ClassCastException and causes timeout

2016-07-20 Thread Emma Tang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386759#comment-15386759 ] Emma Tang commented on SPARK-12675: --- This issue is happening in 1.6.2. I'm on a 32 node

[jira] [Commented] (SPARK-16656) CreateTableAsSelectSuite is flaky

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386750#comment-15386750 ] Apache Spark commented on SPARK-16656: -- User 'yhuai' has created a pull request for

[jira] [Assigned] (SPARK-16656) CreateTableAsSelectSuite is flaky

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16656: Assignee: Yin Huai (was: Apache Spark) > CreateTableAsSelectSuite is flaky >

[jira] [Assigned] (SPARK-16656) CreateTableAsSelectSuite is flaky

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16656: Assignee: Apache Spark (was: Yin Huai) > CreateTableAsSelectSuite is flaky >

[jira] [Created] (SPARK-16656) CreateTableAsSelectSuite is flaky

2016-07-20 Thread Yin Huai (JIRA)
Yin Huai created SPARK-16656: Summary: CreateTableAsSelectSuite is flaky Key: SPARK-16656 URL: https://issues.apache.org/jira/browse/SPARK-16656 Project: Spark Issue Type: Bug Component

[jira] [Updated] (SPARK-16320) Spark 2.0 slower than 1.6 when querying nested columns

2016-07-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-16320: --- Description: I did some test on parquet file with many nested columns (about 30G in 400 parti

[jira] [Updated] (SPARK-16320) Spark 2.0 slower than 1.6 when querying nested columns

2016-07-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-16320: --- Description: I did some test on parquet file with many nested columns (about 30G in 400 parti

[jira] [Updated] (SPARK-16320) Spark 2.0 slower than 1.6 when querying nested columns

2016-07-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-16320: --- Description: I did some test on parquet file with many nested columns (about 30G in 400 parti

[jira] [Commented] (SPARK-16320) Spark 2.0 slower than 1.6 when querying nested columns

2016-07-20 Thread Michael Allman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386586#comment-15386586 ] Michael Allman commented on SPARK-16320: Okay, so the metastore is not a factor.

[jira] [Comment Edited] (SPARK-13301) PySpark Dataframe return wrong results with custom UDF

2016-07-20 Thread Sam Fishman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386564#comment-15386564 ] Sam Fishman edited comment on SPARK-13301 at 7/20/16 8:27 PM: -

[jira] [Commented] (SPARK-2666) Always try to cancel running tasks when a stage is marked as zombie

2016-07-20 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386561#comment-15386561 ] Thomas Graves commented on SPARK-2666: -- thanks for the explanation. I guess we would

[jira] [Commented] (SPARK-13301) PySpark Dataframe return wrong results with custom UDF

2016-07-20 Thread Sam Fishman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386564#comment-15386564 ] Sam Fishman commented on SPARK-13301: - I am having the same issue when applying a udf

[jira] [Commented] (SPARK-16320) Spark 2.0 slower than 1.6 when querying nested columns

2016-07-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386552#comment-15386552 ] Maciej Bryński commented on SPARK-16320: [~michael] Could you check SPARK-16321 ?

[jira] [Commented] (SPARK-2666) Always try to cancel running tasks when a stage is marked as zombie

2016-07-20 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386524#comment-15386524 ] Imran Rashid commented on SPARK-2666: - [~tgraves] [~lianhuiwang]. When there is a fet

[jira] [Updated] (SPARK-16507) Add CRAN checks to SparkR

2016-07-20 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-16507: - Target Version/s: 2.0.0, 2.1.0 (was: 2.1.0) > Add CRAN checks to SparkR > -

[jira] [Commented] (SPARK-16632) Vectorized parquet reader fails to read certain fields from Hive tables

2016-07-20 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386409#comment-15386409 ] Marcelo Vanzin commented on SPARK-16632: Yes, that's the right stack trace. It's

[jira] [Commented] (SPARK-16320) Spark 2.0 slower than 1.6 when querying nested columns

2016-07-20 Thread Michael Allman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386395#comment-15386395 ] Michael Allman commented on SPARK-16320: The code path for reading data from parq

[jira] [Commented] (SPARK-14887) Generated SpecificUnsafeProjection Exceeds JVM Code Size Limits

2016-07-20 Thread Hayri Volkan Agun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386393#comment-15386393 ] Hayri Volkan Agun commented on SPARK-14887: --- Same issue in 1.6.2 can repeated w

[jira] [Updated] (SPARK-15425) Disallow cartesian joins by default

2016-07-20 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-15425: - Labels: release_notes releasenotes (was: ) > Disallow cartesian joins by default > -

[jira] [Updated] (SPARK-16644) constraints propagation may fail the query

2016-07-20 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-16644: - Component/s: SQL > constraints propagation may fail the query > -

[jira] [Updated] (SPARK-16644) constraints propagation may fail the query

2016-07-20 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-16644: - Target Version/s: 2.0.1 (was: 2.0.0) > constraints propagation may fail the query >

[jira] [Updated] (SPARK-16633) lag/lead does not return the default value when the offset row does not exist

2016-07-20 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-16633: - Target Version/s: 2.0.1 (was: 2.0.0) > lag/lead does not return the default value when the offset row do

[jira] [Updated] (SPARK-16642) ResolveWindowFrame should not be triggered on UnresolvedFunctions.

2016-07-20 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-16642: - Target Version/s: 2.0.1 (was: 2.0.0) > ResolveWindowFrame should not be triggered on UnresolvedFunctions

[jira] [Updated] (SPARK-16648) LAST_VALUE(FALSE) OVER () throws IndexOutOfBoundsException

2016-07-20 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-16648: - Target Version/s: 2.0.1 (was: 2.0.0) > LAST_VALUE(FALSE) OVER () throws IndexOutOfBoundsException >

[jira] [Updated] (SPARK-16644) constraints propagation may fail the query

2016-07-20 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-16644: - Affects Version/s: 2.0.0 Target Version/s: 2.0.0 > constraints propagation may fail the query >

[jira] [Commented] (SPARK-16651) No exception using DataFrame.withColumnRenamed when existing column doesn't exist

2016-07-20 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386360#comment-15386360 ] Dongjoon Hyun commented on SPARK-16651: --- I made a PR for you and other people, but

[jira] [Commented] (SPARK-16654) UI Should show blacklisted executors & nodes

2016-07-20 Thread Alex Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386338#comment-15386338 ] Alex Bozarth commented on SPARK-16654: -- Perhaps we can change the status column to "

[jira] [Commented] (SPARK-16320) Spark 2.0 slower than 1.6 when querying nested columns

2016-07-20 Thread Michael Allman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386333#comment-15386333 ] Michael Allman commented on SPARK-16320: [~maver1ck] Would it be possible for you

[jira] [Updated] (SPARK-16642) ResolveWindowFrame should not be triggered on UnresolvedFunctions.

2016-07-20 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-16642: - Target Version/s: 2.0.0 > ResolveWindowFrame should not be triggered on UnresolvedFunctions. > --

[jira] [Created] (SPARK-16655) Spark thrift server application is not stopped if its in ACCEPTED stage

2016-07-20 Thread Yesha Vora (JIRA)
Yesha Vora created SPARK-16655: -- Summary: Spark thrift server application is not stopped if its in ACCEPTED stage Key: SPARK-16655 URL: https://issues.apache.org/jira/browse/SPARK-16655 Project: Spark

[jira] [Assigned] (SPARK-16651) No exception using DataFrame.withColumnRenamed when existing column doesn't exist

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16651: Assignee: (was: Apache Spark) > No exception using DataFrame.withColumnRenamed when ex

[jira] [Commented] (SPARK-16651) No exception using DataFrame.withColumnRenamed when existing column doesn't exist

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386317#comment-15386317 ] Apache Spark commented on SPARK-16651: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Assigned] (SPARK-16651) No exception using DataFrame.withColumnRenamed when existing column doesn't exist

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16651: Assignee: Apache Spark > No exception using DataFrame.withColumnRenamed when existing colu

[jira] [Assigned] (SPARK-16650) Improve documentation of spark.task.maxFailures

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16650: Assignee: Apache Spark > Improve documentation of spark.task.maxFailures >

[jira] [Commented] (SPARK-16650) Improve documentation of spark.task.maxFailures

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386311#comment-15386311 ] Apache Spark commented on SPARK-16650: -- User 'tgravescs' has created a pull request

[jira] [Assigned] (SPARK-16650) Improve documentation of spark.task.maxFailures

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16650: Assignee: (was: Apache Spark) > Improve documentation of spark.task.maxFailures

[jira] [Comment Edited] (SPARK-16651) No exception using DataFrame.withColumnRenamed when existing column doesn't exist

2016-07-20 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386295#comment-15386295 ] Dongjoon Hyun edited comment on SPARK-16651 at 7/20/16 5:55 PM: ---

[jira] [Commented] (SPARK-16651) No exception using DataFrame.withColumnRenamed when existing column doesn't exist

2016-07-20 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386295#comment-15386295 ] Dongjoon Hyun commented on SPARK-16651: --- Also, in Spark 2.0 RC, Dataframe is merged

[jira] [Commented] (SPARK-16651) No exception using DataFrame.withColumnRenamed when existing column doesn't exist

2016-07-20 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386286#comment-15386286 ] Dongjoon Hyun commented on SPARK-16651: --- Hi, [~tomwphillips]. It's a documented beh

[jira] [Resolved] (SPARK-16634) GenericArrayData can't be loaded in certain JVMs

2016-07-20 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-16634. Resolution: Fixed Assignee: Marcelo Vanzin Fix Version/s: 2.1.0

[jira] [Commented] (SPARK-16654) UI Should show blacklisted executors & nodes

2016-07-20 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386242#comment-15386242 ] Imran Rashid commented on SPARK-16654: -- cc [~tgraves] > UI Should show blacklisted

[jira] [Created] (SPARK-16654) UI Should show blacklisted executors & nodes

2016-07-20 Thread Imran Rashid (JIRA)
Imran Rashid created SPARK-16654: Summary: UI Should show blacklisted executors & nodes Key: SPARK-16654 URL: https://issues.apache.org/jira/browse/SPARK-16654 Project: Spark Issue Type: Impr

[jira] [Updated] (SPARK-15951) Change Executors Page to use datatables to support sorting columns and searching

2016-07-20 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-15951: -- Assignee: Kishor Patil > Change Executors Page to use datatables to support sorting columns and

[jira] [Resolved] (SPARK-15951) Change Executors Page to use datatables to support sorting columns and searching

2016-07-20 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-15951. --- Resolution: Fixed Fix Version/s: 2.1.0 > Change Executors Page to use datatables to su

[jira] [Commented] (SPARK-2666) Always try to cancel running tasks when a stage is marked as zombie

2016-07-20 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386235#comment-15386235 ] Thomas Graves commented on SPARK-2666: -- I think eventually adding prestart (MapReduce

[jira] [Commented] (SPARK-15918) unionAll returns wrong result when two dataframes has schema in different order

2016-07-20 Thread Wade Salazar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386225#comment-15386225 ] Wade Salazar commented on SPARK-15918: -- Is the stance on this issue that since other

[jira] [Updated] (SPARK-15968) HiveMetastoreCatalog does not correctly validate partitioned metastore relation when searching the internal table cache

2016-07-20 Thread Michael Allman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Allman updated SPARK-15968: --- Fix Version/s: 2.0.0 > HiveMetastoreCatalog does not correctly validate partitioned metastore

[jira] [Commented] (SPARK-13767) py4j.protocol.Py4JNetworkError: An error occurred while trying to connect to the Java server

2016-07-20 Thread Rahul Bhatia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386192#comment-15386192 ] Rahul Bhatia commented on SPARK-13767: -- I'm seeing the error that Venkata showed as

[jira] [Resolved] (SPARK-16613) RDD.pipe returns values for empty partitions

2016-07-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-16613. - Resolution: Fixed Assignee: Sean Owen Fix Version/s: 2.1.0 2.0.

[jira] [Updated] (SPARK-15968) HiveMetastoreCatalog does not correctly validate partitioned metastore relation when searching the internal table cache

2016-07-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-15968: Fix Version/s: (was: 2.1.0) > HiveMetastoreCatalog does not correctly validate partitioned meta

[jira] [Assigned] (SPARK-16653) Make convergence tolerance param in ANN default value consistent with other algorithm using LBFGS

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16653: Assignee: Apache Spark > Make convergence tolerance param in ANN default value consistent

[jira] [Commented] (SPARK-16653) Make convergence tolerance param in ANN default value consistent with other algorithm using LBFGS

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386028#comment-15386028 ] Apache Spark commented on SPARK-16653: -- User 'WeichenXu123' has created a pull reque

[jira] [Assigned] (SPARK-16653) Make convergence tolerance param in ANN default value consistent with other algorithm using LBFGS

2016-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16653: Assignee: (was: Apache Spark) > Make convergence tolerance param in ANN default value

[jira] [Updated] (SPARK-16653) Make convergence tolerance param in ANN default value consistent with other algorithm using LBFGS

2016-07-20 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-16653: --- Component/s: Optimizer ML > Make convergence tolerance param in ANN default value co

[jira] [Created] (SPARK-16653) Make convergence tolerance param in ANN default value consistent with other algorithm using LBFGS

2016-07-20 Thread Weichen Xu (JIRA)
Weichen Xu created SPARK-16653: -- Summary: Make convergence tolerance param in ANN default value consistent with other algorithm using LBFGS Key: SPARK-16653 URL: https://issues.apache.org/jira/browse/SPARK-16653

[jira] [Commented] (SPARK-3210) Flume Polling Receiver must be more tolerant to connection failures.

2016-07-20 Thread Ian Brooks (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386002#comment-15386002 ] Ian Brooks commented on SPARK-3210: --- Hi, I have also noticed an issue with resiliency f

  1   2   >