[jira] [Commented] (SPARK-17090) Make tree aggregation level in linear/logistic regression configurable

2016-08-17 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425936#comment-15425936 ] DB Tsai commented on SPARK-17090: - We could make it default to 0 which automatically figure out the best

[jira] [Created] (SPARK-17123) Performing set operations that combine string and date / timestamp columns may result in generated projection code which doesn't compile

2016-08-17 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-17123: -- Summary: Performing set operations that combine string and date / timestamp columns may result in generated projection code which doesn't compile Key: SPARK-17123 URL:

[jira] [Commented] (SPARK-17122) Failed to drop database when use the database in Spark 2.0

2016-08-17 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425926#comment-15425926 ] Yi Zhou commented on SPARK-17122: - Thanks [~dongjoon] for your quick response. But the problem is this

[jira] [Commented] (SPARK-17110) Pyspark with locality ANY throw java.io.StreamCorruptedException

2016-08-17 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425925#comment-15425925 ] Miao Wang commented on SPARK-17110: --- The network layer of Spark must have some change for dealing with

[jira] [Updated] (SPARK-16995) TreeNodeException when flat mapping RelationalGroupedDataset created from DataFrame containing a column created with lit/expr

2016-08-17 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-16995: Assignee: Liang-Chi Hsieh > TreeNodeException when flat mapping RelationalGroupedDataset created

[jira] [Resolved] (SPARK-16995) TreeNodeException when flat mapping RelationalGroupedDataset created from DataFrame containing a column created with lit/expr

2016-08-17 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-16995. - Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Commented] (SPARK-17090) Make tree aggregation level in linear/logistic regression configurable

2016-08-17 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425913#comment-15425913 ] Yanbo Liang commented on SPARK-17090: - Making aggregation depth configurable is necessary when

[jira] [Commented] (SPARK-17122) Failed to drop database when use the database in Spark 2.0

2016-08-17 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425908#comment-15425908 ] Dongjoon Hyun commented on SPARK-17122: --- Hi, [~jameszhouyi] It's intentionally implemented in

[jira] [Commented] (SPARK-16714) Fail to create a decimal arrays with literals having different inferred precessions and scales

2016-08-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425893#comment-15425893 ] Apache Spark commented on SPARK-16714: -- User 'petermaxlee' has created a pull request for this

[jira] [Created] (SPARK-17122) Failed to drop database when use the database in Spark 2.0

2016-08-17 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-17122: --- Summary: Failed to drop database when use the database in Spark 2.0 Key: SPARK-17122 URL: https://issues.apache.org/jira/browse/SPARK-17122 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-17117) 'SELECT 1 / NULL` throws AnalysisException, while 'SELECT 1 * NULL` works

2016-08-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17117: Assignee: Apache Spark > 'SELECT 1 / NULL` throws AnalysisException, while 'SELECT 1 *

[jira] [Assigned] (SPARK-17117) 'SELECT 1 / NULL` throws AnalysisException, while 'SELECT 1 * NULL` works

2016-08-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17117: Assignee: (was: Apache Spark) > 'SELECT 1 / NULL` throws AnalysisException, while

[jira] [Commented] (SPARK-17117) 'SELECT 1 / NULL` throws AnalysisException, while 'SELECT 1 * NULL` works

2016-08-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425872#comment-15425872 ] Apache Spark commented on SPARK-17117: -- User 'petermaxlee' has created a pull request for this

[jira] [Commented] (SPARK-17070) Zookeeper server refused to accept the client (mesos-master)

2016-08-17 Thread Anh Nguyen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425847#comment-15425847 ] Anh Nguyen commented on SPARK-17070: hi [~c...@koeninger.org] and [~srowen] if you free time. Please,

[jira] [Commented] (SPARK-16581) Making JVM backend calling functions public

2016-08-17 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425813#comment-15425813 ] Shivaram Venkataraman commented on SPARK-16581: --- Thanks [~yinxusen] - I think SPARK-16611

[jira] [Commented] (SPARK-17114) Adding a 'GROUP BY 1' where first column is literal results in wrong answer

2016-08-17 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425808#comment-15425808 ] Wenchen Fan commented on SPARK-17114: - A simple way is to not remove grouping literals if it's the

[jira] [Commented] (SPARK-17121) Support _HOST replacement for principal

2016-08-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425795#comment-15425795 ] Apache Spark commented on SPARK-17121: -- User 'zjffdu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17121) Support _HOST replacement for principal

2016-08-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17121: Assignee: (was: Apache Spark) > Support _HOST replacement for principal >

[jira] [Assigned] (SPARK-17121) Support _HOST replacement for principal

2016-08-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17121: Assignee: Apache Spark > Support _HOST replacement for principal >

[jira] [Commented] (SPARK-16581) Making JVM backend calling functions public

2016-08-17 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425792#comment-15425792 ] Xusen Yin commented on SPARK-16581: --- I'll find related JIRAs and link them if possible. > Making JVM

[jira] [Created] (SPARK-17121) Support _HOST replacement for principal

2016-08-17 Thread Jeff Zhang (JIRA)
Jeff Zhang created SPARK-17121: -- Summary: Support _HOST replacement for principal Key: SPARK-17121 URL: https://issues.apache.org/jira/browse/SPARK-17121 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-16581) Making JVM backend calling functions public

2016-08-17 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425789#comment-15425789 ] Sun Rui commented on SPARK-16581: - sure. go ahead. But note that the discussion is continuing. There are

[jira] [Commented] (SPARK-17113) Job failure due to Executor OOM in offheap mode

2016-08-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425779#comment-15425779 ] Apache Spark commented on SPARK-17113: -- User 'sitalkedia' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17113) Job failure due to Executor OOM in offheap mode

2016-08-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17113: Assignee: (was: Apache Spark) > Job failure due to Executor OOM in offheap mode >

[jira] [Assigned] (SPARK-17113) Job failure due to Executor OOM in offheap mode

2016-08-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17113: Assignee: Apache Spark > Job failure due to Executor OOM in offheap mode >

[jira] [Commented] (SPARK-16581) Making JVM backend calling functions public

2016-08-17 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425775#comment-15425775 ] Xusen Yin commented on SPARK-16581: --- [~shivaram] [~sunrui] Still work on it? I can help work on this if

[jira] [Commented] (SPARK-17110) Pyspark with locality ANY throw java.io.StreamCorruptedException

2016-08-17 Thread Tomer Kaftan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425752#comment-15425752 ] Tomer Kaftan commented on SPARK-17110: -- It is with the default configuration settings, and this

[jira] [Updated] (SPARK-17113) Job failure due to Executor OOM in offheap mode

2016-08-17 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-17113: Summary: Job failure due to Executor OOM in offheap mode (was: Job failure due to Executor OOM)

[jira] [Commented] (SPARK-16785) dapply doesn't return array or raw columns

2016-08-17 Thread Clark Fitzgerald (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425679#comment-15425679 ] Clark Fitzgerald commented on SPARK-16785: -- Starting on this now. > dapply doesn't return array

[jira] [Created] (SPARK-17120) Analyzer incorrectly optimizes plan to empty LocalRelation

2016-08-17 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-17120: -- Summary: Analyzer incorrectly optimizes plan to empty LocalRelation Key: SPARK-17120 URL: https://issues.apache.org/jira/browse/SPARK-17120 Project: Spark Issue

[jira] [Comment Edited] (SPARK-17110) Pyspark with locality ANY throw java.io.StreamCorruptedException

2016-08-17 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425595#comment-15425595 ] Miao Wang edited comment on SPARK-17110 at 8/18/16 12:30 AM: - It seems that

[jira] [Created] (SPARK-17119) Add configuration property to allow the history server to delete .inprogress files

2016-08-17 Thread Bjorn Jonsson (JIRA)
Bjorn Jonsson created SPARK-17119: - Summary: Add configuration property to allow the history server to delete .inprogress files Key: SPARK-17119 URL: https://issues.apache.org/jira/browse/SPARK-17119

[jira] [Commented] (SPARK-10868) monotonicallyIncreasingId() supports offset for indexing

2016-08-17 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425607#comment-15425607 ] Ted Yu commented on SPARK-10868: bq. this makes sense What was the reasoning behind the initial

[jira] [Comment Edited] (SPARK-10868) monotonicallyIncreasingId() supports offset for indexing

2016-08-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425598#comment-15425598 ] Reynold Xin edited comment on SPARK-10868 at 8/17/16 11:32 PM: --- I'm going

[jira] [Resolved] (SPARK-17038) StreamingSource reports metrics for lastCompletedBatch instead of lastReceivedBatch

2016-08-17 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17038. -- Resolution: Fixed Assignee: Xin Ren Fix Version/s: 2.1.0

[jira] [Comment Edited] (SPARK-10868) monotonicallyIncreasingId() supports offset for indexing

2016-08-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425598#comment-15425598 ] Reynold Xin edited comment on SPARK-10868 at 8/17/16 11:33 PM: --- I'm going

[jira] [Closed] (SPARK-10868) monotonicallyIncreasingId() supports offset for indexing

2016-08-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-10868. --- Resolution: Won't Fix I'm going to close this. I don't think adding an offset will actually satisfy

[jira] [Commented] (SPARK-17110) Pyspark with locality ANY throw java.io.StreamCorruptedException

2016-08-17 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425595#comment-15425595 ] Miao Wang commented on SPARK-17110: --- It seems that it is a network I/O timeout, because there is one

[jira] [Commented] (SPARK-17118) Make examples Python3 compatible

2016-08-17 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425572#comment-15425572 ] Miao Wang commented on SPARK-17118: --- Looks straightforward. I can try to create a PR for this one. >

[jira] [Comment Edited] (SPARK-10747) add support for window specification to include how NULLS are ordered

2016-08-17 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425521#comment-15425521 ] Herman van Hovell edited comment on SPARK-10747 at 8/17/16 10:57 PM: -

[jira] [Commented] (SPARK-10747) add support for window specification to include how NULLS are ordered

2016-08-17 Thread Xin Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425547#comment-15425547 ] Xin Wu commented on SPARK-10747: [~hvanhovell] Yes. Since we have native parser now, we can do this

[jira] [Commented] (SPARK-17116) Allow params to be a {string, value} dict at fit time

2016-08-17 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425522#comment-15425522 ] Manoj Kumar commented on SPARK-17116: - Haha, not really. I just found it odd that setParams accepts

[jira] [Created] (SPARK-17118) Make examples Python3 compatible

2016-08-17 Thread Manoj Kumar (JIRA)
Manoj Kumar created SPARK-17118: --- Summary: Make examples Python3 compatible Key: SPARK-17118 URL: https://issues.apache.org/jira/browse/SPARK-17118 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-10747) add support for window specification to include how NULLS are ordered

2016-08-17 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425521#comment-15425521 ] Herman van Hovell commented on SPARK-10747: --- [~xwu0226] Would be interested to open a PR for

[jira] [Updated] (SPARK-17116) Allow params to be a {string, value} dict at fit time

2016-08-17 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Kumar updated SPARK-17116: Description: Currently, it is possible to override the default params set at constructor time by

[jira] [Commented] (SPARK-17116) Allow params to be a {string, value} dict at fit time

2016-08-17 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425507#comment-15425507 ] holdenk commented on SPARK-17116: - So it seems like doing the second one is more likely to result in

[jira] [Created] (SPARK-17117) 'SELECT 1 / NULL` throws AnalysisException, while 'SELECT 1 * NULL` works

2016-08-17 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-17117: -- Summary: 'SELECT 1 / NULL` throws AnalysisException, while 'SELECT 1 * NULL` works Key: SPARK-17117 URL: https://issues.apache.org/jira/browse/SPARK-17117 Project: Spark

[jira] [Comment Edited] (SPARK-17116) Allow params to be a {string, value} dict at fit time

2016-08-17 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425494#comment-15425494 ] Manoj Kumar edited comment on SPARK-17116 at 8/17/16 10:17 PM: --- [~josephkb]

[jira] [Commented] (SPARK-16320) Spark 2.0 slower than 1.6 when querying nested columns

2016-08-17 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425497#comment-15425497 ] Yin Huai commented on SPARK-16320: -- [~maver1ck] Seems we can close this jira? > Spark 2.0 slower than

[jira] [Commented] (SPARK-17116) Allow params to be a {string, value} dict at fit time

2016-08-17 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425494#comment-15425494 ] Manoj Kumar commented on SPARK-17116: - [~josephkb] [~mlnick] This is not super important, but I do

[jira] [Updated] (SPARK-17116) Allow params to be a {string, value} dict at fit time

2016-08-17 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Kumar updated SPARK-17116: Summary: Allow params to be a {string, value} dict at fit time (was: Allow params to be a

[jira] [Created] (SPARK-17116) Allow params to be a {string, value} dict

2016-08-17 Thread Manoj Kumar (JIRA)
Manoj Kumar created SPARK-17116: --- Summary: Allow params to be a {string, value} dict Key: SPARK-17116 URL: https://issues.apache.org/jira/browse/SPARK-17116 Project: Spark Issue Type:

[jira] [Commented] (SPARK-10141) Number of tasks on executors still become negative after failures

2016-08-17 Thread Jun Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425486#comment-15425486 ] Jun Zhou commented on SPARK-10141: -- Running Spark 1.6.1.5 Seeing the same issue. > Number of tasks on

[jira] [Commented] (SPARK-17114) Adding a 'GROUP BY 1' where first column is literal results in wrong answer

2016-08-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425480#comment-15425480 ] Reynold Xin commented on SPARK-17114: - The problem is that RemoveLiteralFromGroupExpressions

[jira] [Commented] (SPARK-17115) Improve the performance of UnsafeProjection for wide table

2016-08-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425478#comment-15425478 ] Apache Spark commented on SPARK-17115: -- User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17115) Improve the performance of UnsafeProjection for wide table

2016-08-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17115: Assignee: Davies Liu (was: Apache Spark) > Improve the performance of UnsafeProjection

[jira] [Assigned] (SPARK-17115) Improve the performance of UnsafeProjection for wide table

2016-08-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17115: Assignee: Apache Spark (was: Davies Liu) > Improve the performance of UnsafeProjection

[jira] [Issue Comment Deleted] (SPARK-17114) Adding a 'GROUP BY 1' where first column is literal results in wrong answer

2016-08-17 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17114: --- Comment: was deleted (was: Even if we decide that Spark's current interpretation of this query is

[jira] [Commented] (SPARK-17114) Adding a 'GROUP BY 1' where first column is literal results in wrong answer

2016-08-17 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425445#comment-15425445 ] Josh Rosen commented on SPARK-17114: Even if we decide that Spark's current interpretation of this

[jira] [Resolved] (SPARK-17111) Spark2 History server got "Failed to load application attempt"

2016-08-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17111. --- Resolution: Duplicate > Spark2 History server got "Failed to load application attempt" >

[jira] [Resolved] (SPARK-17103) Can not define class variable in repl

2016-08-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17103. --- Resolution: Duplicate Sounds like a different manifestation of the same REPL import issue > Can not

[jira] [Resolved] (SPARK-17096) Fix StreamingQueryListener to return message and stacktrace of actual exception

2016-08-17 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-17096. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by

[jira] [Updated] (SPARK-17101) Provide consistent format identifiers for TextFileFormat and ParquetFileFormat

2016-08-17 Thread Jacek Laskowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Laskowski updated SPARK-17101: Description: Define the format identifier that is used in {{Optimized Logical Plan}} in

[jira] [Updated] (SPARK-17101) Provide consistent format identifiers for TextFileFormat and ParquetFileFormat

2016-08-17 Thread Jacek Laskowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Laskowski updated SPARK-17101: Summary: Provide consistent format identifiers for TextFileFormat and ParquetFileFormat

[jira] [Created] (SPARK-17115) Improve the performance of UnsafeProjection for wide table

2016-08-17 Thread Davies Liu (JIRA)
Davies Liu created SPARK-17115: -- Summary: Improve the performance of UnsafeProjection for wide table Key: SPARK-17115 URL: https://issues.apache.org/jira/browse/SPARK-17115 Project: Spark Issue

[jira] [Created] (SPARK-17114) Adding a 'GROUP BY 1' where first column is literal results in wrong answer

2016-08-17 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-17114: -- Summary: Adding a 'GROUP BY 1' where first column is literal results in wrong answer Key: SPARK-17114 URL: https://issues.apache.org/jira/browse/SPARK-17114 Project:

[jira] [Assigned] (SPARK-16407) Allow users to supply custom StreamSinkProviders

2016-08-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16407: Assignee: (was: Apache Spark) > Allow users to supply custom StreamSinkProviders >

[jira] [Assigned] (SPARK-16407) Allow users to supply custom StreamSinkProviders

2016-08-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16407: Assignee: Apache Spark > Allow users to supply custom StreamSinkProviders >

[jira] [Commented] (SPARK-16407) Allow users to supply custom StreamSinkProviders

2016-08-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425243#comment-15425243 ] Apache Spark commented on SPARK-16407: -- User 'holdenk' has created a pull request for this issue:

[jira] [Commented] (SPARK-17113) Job failure due to Executor OOM

2016-08-17 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425216#comment-15425216 ] Sital Kedia commented on SPARK-17113: - cc - [~davies] > Job failure due to Executor OOM >

[jira] [Created] (SPARK-17113) Job failure due to Executor OOM

2016-08-17 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-17113: --- Summary: Job failure due to Executor OOM Key: SPARK-17113 URL: https://issues.apache.org/jira/browse/SPARK-17113 Project: Spark Issue Type: Bug Affects

[jira] [Resolved] (SPARK-16736) remove redundant FileSystem status checks calls from Spark codebase

2016-08-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-16736. Resolution: Fixed Assignee: Steve Loughran Fix Version/s: 2.1.0 > remove

[jira] [Created] (SPARK-17112) "select if(true, null, null)" via JDBC triggers IllegalArgumentException in Thriftserver

2016-08-17 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-17112: -- Summary: "select if(true, null, null)" via JDBC triggers IllegalArgumentException in Thriftserver Key: SPARK-17112 URL: https://issues.apache.org/jira/browse/SPARK-17112

[jira] [Updated] (SPARK-17111) Spark2 History server got "Failed to load application attempt"

2016-08-17 Thread Yesha Vora (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated SPARK-17111: --- Description: Steps followed: * Run sparkPi application in yarn-client mode * SparkPi app

[jira] [Resolved] (SPARK-16446) Gaussian Mixture Model wrapper in SparkR

2016-08-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-16446. --- Resolution: Fixed Fix Version/s: 2.1.0 > Gaussian Mixture Model wrapper in SparkR >

[jira] [Created] (SPARK-17111) Spark2 History server got "Failed to load application attempt"

2016-08-17 Thread Yesha Vora (JIRA)
Yesha Vora created SPARK-17111: -- Summary: Spark2 History server got "Failed to load application attempt" Key: SPARK-17111 URL: https://issues.apache.org/jira/browse/SPARK-17111 Project: Spark

[jira] [Assigned] (SPARK-16980) Load only catalog table partition metadata required to answer a query

2016-08-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16980: Assignee: Apache Spark > Load only catalog table partition metadata required to answer a

[jira] [Commented] (SPARK-16980) Load only catalog table partition metadata required to answer a query

2016-08-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425089#comment-15425089 ] Apache Spark commented on SPARK-16980: -- User 'mallman' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16980) Load only catalog table partition metadata required to answer a query

2016-08-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16980: Assignee: (was: Apache Spark) > Load only catalog table partition metadata required

[jira] [Resolved] (SPARK-16930) ApplicationMaster's code that waits for SparkContext is race-prone

2016-08-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-16930. Resolution: Fixed Assignee: Marcelo Vanzin Fix Version/s: 2.1.0 >

[jira] [Commented] (SPARK-17109) When we serialize UserDefinedGenerator to json, scala reflection throws an error

2016-08-17 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425082#comment-15425082 ] Iulian Dragos commented on SPARK-17109: --- So the problem is somewhere along the caller list :) There

[jira] [Issue Comment Deleted] (SPARK-14146) Imported implicits can't be found in Spark REPL in some cases

2016-08-17 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-14146: - Comment: was deleted (was: The upstream ticket) > Imported implicits can't be found in Spark

[jira] [Commented] (SPARK-17109) When we serialize UserDefinedGenerator to json, scala reflection throws an error

2016-08-17 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425073#comment-15425073 ] Yin Huai commented on SPARK-17109: -- For Scala 2.10, {{tpe}} is {{Tuple3}}. For Scala 2.11, it is {{(T1,

[jira] [Commented] (SPARK-17103) Can not define class variable in repl

2016-08-17 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425070#comment-15425070 ] Shixiong Zhu commented on SPARK-17103: -- Right now a workaround is just importing classes in the

[jira] [Commented] (SPARK-17103) Can not define class variable in repl

2016-08-17 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425041#comment-15425041 ] Shixiong Zhu commented on SPARK-17103: -- It's similar to https://issues.scala-lang.org/browse/SI-9799

[jira] [Assigned] (SPARK-17098) "SELECT COUNT(NULL) OVER ()" throws UnsupportedOperationException during analysis

2016-08-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17098: Assignee: (was: Apache Spark) > "SELECT COUNT(NULL) OVER ()" throws

[jira] [Commented] (SPARK-17098) "SELECT COUNT(NULL) OVER ()" throws UnsupportedOperationException during analysis

2016-08-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425001#comment-15425001 ] Apache Spark commented on SPARK-17098: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-17098) "SELECT COUNT(NULL) OVER ()" throws UnsupportedOperationException during analysis

2016-08-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17098: Assignee: Apache Spark > "SELECT COUNT(NULL) OVER ()" throws

[jira] [Commented] (SPARK-17109) When we serialize UserDefinedGenerator to json, scala reflection throws an error

2016-08-17 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15424979#comment-15424979 ] Iulian Dragos commented on SPARK-17109: --- No, it seems strange. I suggest you print out the `tpe`

[jira] [Comment Edited] (SPARK-17103) Can not define class variable in repl

2016-08-17 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15424228#comment-15424228 ] Jakob Odersky edited comment on SPARK-17103 at 8/17/16 5:28 PM: That's

[jira] [Commented] (SPARK-17098) "SELECT COUNT(NULL) OVER ()" throws UnsupportedOperationException during analysis

2016-08-17 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15424955#comment-15424955 ] Dongjoon Hyun commented on SPARK-17098: --- Actually, `SELECT COUNT(1 + NULL) OVER ()` also should

[jira] [Updated] (SPARK-17110) Pyspark with locality ANY throw java.io.StreamCorruptedException

2016-08-17 Thread Tomer Kaftan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomer Kaftan updated SPARK-17110: - Environment: Cluster of 2 AWS r3.xlarge nodes launched via ec2 scripts, Spark 2.0.0, hadoop:

[jira] [Updated] (SPARK-17110) Pyspark with locality ANY throw java.io.StreamCorruptedException

2016-08-17 Thread Tomer Kaftan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomer Kaftan updated SPARK-17110: - Description: In Pyspark 2.0.0, any task that accesses cached data non-locally throws a

[jira] [Updated] (SPARK-17110) Pyspark with locality ANY throw java.io.StreamCorruptedException

2016-08-17 Thread Tomer Kaftan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomer Kaftan updated SPARK-17110: - Description: In Pyspark 2.0.0, any task that accesses cached data non-locally throws a

[jira] [Updated] (SPARK-17110) Pyspark with locality ANY throw java.io.StreamCorruptedException

2016-08-17 Thread Tomer Kaftan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomer Kaftan updated SPARK-17110: - Description: In Pyspark 2.0.0, any task that accesses cached data non-locally throws a

[jira] [Updated] (SPARK-17110) Pyspark with locality ANY throw java.io.StreamCorruptedException

2016-08-17 Thread Tomer Kaftan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomer Kaftan updated SPARK-17110: - Description: In Pyspark 2.0.0, any task that accesses cached data non-locally throws a

[jira] [Updated] (SPARK-17110) Pyspark with locality ANY throw java.io.StreamCorruptedException

2016-08-17 Thread Tomer Kaftan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomer Kaftan updated SPARK-17110: - Description: In Pyspark 2.0.0, any task that accesses cached data non-locally throws a

[jira] [Commented] (SPARK-17109) When we serialize UserDefinedGenerator to json, scala reflection throws an error

2016-08-17 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15424907#comment-15424907 ] Yin Huai commented on SPARK-17109: -- Yea, seems scala 2.11 is fine (I also tested this test in the branch

[jira] [Commented] (SPARK-17098) "SELECT COUNT(NULL) OVER ()" throws UnsupportedOperationException during analysis

2016-08-17 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15424906#comment-15424906 ] Dongjoon Hyun commented on SPARK-17098: --- Hi, [~joshrosen]. According to your error message `cast(0

[jira] [Created] (SPARK-17110) Pyspark with locality ANY throw java.io.StreamCorruptedException

2016-08-17 Thread Tomer Kaftan (JIRA)
Tomer Kaftan created SPARK-17110: Summary: Pyspark with locality ANY throw java.io.StreamCorruptedException Key: SPARK-17110 URL: https://issues.apache.org/jira/browse/SPARK-17110 Project: Spark

  1   2   >