[jira] [Created] (SPARK-17286) The fetched data of Shuffle read stored in the ManagedBuffer, so its underlying data stored in off-memory or file?

2016-08-28 Thread song fengfei (JIRA)
song fengfei created SPARK-17286: Summary: The fetched data of Shuffle read stored in the ManagedBuffer, so its underlying data stored in off-memory or file? Key: SPARK-17286 URL:

[jira] [Commented] (SPARK-17198) ORC fixed char literal filter does not work

2016-08-28 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15444781#comment-15444781 ] Dongjoon Hyun commented on SPARK-17198: --- Yes. Spark 2.0 improves SQL features greatly. I think that

[jira] [Commented] (SPARK-17198) ORC fixed char literal filter does not work

2016-08-28 Thread tuming (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15444755#comment-15444755 ] tuming commented on SPARK-17198: yes, it works fine on spark2.0. I am using spark1.5.1. I can get the

[jira] [Commented] (SPARK-17261) Using HiveContext after re-creating SparkContext in Spark 2.0 throws "Java.lang.illegalStateException: Cannot call methods on a stopped sparkContext"

2016-08-28 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15444694#comment-15444694 ] Jeff Zhang commented on SPARK-17261: [~dongjoon] spark-shell works well for me. It seems your case is

[jira] [Assigned] (SPARK-17261) Using HiveContext after re-creating SparkContext in Spark 2.0 throws "Java.lang.illegalStateException: Cannot call methods on a stopped sparkContext"

2016-08-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17261: Assignee: Apache Spark > Using HiveContext after re-creating SparkContext in Spark 2.0

[jira] [Assigned] (SPARK-17261) Using HiveContext after re-creating SparkContext in Spark 2.0 throws "Java.lang.illegalStateException: Cannot call methods on a stopped sparkContext"

2016-08-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17261: Assignee: (was: Apache Spark) > Using HiveContext after re-creating SparkContext in

[jira] [Commented] (SPARK-17261) Using HiveContext after re-creating SparkContext in Spark 2.0 throws "Java.lang.illegalStateException: Cannot call methods on a stopped sparkContext"

2016-08-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15444690#comment-15444690 ] Apache Spark commented on SPARK-17261: -- User 'zjffdu' has created a pull request for this issue:

[jira] [Commented] (SPARK-17061) Incorrect results returned following a join of two datasets and a map step where total number of columns >100

2016-08-28 Thread Liwei Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15444614#comment-15444614 ] Liwei Lin commented on SPARK-17061: --- Oh cool! Thank you for the well-formed reproducer! > Incorrect

[jira] [Commented] (SPARK-13525) SparkR: java.net.SocketTimeoutException: Accept timed out when running any dataframe function

2016-08-28 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15444579#comment-15444579 ] Sun Rui commented on SPARK-13525: - Could you help to do some debugging by modifying R code? The steps

[jira] [Created] (SPARK-17285) ZeroOutPaddingBytes Causing Fatal JVM Error

2016-08-28 Thread Dean Chen (JIRA)
Dean Chen created SPARK-17285: - Summary: ZeroOutPaddingBytes Causing Fatal JVM Error Key: SPARK-17285 URL: https://issues.apache.org/jira/browse/SPARK-17285 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-17241) SparkR spark.glm should have configurable regularization parameter

2016-08-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15444279#comment-15444279 ] Apache Spark commented on SPARK-17241: -- User 'keypointt' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17241) SparkR spark.glm should have configurable regularization parameter

2016-08-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17241: Assignee: (was: Apache Spark) > SparkR spark.glm should have configurable

[jira] [Assigned] (SPARK-17241) SparkR spark.glm should have configurable regularization parameter

2016-08-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17241: Assignee: Apache Spark > SparkR spark.glm should have configurable regularization

[jira] [Assigned] (SPARK-17284) Remove statistics-related table properties from SHOW CREATE TABLE

2016-08-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17284: Assignee: (was: Apache Spark) > Remove statistics-related table properties from SHOW

[jira] [Commented] (SPARK-17284) Remove statistics-related table properties from SHOW CREATE TABLE

2016-08-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15444259#comment-15444259 ] Apache Spark commented on SPARK-17284: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17284) Remove statistics-related table properties from SHOW CREATE TABLE

2016-08-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17284: Assignee: Apache Spark > Remove statistics-related table properties from SHOW CREATE

[jira] [Created] (SPARK-17284) Remove statistics-related table properties from SHOW CREATE TABLE

2016-08-28 Thread Xiao Li (JIRA)
Xiao Li created SPARK-17284: --- Summary: Remove statistics-related table properties from SHOW CREATE TABLE Key: SPARK-17284 URL: https://issues.apache.org/jira/browse/SPARK-17284 Project: Spark

[jira] [Commented] (SPARK-17283) Cancel job in RDD.take() as soon as enough output is receieved

2016-08-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15443975#comment-15443975 ] Apache Spark commented on SPARK-17283: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17283) Cancel job in RDD.take() as soon as enough output is receieved

2016-08-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17283: Assignee: Apache Spark (was: Josh Rosen) > Cancel job in RDD.take() as soon as enough

[jira] [Assigned] (SPARK-17283) Cancel job in RDD.take() as soon as enough output is receieved

2016-08-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17283: Assignee: Josh Rosen (was: Apache Spark) > Cancel job in RDD.take() as soon as enough

[jira] [Created] (SPARK-17283) Cancel job in RDD.take() as soon as enough output is receieved

2016-08-28 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-17283: -- Summary: Cancel job in RDD.take() as soon as enough output is receieved Key: SPARK-17283 URL: https://issues.apache.org/jira/browse/SPARK-17283 Project: Spark

[jira] [Resolved] (SPARK-17271) Planner adds un-necessary Sort even if child ordering is semantically same as required ordering

2016-08-28 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17271. --- Resolution: Fixed Assignee: Tejas Patil Fix Version/s: 2.1.0 >

[jira] [Commented] (SPARK-17041) Columns in schema are no longer case sensitive when reading csv file

2016-08-28 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15443779#comment-15443779 ] koert kuipers commented on SPARK-17041: --- note that the docs say: "It is highly discouraged to turn

[jira] [Updated] (SPARK-17282) Implement ALTER TABLE UPDATE STATISTICS SET

2016-08-28 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-17282: Issue Type: Sub-task (was: Improvement) Parent: SPARK-16026 > Implement ALTER TABLE UPDATE

[jira] [Created] (SPARK-17282) Implement ALTER TABLE UPDATE STATISTICS SET

2016-08-28 Thread Xiao Li (JIRA)
Xiao Li created SPARK-17282: --- Summary: Implement ALTER TABLE UPDATE STATISTICS SET Key: SPARK-17282 URL: https://issues.apache.org/jira/browse/SPARK-17282 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-12619) Combine small files in a hadoop directory into single split

2016-08-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-12619. --- Resolution: Duplicate > Combine small files in a hadoop directory into single split >

[jira] [Resolved] (SPARK-17214) How to deal with dots (.) present in column names in SparkR

2016-08-28 Thread Mohit Bansal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohit Bansal resolved SPARK-17214. -- Resolution: Later > How to deal with dots (.) present in column names in SparkR >

[jira] [Commented] (SPARK-16647) sparksql1.6.2 on yarn with hive metastore1.0.0 thows "alter_table_with_cascade" exception

2016-08-28 Thread fengchaoge (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15443140#comment-15443140 ] fengchaoge commented on SPARK-16647: Do you have resolve this problem? I have the same problem. my

[jira] [Closed] (SPARK-15044) spark-sql will throw "input path does not exist" exception if it handles a partition which exists in hive table, but the path is removed manually

2016-08-28 Thread huangyu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huangyu closed SPARK-15044. --- > spark-sql will throw "input path does not exist" exception if it handles a > partition which exists in hive

[jira] [Comment Edited] (SPARK-17214) How to deal with dots (.) present in column names in SparkR

2016-08-28 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15442870#comment-15442870 ] Felix Cheung edited comment on SPARK-17214 at 8/28/16 6:14 AM: --- I think the

[jira] [Comment Edited] (SPARK-17214) How to deal with dots (.) present in column names in SparkR

2016-08-28 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15442870#comment-15442870 ] Felix Cheung edited comment on SPARK-17214 at 8/28/16 6:15 AM: --- I think the

[jira] [Commented] (SPARK-17214) How to deal with dots (.) present in column names in SparkR

2016-08-28 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15442870#comment-15442870 ] Felix Cheung commented on SPARK-17214: -- I think the underlining issue is that we should either

[jira] [Commented] (SPARK-17214) How to deal with dots (.) present in column names in SparkR

2016-08-28 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15442865#comment-15442865 ] Felix Cheung commented on SPARK-17214: -- [~bansalism] what version of Spark + SparkR are you testing