[jira] [Resolved] (SPARK-15160) support data source table in InMemoryCatalog

2016-05-11 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-15160. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12935 [https://github.com/

[jira] [Commented] (SPARK-7703) Task failure caused by block fetch failure in BlockManager.doGetRemote() when using TorrentBroadcast

2016-05-11 Thread jianbo li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281257#comment-15281257 ] jianbo li commented on SPARK-7703: -- Hi Hailong Wen, Do you know when will this issue hap

[jira] [Assigned] (SPARK-15288) Mesos dispatcher should handle gracefully when any thread gets UncaughtException

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15288: Assignee: (was: Apache Spark) > Mesos dispatcher should handle gracefully when any thr

[jira] [Commented] (SPARK-15288) Mesos dispatcher should handle gracefully when any thread gets UncaughtException

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281249#comment-15281249 ] Apache Spark commented on SPARK-15288: -- User 'devaraj-kavali' has created a pull req

[jira] [Assigned] (SPARK-15288) Mesos dispatcher should handle gracefully when any thread gets UncaughtException

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15288: Assignee: Apache Spark > Mesos dispatcher should handle gracefully when any thread gets >

[jira] [Created] (SPARK-15288) Mesos dispatcher should handle gracefully when any thread gets UncaughtException

2016-05-11 Thread Devaraj K (JIRA)
Devaraj K created SPARK-15288: - Summary: Mesos dispatcher should handle gracefully when any thread gets UncaughtException Key: SPARK-15288 URL: https://issues.apache.org/jira/browse/SPARK-15288 Project: S

[jira] [Assigned] (SPARK-15281) PySpark ML GBTRegressor lacks impurity param

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15281: Assignee: (was: Apache Spark) > PySpark ML GBTRegressor lacks impurity param > ---

[jira] [Assigned] (SPARK-15281) PySpark ML GBTRegressor lacks impurity param

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15281: Assignee: Apache Spark > PySpark ML GBTRegressor lacks impurity param > --

[jira] [Commented] (SPARK-15281) PySpark ML GBTRegressor lacks impurity param

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281236#comment-15281236 ] Apache Spark commented on SPARK-15281: -- User 'holdenk' has created a pull request fo

[jira] [Comment Edited] (SPARK-15247) sqlCtx.read.parquet yields at least n_executors * n_cores tasks

2016-05-11 Thread Johnny W. (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281221#comment-15281221 ] Johnny W. edited comment on SPARK-15247 at 5/12/16 5:28 AM: T

[jira] [Comment Edited] (SPARK-15247) sqlCtx.read.parquet yields at least n_executors * n_cores tasks

2016-05-11 Thread Johnny W. (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281221#comment-15281221 ] Johnny W. edited comment on SPARK-15247 at 5/12/16 5:28 AM: T

[jira] [Commented] (SPARK-15247) sqlCtx.read.parquet yields at least n_executors * n_cores tasks

2016-05-11 Thread Johnny W. (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281221#comment-15281221 ] Johnny W. commented on SPARK-15247: --- Takeshi, thanks for your reply. However, I don't

[jira] [Commented] (SPARK-15287) Spark SQL partition filter clause with different literal type will scan all hive partitions

2016-05-11 Thread Tao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281219#comment-15281219 ] Tao Li commented on SPARK-15287: [~davies] > Spark SQL partition filter clause with diff

[jira] [Updated] (SPARK-15247) sqlCtx.read.parquet yields at least n_executors * n_cores tasks

2016-05-11 Thread Johnny W. (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johnny W. updated SPARK-15247: -- Description: sqlCtx.read.parquet always yields at least n_executors * n_cores tasks, even though this

[jira] [Created] (SPARK-15287) Spark SQL partition filter clause with different literal type will scan all hive partitions

2016-05-11 Thread Tao Li (JIRA)
Tao Li created SPARK-15287: -- Summary: Spark SQL partition filter clause with different literal type will scan all hive partitions Key: SPARK-15287 URL: https://issues.apache.org/jira/browse/SPARK-15287 Proje

[jira] [Commented] (SPARK-14814) ML 2.0 QA: API: Java compatibility, docs

2016-05-11 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281214#comment-15281214 ] yuhao yang commented on SPARK-14814: The PR has been merged. I'll resolve the issue i

[jira] [Assigned] (SPARK-15286) Make the output readable for EXPLAIN CREATE TABLE and DESC EXTENDED

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15286: Assignee: Apache Spark > Make the output readable for EXPLAIN CREATE TABLE and DESC EXTEND

[jira] [Assigned] (SPARK-15286) Make the output readable for EXPLAIN CREATE TABLE and DESC EXTENDED

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15286: Assignee: (was: Apache Spark) > Make the output readable for EXPLAIN CREATE TABLE and

[jira] [Commented] (SPARK-15286) Make the output readable for EXPLAIN CREATE TABLE and DESC EXTENDED

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281205#comment-15281205 ] Apache Spark commented on SPARK-15286: -- User 'gatorsmile' has created a pull request

[jira] [Commented] (SPARK-15072) Remove SparkSession.withHiveSupport

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281199#comment-15281199 ] Apache Spark commented on SPARK-15072: -- User 'yhuai' has created a pull request for

[jira] [Updated] (SPARK-15286) Make the output readable for EXPLAIN CREATE TABLE and DESC EXTENDED

2016-05-11 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-15286: Description: Currently, the outputs of EXPLAIN CREATE TABLE and DESC EXTENDED are not readable. For exam

[jira] [Created] (SPARK-15286) Make the output readable for EXPLAIN CREATE TABLE and DESC EXTENDED

2016-05-11 Thread Xiao Li (JIRA)
Xiao Li created SPARK-15286: --- Summary: Make the output readable for EXPLAIN CREATE TABLE and DESC EXTENDED Key: SPARK-15286 URL: https://issues.apache.org/jira/browse/SPARK-15286 Project: Spark I

[jira] [Comment Edited] (SPARK-15174) DataFrame does not have correct number of rows after dropDuplicates

2016-05-11 Thread hujiayin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273805#comment-15273805 ] hujiayin edited comment on SPARK-15174 at 5/12/16 4:09 AM: --- I c

[jira] [Resolved] (SPARK-14346) SHOW CREATE TABLE command (Native)

2016-05-11 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-14346. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12781 [https://github.com/

[jira] [Updated] (SPARK-14346) SHOW CREATE TABLE command (Native)

2016-05-11 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-14346: - Assignee: Cheng Lian > SHOW CREATE TABLE command (Native) > -- > >

[jira] [Resolved] (SPARK-15080) Break copyAndReset into copy and reset

2016-05-11 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-15080. - Resolution: Fixed Issue resolved by pull request 12936 [https://github.com/apache/spark/pull/1293

[jira] [Comment Edited] (SPARK-14750) Make historyServer refer application log in hdfs

2016-05-11 Thread SuYan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281144#comment-15281144 ] SuYan edited comment on SPARK-14750 at 5/12/16 3:08 AM: [~vanzin]

[jira] [Commented] (SPARK-14750) Make historyServer refer application log in hdfs

2016-05-11 Thread SuYan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281144#comment-15281144 ] SuYan commented on SPARK-14750: --- ah, Thanks to help me find a simple way to do this... > M

[jira] [Commented] (SPARK-15159) SparkSession R API

2016-05-11 Thread Kai Jiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281118#comment-15281118 ] Kai Jiang commented on SPARK-15159: --- I'd like to take a try on this one. > SparkSessio

[jira] [Comment Edited] (SPARK-15159) SparkSession R API

2016-05-11 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281105#comment-15281105 ] Sun Rui edited comment on SPARK-15159 at 5/12/16 2:22 AM: -- maybe

[jira] [Created] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-11 Thread Konstantin Shaposhnikov (JIRA)
Konstantin Shaposhnikov created SPARK-15285: --- Summary: Generated SpecificSafeProjection.apply method grows beyond 64 KB Key: SPARK-15285 URL: https://issues.apache.org/jira/browse/SPARK-15285

[jira] [Updated] (SPARK-15284) Memory when use SparkSQL in SparkStream

2016-05-11 Thread kramer1987 (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kramer1987 updated SPARK-15284: --- Environment: centos7 , spark1.6.1, python2.7.5 Description: http://apache-spark-user-list.1001560

[jira] [Created] (SPARK-15284) Memory when use SparkSQL in SparkStream

2016-05-11 Thread kramer1987 (JIRA)
kramer1987 created SPARK-15284: -- Summary: Memory when use SparkSQL in SparkStream Key: SPARK-15284 URL: https://issues.apache.org/jira/browse/SPARK-15284 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-15159) SparkSession R API

2016-05-11 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281105#comment-15281105 ] Sun Rui commented on SPARK-15159: - maybe this can be split into several subtasks. > Spar

[jira] [Commented] (SPARK-15159) SparkSession R API

2016-05-11 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281104#comment-15281104 ] Sun Rui commented on SPARK-15159: - @Vijary Parmar, I changed the scope of this PR. You ca

[jira] [Updated] (SPARK-15159) SparkSession R API

2016-05-11 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui updated SPARK-15159: Summary: SparkSession R API (was: Remove usage of HiveContext in SparkR.) > SparkSession R API > -

[jira] [Commented] (SPARK-15159) Remove usage of HiveContext in SparkR.

2016-05-11 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281100#comment-15281100 ] Sun Rui commented on SPARK-15159: - ok, let's do more to match pyspark:) > Remove usage o

[jira] [Commented] (SPARK-14082) Add support for GPU resource when running on Mesos

2016-05-11 Thread Kevin Klues (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281091#comment-15281091 ] Kevin Klues commented on SPARK-14082: - We are getting close to finalizing our support

[jira] [Resolved] (SPARK-15283) YARN Application should get FAILED state if driver is failed in yarn-client mode

2016-05-11 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-15283. Resolution: Won't Fix Sorry but there's no reliable way to do this in client mode. We've tr

[jira] [Updated] (SPARK-15283) YARN Application should get FAILED state if driver is failed in yarn-client mode

2016-05-11 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-15283: Component/s: YARN > YARN Application should get FAILED state if driver is failed in yarn-client >

[jira] [Created] (SPARK-15283) YARN Application should get FAILED state if driver is failed in yarn-client mode

2016-05-11 Thread Yesha Vora (JIRA)
Yesha Vora created SPARK-15283: -- Summary: YARN Application should get FAILED state if driver is failed in yarn-client mode Key: SPARK-15283 URL: https://issues.apache.org/jira/browse/SPARK-15283 Project:

[jira] [Commented] (SPARK-15282) UDF functioin executed twice when filter on new column created by withColumn

2016-05-11 Thread Linbo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281075#comment-15281075 ] Linbo commented on SPARK-15282: --- There is a related JIRA issue: https://issues.apache.org/

[jira] [Updated] (SPARK-15282) UDF functioin executed twice when filter on new column created by withColumn

2016-05-11 Thread Linbo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Linbo updated SPARK-15282: -- Description: I found this problem on spark version 1.6.1 and based on [~tedyu] in current master branch, the

[jira] [Created] (SPARK-15282) UDF functioin executed twice when filter on new column created by withColumn

2016-05-11 Thread Linbo (JIRA)
Linbo created SPARK-15282: - Summary: UDF functioin executed twice when filter on new column created by withColumn Key: SPARK-15282 URL: https://issues.apache.org/jira/browse/SPARK-15282 Project: Spark

[jira] [Resolved] (SPARK-15264) Spark 2.0 CSV Reader: NPE on Blank Column Names

2016-05-11 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15264. --- Resolution: Fixed Fix Version/s: 2.0.0 > Spark 2.0 CSV Reader: NPE on Blank Column Names > ---

[jira] [Resolved] (SPARK-15274) CSV default column names should be consistent

2016-05-11 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15274. --- Resolution: Fixed Fix Version/s: 2.0.0 > CSV default column names should be consistent > -

[jira] [Assigned] (SPARK-15279) Disallow ROW FORMAT and STORED AS (parquet | orc | avro etc.)

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15279: Assignee: Andrew Or (was: Apache Spark) > Disallow ROW FORMAT and STORED AS (parquet | or

[jira] [Assigned] (SPARK-15279) Disallow ROW FORMAT and STORED AS (parquet | orc | avro etc.)

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15279: Assignee: Apache Spark (was: Andrew Or) > Disallow ROW FORMAT and STORED AS (parquet | or

[jira] [Commented] (SPARK-15279) Disallow ROW FORMAT and STORED AS (parquet | orc | avro etc.)

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281055#comment-15281055 ] Apache Spark commented on SPARK-15279: -- User 'andrewor14' has created a pull request

[jira] [Updated] (SPARK-13522) Executor should kill itself when it's unable to heartbeat to the driver more than N times

2016-05-11 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-13522: - Fix Version/s: 1.6.2 > Executor should kill itself when it's unable to heartbeat to the driver mo

[jira] [Resolved] (SPARK-15276) CREATE TABLE with LOCATION should imply EXTERNAL

2016-05-11 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15276. --- Resolution: Fixed Fix Version/s: 2.0.0 > CREATE TABLE with LOCATION should imply EXTERNAL > --

[jira] [Updated] (SPARK-13522) Executor should kill itself when it's unable to heartbeat to the driver more than N times

2016-05-11 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13522: --- Fix Version/s: (was: 1.6.2) > Executor should kill itself when it's unable to heartbeat to the dr

[jira] [Updated] (SPARK-15260) UnifiedMemoryManager could be in bad state if any exception happen while evicting blocks

2016-05-11 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15260: --- Fix Version/s: 1.6.2 > UnifiedMemoryManager could be in bad state if any exception happen while > ev

[jira] [Updated] (SPARK-13522) Executor should kill itself when it's unable to heartbeat to the driver more than N times

2016-05-11 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13522: --- Fix Version/s: 1.6.2 > Executor should kill itself when it's unable to heartbeat to the driver more

[jira] [Created] (SPARK-15281) PySpark ML GBTRegressor lacks impurity param

2016-05-11 Thread holdenk (JIRA)
holdenk created SPARK-15281: --- Summary: PySpark ML GBTRegressor lacks impurity param Key: SPARK-15281 URL: https://issues.apache.org/jira/browse/SPARK-15281 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-14209) Application failure during preemption.

2016-05-11 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280980#comment-15280980 ] Marcelo Vanzin commented on SPARK-14209: Hi [~milesc], I look at your updates aft

[jira] [Commented] (SPARK-4131) Support "Writing data into the filesystem from queries"

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280961#comment-15280961 ] Apache Spark commented on SPARK-4131: - User 'Parth-Brahmbhatt' has created a pull requ

[jira] [Assigned] (SPARK-15280) Extract ORC serialization logic from OrcOutputWriter for reusability

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15280: Assignee: Apache Spark > Extract ORC serialization logic from OrcOutputWriter for reusabi

[jira] [Commented] (SPARK-15280) Extract ORC serialization logic from OrcOutputWriter for reusability

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280958#comment-15280958 ] Apache Spark commented on SPARK-15280: -- User 'seyfe' has created a pull request for

[jira] [Assigned] (SPARK-15280) Extract ORC serialization logic from OrcOutputWriter for reusability

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15280: Assignee: (was: Apache Spark) > Extract ORC serialization logic from OrcOutputWriter

[jira] [Created] (SPARK-15280) Extract ORC serialization logic from OrcOutputWriter for reusability

2016-05-11 Thread Ergin Seyfe (JIRA)
Ergin Seyfe created SPARK-15280: --- Summary: Extract ORC serialization logic from OrcOutputWriter for reusability Key: SPARK-15280 URL: https://issues.apache.org/jira/browse/SPARK-15280 Project: Spark

[jira] [Created] (SPARK-15279) Disallow ROW FORMAT and STORED AS (parquet | orc | avro etc.)

2016-05-11 Thread Andrew Or (JIRA)
Andrew Or created SPARK-15279: - Summary: Disallow ROW FORMAT and STORED AS (parquet | orc | avro etc.) Key: SPARK-15279 URL: https://issues.apache.org/jira/browse/SPARK-15279 Project: Spark Issu

[jira] [Assigned] (SPARK-15214) Enable code generation for Generate

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15214: Assignee: Herman van Hovell (was: Apache Spark) > Enable code generation for Generate > -

[jira] [Assigned] (SPARK-15214) Enable code generation for Generate

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15214: Assignee: Apache Spark (was: Herman van Hovell) > Enable code generation for Generate > -

[jira] [Commented] (SPARK-15214) Enable code generation for Generate

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280934#comment-15280934 ] Apache Spark commented on SPARK-15214: -- User 'hvanhovell' has created a pull request

[jira] [Commented] (SPARK-13872) Memory leak in SortMergeOuterJoin

2016-05-11 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280932#comment-15280932 ] Tejas Patil commented on SPARK-13872: - [~ianlcsd] Can you add a small repro query to

[jira] [Resolved] (SPARK-15257) Require CREATE EXTERNAL TABLE to specify LOCATION

2016-05-11 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-15257. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13032 [https://github.com/

[jira] [Updated] (SPARK-15256) Clarify the docstring for DataFrameReader.jdbc()

2016-05-11 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15256: --- Assignee: Nicholas Chammas > Clarify the docstring for DataFrameReader.jdbc() > -

[jira] [Resolved] (SPARK-15256) Clarify the docstring for DataFrameReader.jdbc()

2016-05-11 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15256. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13034 [https://github.

[jira] [Comment Edited] (SPARK-15269) Creating external table in test code leaves empty directory under warehouse directory

2016-05-11 Thread Xin Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280877#comment-15280877 ] Xin Wu edited comment on SPARK-15269 at 5/11/16 10:13 PM: -- The r

[jira] [Resolved] (SPARK-15278) Remove experimental tag from Python DataFrame

2016-05-11 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15278. Resolution: Fixed Issue resolved by pull request 13062 [https://github.com/apache/spark/pull/13062]

[jira] [Updated] (SPARK-14803) A bug in EliminateSerialization rule in Catalyst Optimizer

2016-05-11 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-14803: - Target Version/s: 2.0.0 > A bug in EliminateSerialization rule in Catalyst Optimizer > -

[jira] [Assigned] (SPARK-15061) Upgrade Py4J to 0.10.1

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15061: Assignee: Apache Spark (was: holdenk) > Upgrade Py4J to 0.10.1 > -- >

[jira] [Commented] (SPARK-15061) Upgrade Py4J to 0.10.1

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280898#comment-15280898 ] Apache Spark commented on SPARK-15061: -- User 'holdenk' has created a pull request fo

[jira] [Assigned] (SPARK-15061) Upgrade Py4J to 0.10.1

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15061: Assignee: holdenk (was: Apache Spark) > Upgrade Py4J to 0.10.1 > -- >

[jira] [Commented] (SPARK-14959) ​Problem Reading partitioned ORC or Parquet files

2016-05-11 Thread Jurriaan Pruis (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280895#comment-15280895 ] Jurriaan Pruis commented on SPARK-14959: I have the same issue reading a partitio

[jira] [Commented] (SPARK-15269) Creating external table in test code leaves empty directory under warehouse directory

2016-05-11 Thread Xin Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280888#comment-15280888 ] Xin Wu commented on SPARK-15269: In spark-shell, I can recreate it as following: {code} s

[jira] [Comment Edited] (SPARK-15269) Creating external table in test code leaves empty directory under warehouse directory

2016-05-11 Thread Xin Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280877#comment-15280877 ] Xin Wu edited comment on SPARK-15269 at 5/11/16 9:47 PM: - The roo

[jira] [Commented] (SPARK-15072) Remove SparkSession.withHiveSupport

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280878#comment-15280878 ] Apache Spark commented on SPARK-15072: -- User 'techaddict' has created a pull request

[jira] [Commented] (SPARK-15269) Creating external table in test code leaves empty directory under warehouse directory

2016-05-11 Thread Xin Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280877#comment-15280877 ] Xin Wu commented on SPARK-15269: The root cause maybe the following? When the first tabl

[jira] [Commented] (SPARK-15278) Remove experimental tag from Python DataFrame

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280872#comment-15280872 ] Apache Spark commented on SPARK-15278: -- User 'rxin' has created a pull request for t

[jira] [Assigned] (SPARK-15278) Remove experimental tag from Python DataFrame

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15278: Assignee: Reynold Xin (was: Apache Spark) > Remove experimental tag from Python DataFrame

[jira] [Assigned] (SPARK-15278) Remove experimental tag from Python DataFrame

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15278: Assignee: Apache Spark (was: Reynold Xin) > Remove experimental tag from Python DataFrame

[jira] [Created] (SPARK-15278) Remove experimental tag from Python DataFrame

2016-05-11 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-15278: --- Summary: Remove experimental tag from Python DataFrame Key: SPARK-15278 URL: https://issues.apache.org/jira/browse/SPARK-15278 Project: Spark Issue Type: Sub-t

[jira] [Commented] (SPARK-14279) Improve the spark build to pick the version information from the pom file and add git commit information

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280871#comment-15280871 ] Apache Spark commented on SPARK-14279: -- User 'dhruve' has created a pull request for

[jira] [Updated] (SPARK-15275) CatalogTable should store sort ordering for sorted columns

2016-05-11 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15275: -- Summary: CatalogTable should store sort ordering for sorted columns (was: [SQL] CatalogTable should st

[jira] [Updated] (SPARK-15275) CatalogTable should store sort ordering for sorted columns

2016-05-11 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15275: -- Priority: Major (was: Trivial) > CatalogTable should store sort ordering for sorted columns >

[jira] [Updated] (SPARK-15275) CatalogTable should store sort ordering for sorted columns

2016-05-11 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15275: -- Assignee: Tejas Patil > CatalogTable should store sort ordering for sorted columns > --

[jira] [Updated] (SPARK-15275) CatalogTable should store sort ordering for sorted columns

2016-05-11 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15275: -- Affects Version/s: (was: 1.6.1) 2.0.0 > CatalogTable should store sort order

[jira] [Commented] (SPARK-15276) CREATE TABLE with LOCATION should imply EXTERNAL

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280851#comment-15280851 ] Apache Spark commented on SPARK-15276: -- User 'andrewor14' has created a pull request

[jira] [Assigned] (SPARK-15277) Checking Partition Spec Existence Before Dropping

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15277: Assignee: (was: Apache Spark) > Checking Partition Spec Existence Before Dropping > --

[jira] [Commented] (SPARK-15277) Checking Partition Spec Existence Before Dropping

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280852#comment-15280852 ] Apache Spark commented on SPARK-15277: -- User 'gatorsmile' has created a pull request

[jira] [Assigned] (SPARK-15276) CREATE TABLE with LOCATION should imply EXTERNAL

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15276: Assignee: Apache Spark (was: Andrew Or) > CREATE TABLE with LOCATION should imply EXTERNA

[jira] [Created] (SPARK-15277) Checking Partition Spec Existence Before Dropping

2016-05-11 Thread Xiao Li (JIRA)
Xiao Li created SPARK-15277: --- Summary: Checking Partition Spec Existence Before Dropping Key: SPARK-15277 URL: https://issues.apache.org/jira/browse/SPARK-15277 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-15276) CREATE TABLE with LOCATION should imply EXTERNAL

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15276: Assignee: Andrew Or (was: Apache Spark) > CREATE TABLE with LOCATION should imply EXTERNA

[jira] [Assigned] (SPARK-15277) Checking Partition Spec Existence Before Dropping

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15277: Assignee: Apache Spark > Checking Partition Spec Existence Before Dropping > -

[jira] [Resolved] (SPARK-15270) Creating HiveContext does not work

2016-05-11 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15270. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13056 [https://github.

[jira] [Closed] (SPARK-15026) Disallow Dropping Multi Partitions By a Single Alter Table DDL Command

2016-05-11 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li closed SPARK-15026. --- Resolution: Won't Fix > Disallow Dropping Multi Partitions By a Single Alter Table DDL Command >

[jira] [Commented] (SPARK-13850) TimSort Comparison method violates its general contract

2016-05-11 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280841#comment-15280841 ] Yin Huai commented on SPARK-13850: -- SPARK-12055 duplicates https://issues.apache.org/ji

[jira] [Commented] (SPARK-15275) [SQL] CatalogTable should store sort ordering for sorted columns

2016-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280831#comment-15280831 ] Apache Spark commented on SPARK-15275: -- User 'tejasapatil' has created a pull reques

  1   2   3   >