[jira] [Updated] (SPARK-18123) org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils.saveTable the case senstivity issue

2016-10-26 Thread Paul Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Wu updated SPARK-18123: Description: Blindly quoting every field name for inserting is the issue (Line 110-119,

[jira] [Commented] (SPARK-18123) org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils.saveTable the case senstivity issue

2016-10-26 Thread Paul Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610739#comment-15610739 ] Paul Wu commented on SPARK-18123: - Just tried if it worked for the issue. But after I found the code, I

[jira] [Updated] (SPARK-18132) spark 2.0 branch's spark-release-publish failed because style check failed.

2016-10-26 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-18132: - Target Version/s: 2.1.0, 2.0.3 (was: 2.0.2, 2.1.0) > spark 2.0 branch's spark-release-publish failed

[jira] [Resolved] (SPARK-18132) spark 2.0 branch's spark-release-publish failed because style check failed.

2016-10-26 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-18132. -- Resolution: Fixed Fix Version/s: 2.1.0 2.0.3 Issue resolved by pull request

[jira] [Comment Edited] (SPARK-18123) org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils.saveTable the case senstivity issue

2016-10-26 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610713#comment-15610713 ] Dongjoon Hyun edited comment on SPARK-18123 at 10/27/16 5:21 AM: - Hi,

[jira] [Commented] (SPARK-18123) org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils.saveTable the case senstivity issue

2016-10-26 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610713#comment-15610713 ] Dongjoon Hyun commented on SPARK-18123: --- Hi, [~zwu@gmail.com]. Thank you for reporting. BTW,

[jira] [Updated] (SPARK-18009) Spark 2.0.1 SQL Thrift Error

2016-10-26 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-18009: Fix Version/s: (was: 2.0.3) 2.0.2 > Spark 2.0.1 SQL Thrift Error >

[jira] [Updated] (SPARK-18009) Spark 2.0.1 SQL Thrift Error

2016-10-26 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-18009: Fix Version/s: (was: 2.0.2) 2.0.3 > Spark 2.0.1 SQL Thrift Error >

[jira] [Updated] (SPARK-18009) Spark 2.0.1 SQL Thrift Error

2016-10-26 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-18009: Assignee: Dilip Biswal > Spark 2.0.1 SQL Thrift Error > > >

[jira] [Resolved] (SPARK-18009) Spark 2.0.1 SQL Thrift Error

2016-10-26 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-18009. - Resolution: Fixed Fix Version/s: 2.1.0 2.0.2 Issue resolved by pull

[jira] [Commented] (SPARK-18132) spark 2.0 branch's spark-release-publish failed because style check failed.

2016-10-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610693#comment-15610693 ] Apache Spark commented on SPARK-18132: -- User 'yhuai' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18132) spark 2.0 branch's spark-release-publish failed because style check failed.

2016-10-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18132: Assignee: Yin Huai (was: Apache Spark) > spark 2.0 branch's spark-release-publish failed

[jira] [Assigned] (SPARK-18132) spark 2.0 branch's spark-release-publish failed because style check failed.

2016-10-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18132: Assignee: Apache Spark (was: Yin Huai) > spark 2.0 branch's spark-release-publish failed

[jira] [Assigned] (SPARK-18132) spark 2.0 branch's spark-release-publish failed because style check failed.

2016-10-26 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai reassigned SPARK-18132: Assignee: Yin Huai > spark 2.0 branch's spark-release-publish failed because style check failed.

[jira] [Created] (SPARK-18132) spark 2.0 branch's spark-release-publish failed because style check failed.

2016-10-26 Thread Yin Huai (JIRA)
Yin Huai created SPARK-18132: Summary: spark 2.0 branch's spark-release-publish failed because style check failed. Key: SPARK-18132 URL: https://issues.apache.org/jira/browse/SPARK-18132 Project: Spark

[jira] [Created] (SPARK-18131) Support returning Vector/Dense Vector from backend

2016-10-26 Thread Miao Wang (JIRA)
Miao Wang created SPARK-18131: - Summary: Support returning Vector/Dense Vector from backend Key: SPARK-18131 URL: https://issues.apache.org/jira/browse/SPARK-18131 Project: Spark Issue Type: New

[jira] [Commented] (SPARK-9279) Spark Master Refuses to Bind WebUI to a Privileged Port

2016-10-26 Thread Suresh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610642#comment-15610642 ] Suresh commented on SPARK-9279: --- Folks this limitation to not allow binding Spark UI to port 80, even as

[jira] [Commented] (SPARK-18010) Remove unneeded heavy work performed by FsHistoryProvider for building up the application listing UI page

2016-10-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610597#comment-15610597 ] Apache Spark commented on SPARK-18010: -- User 'vijoshi' has created a pull request for this issue:

[jira] [Updated] (SPARK-18113) Sending AskPermissionToCommitOutput failed, driver enter into task deadloop

2016-10-26 Thread xuqing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xuqing updated SPARK-18113: --- Description: Executor sends *AskPermissionToCommitOutput* to driver failed, and retry another sending.

[jira] [Updated] (SPARK-18113) Sending AskPermissionToCommitOutput failed, driver enter into task deadloop

2016-10-26 Thread xuqing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xuqing updated SPARK-18113: --- Description: Executor sends *AskPermissionToCommitOutput* to driver and failed, and retry another sending.

[jira] [Updated] (SPARK-18073) Migrate wiki to spark.apache.org web site

2016-10-26 Thread Sandeep Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-18073: -- Description: Per

[jira] [Updated] (SPARK-18113) Sending AskPermissionToCommitOutput failed, driver enter into task deadloop

2016-10-26 Thread xuqing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xuqing updated SPARK-18113: --- Summary: Sending AskPermissionToCommitOutput failed, driver enter into task deadloop (was:

[jira] [Commented] (SPARK-16881) Migrate Mesos configs to use ConfigEntry

2016-10-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610485#comment-15610485 ] Apache Spark commented on SPARK-16881: -- User 'techaddict' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16881) Migrate Mesos configs to use ConfigEntry

2016-10-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16881: Assignee: (was: Apache Spark) > Migrate Mesos configs to use ConfigEntry >

[jira] [Assigned] (SPARK-16881) Migrate Mesos configs to use ConfigEntry

2016-10-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16881: Assignee: Apache Spark > Migrate Mesos configs to use ConfigEntry >

[jira] [Comment Edited] (SPARK-18107) Insert overwrite statement runs much slower in spark-sql than it does in hive-client

2016-10-26 Thread J.P Feng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610454#comment-15610454 ] J.P Feng edited comment on SPARK-18107 at 10/27/16 3:01 AM: sure, the source

[jira] [Commented] (SPARK-15616) Metastore relation should fallback to HDFS size of partitions that are involved in Query if statistics are not available.

2016-10-26 Thread Lianhui Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610455#comment-15610455 ] Lianhui Wang commented on SPARK-15616: -- if filter is for partition key, So this issue can did it for

[jira] [Commented] (SPARK-18107) Insert overwrite statement runs much slower in spark-sql than it does in hive-client

2016-10-26 Thread J.P Feng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610454#comment-15610454 ] J.P Feng commented on SPARK-18107: -- sure, the source table and the target table are both Hive table,

[jira] [Issue Comment Deleted] (SPARK-18107) Insert overwrite statement runs much slower in spark-sql than it does in hive-client

2016-10-26 Thread J.P Feng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] J.P Feng updated SPARK-18107: - Comment: was deleted (was: sure, the source table and the target table are both Hive table, like this:

[jira] [Comment Edited] (SPARK-18107) Insert overwrite statement runs much slower in spark-sql than it does in hive-client

2016-10-26 Thread J.P Feng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610429#comment-15610429 ] J.P Feng edited comment on SPARK-18107 at 10/27/16 2:54 AM: sure, the source

[jira] [Comment Edited] (SPARK-18107) Insert overwrite statement runs much slower in spark-sql than it does in hive-client

2016-10-26 Thread J.P Feng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610429#comment-15610429 ] J.P Feng edited comment on SPARK-18107 at 10/27/16 2:51 AM: sure, the source

[jira] [Issue Comment Deleted] (SPARK-18107) Insert overwrite statement runs much slower in spark-sql than it does in hive-client

2016-10-26 Thread J.P Feng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] J.P Feng updated SPARK-18107: - Comment: was deleted (was: sure, the source table and the target table are both Hive table, like this:)

[jira] [Commented] (SPARK-18130) Don't add inferred redundant isnotnull condition from constraints

2016-10-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610410#comment-15610410 ] Apache Spark commented on SPARK-18130: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18130) Don't add inferred redundant isnotnull condition from constraints

2016-10-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18130: Assignee: (was: Apache Spark) > Don't add inferred redundant isnotnull condition from

[jira] [Commented] (SPARK-18107) Insert overwrite statement runs much slower in spark-sql than it does in hive-client

2016-10-26 Thread J.P Feng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610430#comment-15610430 ] J.P Feng commented on SPARK-18107: -- sure, the source table and the target table are both Hive table,

[jira] [Commented] (SPARK-18107) Insert overwrite statement runs much slower in spark-sql than it does in hive-client

2016-10-26 Thread J.P Feng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610429#comment-15610429 ] J.P Feng commented on SPARK-18107: -- sure, the source table and the target table are both Hive table,

[jira] [Assigned] (SPARK-18130) Don't add inferred redundant isnotnull condition from constraints

2016-10-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18130: Assignee: Apache Spark > Don't add inferred redundant isnotnull condition from

[jira] [Created] (SPARK-18130) Don't add inferred redundant isnotnull condition from constraints

2016-10-26 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-18130: --- Summary: Don't add inferred redundant isnotnull condition from constraints Key: SPARK-18130 URL: https://issues.apache.org/jira/browse/SPARK-18130 Project:

[jira] [Commented] (SPARK-17829) Stable format for offset log

2016-10-26 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610399#comment-15610399 ] Cody Koeninger commented on SPARK-17829: I'm not telling you to do it that way, just asking if

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

2016-10-26 Thread KaiXu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610286#comment-15610286 ] KaiXu commented on SPARK-18112: --- Spark 2.0 removes JavaSparkListener and change SparkListener from a trait

[jira] [Updated] (SPARK-18113) AskPermissionToCommitOutput response lost, driver enter into task deadloop

2016-10-26 Thread xuqing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xuqing updated SPARK-18113: --- Priority: Major (was: Minor) > AskPermissionToCommitOutput response lost, driver enter into task deadloop >

[jira] [Resolved] (SPARK-17770) Make ObjectType SQL Type Public

2016-10-26 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-17770. -- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 15453

[jira] [Created] (SPARK-18129) Sign pip artifacts

2016-10-26 Thread holdenk (JIRA)
holdenk created SPARK-18129: --- Summary: Sign pip artifacts Key: SPARK-18129 URL: https://issues.apache.org/jira/browse/SPARK-18129 Project: Spark Issue Type: Improvement Components:

[jira] [Created] (SPARK-18128) Add support for publishing to PyPI

2016-10-26 Thread holdenk (JIRA)
holdenk created SPARK-18128: --- Summary: Add support for publishing to PyPI Key: SPARK-18128 URL: https://issues.apache.org/jira/browse/SPARK-18128 Project: Spark Issue Type: Improvement

[jira] [Reopened] (SPARK-1267) Add a pip installer for PySpark

2016-10-26 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk reopened SPARK-1267: re-opening after discussion on mailing list and PR thread. > Add a pip installer for PySpark >

[jira] [Comment Edited] (SPARK-18111) Wrong ApproximatePercentile answer when multiple records have the minimum value

2016-10-26 Thread Zhenhua Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610192#comment-15610192 ] Zhenhua Wang edited comment on SPARK-18111 at 10/27/16 12:46 AM: -

[jira] [Comment Edited] (SPARK-18111) Wrong ApproximatePercentile answer when multiple records have the minimum value

2016-10-26 Thread Zhenhua Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610192#comment-15610192 ] Zhenhua Wang edited comment on SPARK-18111 at 10/27/16 12:46 AM: -

[jira] [Commented] (SPARK-18111) Wrong ApproximatePercentile answer when multiple records have the minimum value

2016-10-26 Thread Zhenhua Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610192#comment-15610192 ] Zhenhua Wang commented on SPARK-18111: -- [~srowen] The minimum is not only skipped once in the whole

[jira] [Updated] (SPARK-16963) Change Source API so that sources do not need to keep unbounded state

2016-10-26 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-16963: - Affects Version/s: 2.0.1 > Change Source API so that sources do not need to keep unbounded state

[jira] [Resolved] (SPARK-16963) Change Source API so that sources do not need to keep unbounded state

2016-10-26 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-16963. -- Resolution: Fixed Assignee: Frederick Reiss Fix Version/s: 2.1.0

[jira] [Commented] (SPARK-10697) Lift Calculation in Association Rule mining

2016-10-26 Thread Shreya Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610163#comment-15610163 ] Shreya Agarwal commented on SPARK-10697: [~yashkumar1992] Hey, so is someone fixing this? >

[jira] [Assigned] (SPARK-16987) Add spark-default.conf property to define https port for spark history server

2016-10-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16987: Assignee: (was: Apache Spark) > Add spark-default.conf property to define https port

[jira] [Assigned] (SPARK-16987) Add spark-default.conf property to define https port for spark history server

2016-10-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16987: Assignee: Apache Spark > Add spark-default.conf property to define https port for spark

[jira] [Commented] (SPARK-16987) Add spark-default.conf property to define https port for spark history server

2016-10-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610150#comment-15610150 ] Apache Spark commented on SPARK-16987: -- User 'hayashidac' has created a pull request for this issue:

[jira] [Commented] (SPARK-17972) Query planning slows down dramatically for large query plans even when sub-trees are cached

2016-10-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610110#comment-15610110 ] Apache Spark commented on SPARK-17972: -- User 'liancheng' has created a pull request for this issue:

[jira] [Commented] (SPARK-18084) write.partitionBy() does not recognize nested columns that select() can access

2016-10-26 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610097#comment-15610097 ] Michael Armbrust commented on SPARK-18084: -- I'd put this in the general class of issues where it

[jira] [Updated] (SPARK-18127) Add hooks and extension points to Spark

2016-10-26 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-18127: - Assignee: Herman van Hovell > Add hooks and extension points to Spark >

[jira] [Created] (SPARK-18127) Add hooks and extension points to Spark

2016-10-26 Thread Srinath (JIRA)
Srinath created SPARK-18127: --- Summary: Add hooks and extension points to Spark Key: SPARK-18127 URL: https://issues.apache.org/jira/browse/SPARK-18127 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-18009) Spark 2.0.1 SQL Thrift Error

2016-10-26 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610061#comment-15610061 ] Xiao Li commented on SPARK-18009: - We try to fix it in 2.0.2. To get the fix, you have to use the new

[jira] [Updated] (SPARK-18009) Spark 2.0.1 SQL Thrift Error

2016-10-26 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-18009: Target Version/s: 2.0.2, 2.1.0 (was: 2.0.1, 2.1.0) > Spark 2.0.1 SQL Thrift Error >

[jira] [Commented] (SPARK-15616) Metastore relation should fallback to HDFS size of partitions that are involved in Query if statistics are not available.

2016-10-26 Thread Franck Tago (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610024#comment-15610024 ] Franck Tago commented on SPARK-15616: - Does spark currently support pruning statistics for hive

[jira] [Commented] (SPARK-17157) Add multiclass logistic regression SparkR Wrapper

2016-10-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610013#comment-15610013 ] Apache Spark commented on SPARK-17157: -- User 'felixcheung' has created a pull request for this

[jira] [Comment Edited] (SPARK-15616) Metastore relation should fallback to HDFS size of partitions that are involved in Query if statistics are not available.

2016-10-26 Thread Franck Tago (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15607872#comment-15607872 ] Franck Tago edited comment on SPARK-15616 at 10/26/16 11:20 PM: Hi

[jira] [Updated] (SPARK-18126) getIteratorZipWithIndex accepts negative value as index.

2016-10-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18126: Issue Type: Improvement (was: Bug) > getIteratorZipWithIndex accepts negative value as index. >

[jira] [Resolved] (SPARK-18126) getIteratorZipWithIndex accepts negative value as index.

2016-10-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18126. - Resolution: Fixed Assignee: Miao Wang Fix Version/s: 2.1.0 >

[jira] [Updated] (SPARK-18126) getIteratorZipWithIndex accepts negative value as index.

2016-10-26 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miao Wang updated SPARK-18126: -- Summary: getIteratorZipWithIndex accepts negative value as index. (was: zipWithIndex accepts negative

[jira] [Commented] (SPARK-18126) zipWithIndex accepts negative value as index.

2016-10-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15609986#comment-15609986 ] Apache Spark commented on SPARK-18126: -- User 'wangmiao1981' has created a pull request for this

[jira] [Assigned] (SPARK-18126) zipWithIndex accepts negative value as index.

2016-10-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18126: Assignee: Apache Spark > zipWithIndex accepts negative value as index. >

[jira] [Assigned] (SPARK-18126) zipWithIndex accepts negative value as index.

2016-10-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18126: Assignee: (was: Apache Spark) > zipWithIndex accepts negative value as index. >

[jira] [Created] (SPARK-18126) zipWithIndex accepts negative value as index.

2016-10-26 Thread Miao Wang (JIRA)
Miao Wang created SPARK-18126: - Summary: zipWithIndex accepts negative value as index. Key: SPARK-18126 URL: https://issues.apache.org/jira/browse/SPARK-18126 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-18123) org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils.saveTable the case senstivity issue

2016-10-26 Thread Paul Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Wu updated SPARK-18123: Description: Blindly quoting every field name for inserting is the issue (Line 110-119,

[jira] [Commented] (SPARK-17829) Stable format for offset log

2016-10-26 Thread Tyson Condie (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15609907#comment-15609907 ] Tyson Condie commented on SPARK-17829: -- Makes sense to me. I'm still wrapping my head around "best

[jira] [Created] (SPARK-18125) Spark generated code causes CompileException when groupByKey, reduceGroups and map(_._2) are used

2016-10-26 Thread Ray Qiu (JIRA)
Ray Qiu created SPARK-18125: --- Summary: Spark generated code causes CompileException when groupByKey, reduceGroups and map(_._2) are used Key: SPARK-18125 URL: https://issues.apache.org/jira/browse/SPARK-18125

[jira] [Assigned] (SPARK-18124) Implement watermarking for handling late data

2016-10-26 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reassigned SPARK-18124: - Assignee: Tathagata Das > Implement watermarking for handling late data >

[jira] [Created] (SPARK-18124) Implement watermarking for handling late data

2016-10-26 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-18124: - Summary: Implement watermarking for handling late data Key: SPARK-18124 URL: https://issues.apache.org/jira/browse/SPARK-18124 Project: Spark Issue Type:

[jira] [Created] (SPARK-18123) org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils.saveTable the case senstivity issue

2016-10-26 Thread Paul Wu (JIRA)
Paul Wu created SPARK-18123: --- Summary: org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils.saveTable the case senstivity issue Key: SPARK-18123 URL: https://issues.apache.org/jira/browse/SPARK-18123

[jira] [Resolved] (SPARK-18094) Move group analytics test cases from `SQLQuerySuite` into a query file test

2016-10-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18094. - Resolution: Fixed Assignee: Jiang Xingbo Fix Version/s: 2.1.0 > Move group

[jira] [Created] (SPARK-18122) Fallback to Kryo for unknown classes in ExpressionEncoder

2016-10-26 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-18122: Summary: Fallback to Kryo for unknown classes in ExpressionEncoder Key: SPARK-18122 URL: https://issues.apache.org/jira/browse/SPARK-18122 Project: Spark

[jira] [Commented] (SPARK-17829) Stable format for offset log

2016-10-26 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15609653#comment-15609653 ] Cody Koeninger commented on SPARK-17829: Have you considered using a typeclass? > Stable format

[jira] [Assigned] (SPARK-18121) Unable to query global temp views when hive support is enabled.

2016-10-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18121: Assignee: (was: Apache Spark) > Unable to query global temp views when hive support

[jira] [Assigned] (SPARK-18121) Unable to query global temp views when hive support is enabled.

2016-10-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18121: Assignee: Apache Spark > Unable to query global temp views when hive support is enabled.

[jira] [Commented] (SPARK-18121) Unable to query global temp views when hive support is enabled.

2016-10-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15609607#comment-15609607 ] Apache Spark commented on SPARK-18121: -- User 'skambha' has created a pull request for this issue:

[jira] [Updated] (SPARK-14300) Scala MLlib examples code merge and clean up

2016-10-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14300: -- Assignee: Xin Ren > Scala MLlib examples code merge and clean up >

[jira] [Resolved] (SPARK-14300) Scala MLlib examples code merge and clean up

2016-10-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-14300. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 12195

[jira] [Updated] (SPARK-18121) Unable to query global temp views when hive support is enabled.

2016-10-26 Thread Sunitha Kambhampati (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunitha Kambhampati updated SPARK-18121: Description: Querying on a global temp view throws Table or view not found

[jira] [Updated] (SPARK-14300) Scala MLlib examples code merge and clean up

2016-10-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14300: -- Description: Duplicated code that I found in scala/examples/mllib: * scala/mllib **

[jira] [Commented] (SPARK-18121) Unable to query global temp views when hive support is enabled.

2016-10-26 Thread Sunitha Kambhampati (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15609579#comment-15609579 ] Sunitha Kambhampati commented on SPARK-18121: - I have a fix for this, will submit a PR soon.

[jira] [Commented] (SPARK-18119) Namenode safemode check is only performed on one namenode which can stuck the startup of SparkHistory server

2016-10-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15609425#comment-15609425 ] Apache Spark commented on SPARK-18119: -- User 'ashangit' has created a pull request for this issue:

[jira] [Created] (SPARK-18121) Unable to query global temp views when hive support is enabled.

2016-10-26 Thread Sunitha Kambhampati (JIRA)
Sunitha Kambhampati created SPARK-18121: --- Summary: Unable to query global temp views when hive support is enabled. Key: SPARK-18121 URL: https://issues.apache.org/jira/browse/SPARK-18121

[jira] [Resolved] (SPARK-17961) Add storageLevel to Dataset for SparkR

2016-10-26 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-17961. -- Resolution: Fixed Assignee: Weichen Xu Fix Version/s: 2.1.0 > Add storageLevel

[jira] [Created] (SPARK-18120) QueryExecutionListener method doesnt' get executed for DataFrameWriter methods

2016-10-26 Thread Salil Surendran (JIRA)
Salil Surendran created SPARK-18120: --- Summary: QueryExecutionListener method doesnt' get executed for DataFrameWriter methods Key: SPARK-18120 URL: https://issues.apache.org/jira/browse/SPARK-18120

[jira] [Updated] (SPARK-13747) Concurrent execution in SQL doesn't work with Scala ForkJoinPool

2016-10-26 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-13747: - Affects Version/s: 2.0.1 > Concurrent execution in SQL doesn't work with Scala ForkJoinPool >

[jira] [Resolved] (SPARK-13747) Concurrent execution in SQL doesn't work with Scala ForkJoinPool

2016-10-26 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-13747. -- Resolution: Fixed Fix Version/s: 2.0.2 > Concurrent execution in SQL doesn't work with

[jira] [Commented] (SPARK-17829) Stable format for offset log

2016-10-26 Thread Tyson Condie (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15609499#comment-15609499 ] Tyson Condie commented on SPARK-17829: -- In the process of adopting a complete JSON serialization

[jira] [Assigned] (SPARK-18119) Namenode safemode check is only performed on one namenode which can stuck the startup of SparkHistory server

2016-10-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18119: Assignee: Apache Spark > Namenode safemode check is only performed on one namenode which

[jira] [Assigned] (SPARK-18119) Namenode safemode check is only performed on one namenode which can stuck the startup of SparkHistory server

2016-10-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18119: Assignee: (was: Apache Spark) > Namenode safemode check is only performed on one

[jira] [Created] (SPARK-18119) Namenode safemode check is only performed on one namenode which can stuck the startup of SparkHistory server

2016-10-26 Thread Nicolas Fraison (JIRA)
Nicolas Fraison created SPARK-18119: --- Summary: Namenode safemode check is only performed on one namenode which can stuck the startup of SparkHistory server Key: SPARK-18119 URL:

[jira] [Updated] (SPARK-18118) SpecificSafeProjection.apply of Java Object from Dataset to JavaRDD Grows Beyond 64 KB

2016-10-26 Thread Aleksander Eskilson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksander Eskilson updated SPARK-18118: Description: For sufficiently wide or nested Java Objects, when

[jira] [Updated] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

2016-10-26 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-18105: --- Description: When lz4 is used to compress the shuffle files, it may fail to decompress it as

[jira] [Commented] (SPARK-18009) Spark 2.0.1 SQL Thrift Error

2016-10-26 Thread Dilip Biswal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15609295#comment-15609295 ] Dilip Biswal commented on SPARK-18009: -- [~martha.solarte] Not sure [~smilegator] Sean, do we back

  1   2   >