[jira] [Comment Edited] (SPARK-21725) spark thriftserver insert overwrite table partition select

2017-10-31 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233578#comment-16233578 ] xinzhang edited comment on SPARK-21725 at 11/1/17 5:25 AM: --- [~mgaido] 1. hive

[jira] [Comment Edited] (SPARK-21725) spark thriftserver insert overwrite table partition select

2017-10-31 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233578#comment-16233578 ] xinzhang edited comment on SPARK-21725 at 11/1/17 3:51 AM: --- [~mgaido] 1. hive

[jira] [Comment Edited] (SPARK-21725) spark thriftserver insert overwrite table partition select

2017-10-31 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233578#comment-16233578 ] xinzhang edited comment on SPARK-21725 at 11/1/17 3:49 AM: --- [~mgaido] 1. hive

[jira] [Comment Edited] (SPARK-21725) spark thriftserver insert overwrite table partition select

2017-10-31 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233578#comment-16233578 ] xinzhang edited comment on SPARK-21725 at 11/1/17 3:48 AM: --- [~mgaido] 1. hive

[jira] [Comment Edited] (SPARK-21725) spark thriftserver insert overwrite table partition select

2017-10-31 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233578#comment-16233578 ] xinzhang edited comment on SPARK-21725 at 11/1/17 3:46 AM: --- [~mgaido] 1. hive

[jira] [Comment Edited] (SPARK-21725) spark thriftserver insert overwrite table partition select

2017-10-31 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233578#comment-16233578 ] xinzhang edited comment on SPARK-21725 at 11/1/17 3:16 AM: --- [~mgaido] 1. hive

[jira] [Comment Edited] (SPARK-21725) spark thriftserver insert overwrite table partition select

2017-10-31 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233578#comment-16233578 ] xinzhang edited comment on SPARK-21725 at 11/1/17 3:08 AM: --- [~mgaido] 1. hive

[jira] [Comment Edited] (SPARK-21725) spark thriftserver insert overwrite table partition select

2017-10-31 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233578#comment-16233578 ] xinzhang edited comment on SPARK-21725 at 11/1/17 2:59 AM: --- [~mgaido] 1. hive

[jira] [Comment Edited] (SPARK-21725) spark thriftserver insert overwrite table partition select

2017-10-31 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233578#comment-16233578 ] xinzhang edited comment on SPARK-21725 at 11/1/17 2:52 AM: --- [~mgaido] 1. hive

[jira] [Comment Edited] (SPARK-21827) Task fail due to executor exception when enable Sasl Encryption

2017-10-31 Thread Mario Molina (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233587#comment-16233587 ] Mario Molina edited comment on SPARK-21827 at 11/1/17 2:50 AM: --- Are you

[jira] [Comment Edited] (SPARK-21827) Task fail due to executor exception when enable Sasl Encryption

2017-10-31 Thread Mario Molina (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233587#comment-16233587 ] Mario Molina edited comment on SPARK-21827 at 11/1/17 2:49 AM: --- Are you

[jira] [Commented] (SPARK-21827) Task fail due to executor exception when enable Sasl Encryption

2017-10-31 Thread Mario Molina (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233587#comment-16233587 ] Mario Molina commented on SPARK-21827: -- Are you trying to read/write data to some DB or HDFS or

[jira] [Comment Edited] (SPARK-21725) spark thriftserver insert overwrite table partition select

2017-10-31 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233578#comment-16233578 ] xinzhang edited comment on SPARK-21725 at 11/1/17 2:38 AM: --- [~mgaido] 1. hive

[jira] [Comment Edited] (SPARK-21725) spark thriftserver insert overwrite table partition select

2017-10-31 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233578#comment-16233578 ] xinzhang edited comment on SPARK-21725 at 11/1/17 2:37 AM: --- [~mgaido] 1. hive

[jira] [Comment Edited] (SPARK-21725) spark thriftserver insert overwrite table partition select

2017-10-31 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233578#comment-16233578 ] xinzhang edited comment on SPARK-21725 at 11/1/17 2:32 AM: --- [~mgaido] 1. hive

[jira] [Comment Edited] (SPARK-21725) spark thriftserver insert overwrite table partition select

2017-10-31 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233578#comment-16233578 ] xinzhang edited comment on SPARK-21725 at 11/1/17 2:31 AM: --- [~mgaido] 1. hive

[jira] [Comment Edited] (SPARK-21725) spark thriftserver insert overwrite table partition select

2017-10-31 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233578#comment-16233578 ] xinzhang edited comment on SPARK-21725 at 11/1/17 2:30 AM: --- [~mgaido] 1. hive

[jira] [Commented] (SPARK-21725) spark thriftserver insert overwrite table partition select

2017-10-31 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233578#comment-16233578 ] xinzhang commented on SPARK-21725: -- 1. hive 1.2.1 download a new tar only change hive-site.xml

[jira] [Created] (SPARK-22406) pyspark version tag is wrong on PyPi

2017-10-31 Thread Kerrick Staley (JIRA)
Kerrick Staley created SPARK-22406: -- Summary: pyspark version tag is wrong on PyPi Key: SPARK-22406 URL: https://issues.apache.org/jira/browse/SPARK-22406 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-22405) Enrich the event information and add new event of ExternalCatalogEvent

2017-10-31 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233569#comment-16233569 ] Saisai Shao commented on SPARK-22405: - Thanks [~hvanhovell] for your comments. bq. When implementing

[jira] [Commented] (SPARK-2984) FileNotFoundException on _temporary directory

2017-10-31 Thread Darron Fuller (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233524#comment-16233524 ] Darron Fuller commented on SPARK-2984: -- Similar issue as well. Not related to S3 or HDFS as I am

[jira] [Reopened] (SPARK-22243) streaming job failed to restart from checkpoint

2017-10-31 Thread StephenZou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] StephenZou reopened SPARK-22243: reopen this issue need to merge. > streaming job failed to restart from checkpoint >

[jira] [Resolved] (SPARK-22242) streaming job failed to restart from checkpoint

2017-10-31 Thread StephenZou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] StephenZou resolved SPARK-22242. Resolution: Duplicate > streaming job failed to restart from checkpoint >

[jira] [Updated] (SPARK-21930) When the number of attempting to restart receiver greater than 0,spark do nothing in 'else'

2017-10-31 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-21930: - Component/s: (was: Structured Streaming) DStreams > When the number of

[jira] [Updated] (SPARK-17556) Executor side broadcast for broadcast joins

2017-10-31 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-17556: Target Version/s: (was: 2.3.0) > Executor side broadcast for broadcast joins >

[jira] [Commented] (SPARK-22315) Check for version match between R package and JVM

2017-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16227450#comment-16227450 ] Apache Spark commented on SPARK-22315: -- User 'shivaram' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22315) Check for version match between R package and JVM

2017-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22315: Assignee: (was: Apache Spark) > Check for version match between R package and JVM >

[jira] [Assigned] (SPARK-22315) Check for version match between R package and JVM

2017-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22315: Assignee: Apache Spark > Check for version match between R package and JVM >

[jira] [Resolved] (SPARK-22305) HDFSBackedStateStoreProvider fails with StackOverflowException when attempting to recover state

2017-10-31 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-22305. -- Resolution: Fixed Fix Version/s: 2.3.0 > HDFSBackedStateStoreProvider fails with

[jira] [Assigned] (SPARK-22305) HDFSBackedStateStoreProvider fails with StackOverflowException when attempting to recover state

2017-10-31 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-22305: Assignee: Jose Torres > HDFSBackedStateStoreProvider fails with StackOverflowException

[jira] [Commented] (SPARK-22403) StructuredKafkaWordCount example fails in YARN cluster mode

2017-10-31 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16227321#comment-16227321 ] Shixiong Zhu commented on SPARK-22403: -- Yeah, feel free to submit a PR to improve the example. >

[jira] [Commented] (SPARK-22403) StructuredKafkaWordCount example fails in YARN cluster mode

2017-10-31 Thread Wing Yew Poon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16227298#comment-16227298 ] Wing Yew Poon commented on SPARK-22403: --- I realize that in a production application, one would set

[jira] [Commented] (SPARK-22403) StructuredKafkaWordCount example fails in YARN cluster mode

2017-10-31 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16227266#comment-16227266 ] Shixiong Zhu commented on SPARK-22403: -- Yeah, Spark creates a temp directory for you. You can set

[jira] [Updated] (SPARK-22333) ColumnReference should get higher priority than timeFunctionCall(CURRENT_DATE, CURRENT_TIMESTAMP)

2017-10-31 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-22333: Fix Version/s: 2.2.1 > ColumnReference should get higher priority than > timeFunctionCall(CURRENT_DATE,

[jira] [Commented] (SPARK-21997) Spark shows different results on char/varchar columns on Parquet

2017-10-31 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16227154#comment-16227154 ] Dongjoon Hyun commented on SPARK-21997: --- Thank you, [~cloud_fan]. Yes, I concur. > Spark shows

[jira] [Closed] (SPARK-21125) PySpark context missing function to set Job Description.

2017-10-31 Thread Shane Jarvie (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shane Jarvie closed SPARK-21125. This has been patched and will be live in Spark 2.3.0 > PySpark context missing function to set Job

[jira] [Resolved] (SPARK-11334) numRunningTasks can't be less than 0, or it will affect executor allocation

2017-10-31 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-11334. Resolution: Fixed Assignee: Sital Kedia (was: meiyoula) Target

[jira] [Commented] (SPARK-13030) Change OneHotEncoder to Estimator

2017-10-31 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16227094#comment-16227094 ] yuhao yang commented on SPARK-13030: I see. Thanks for the response [~mlnick]. The Estimator is

[jira] [Commented] (SPARK-22405) Enrich the event information and add new event of ExternalCatalogEvent

2017-10-31 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16227039#comment-16227039 ] Herman van Hovell commented on SPARK-22405: --- For some context. When implementing this we just

[jira] [Updated] (SPARK-22403) StructuredKafkaWordCount example fails in YARN cluster mode

2017-10-31 Thread Wing Yew Poon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wing Yew Poon updated SPARK-22403: -- Description: When I run the StructuredKafkaWordCount example in YARN client mode, it runs

[jira] [Updated] (SPARK-22403) StructuredKafkaWordCount example fails in YARN cluster mode

2017-10-31 Thread Wing Yew Poon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wing Yew Poon updated SPARK-22403: -- Description: When I run the StructuredKafkaWordCount example in YARN client mode, it runs

[jira] [Commented] (SPARK-22078) clarify exception behaviors for all data source v2 interfaces

2017-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16227024#comment-16227024 ] Apache Spark commented on SPARK-22078: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22078) clarify exception behaviors for all data source v2 interfaces

2017-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22078: Assignee: Apache Spark > clarify exception behaviors for all data source v2 interfaces >

[jira] [Assigned] (SPARK-22078) clarify exception behaviors for all data source v2 interfaces

2017-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22078: Assignee: (was: Apache Spark) > clarify exception behaviors for all data source v2

[jira] [Commented] (SPARK-14540) Support Scala 2.12 closures and Java 8 lambdas in ClosureCleaner

2017-10-31 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226995#comment-16226995 ] Sean Owen commented on SPARK-14540: --- [~joshrosen] was right that this is actually the hard part. A few

[jira] [Commented] (SPARK-22398) Partition directories with leading 0s cause wrong results

2017-10-31 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226991#comment-16226991 ] Marco Gaido commented on SPARK-22398: - you just need to set

[jira] [Commented] (SPARK-21997) Spark shows different results on char/varchar columns on Parquet

2017-10-31 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226957#comment-16226957 ] Wenchen Fan commented on SPARK-21997: - A better fix is to add special handling of varchar type in the

[jira] [Commented] (SPARK-21997) Spark shows different results on char/varchar columns on Parquet

2017-10-31 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226953#comment-16226953 ] Wenchen Fan commented on SPARK-21997: - I think a simple fix is to disable `convertMetastoreParquet`

[jira] [Assigned] (SPARK-22306) INFER_AND_SAVE overwrites important metadata in Parquet Metastore table

2017-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22306: Assignee: Apache Spark > INFER_AND_SAVE overwrites important metadata in Parquet

[jira] [Assigned] (SPARK-22306) INFER_AND_SAVE overwrites important metadata in Parquet Metastore table

2017-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22306: Assignee: (was: Apache Spark) > INFER_AND_SAVE overwrites important metadata in

[jira] [Commented] (SPARK-22306) INFER_AND_SAVE overwrites important metadata in Parquet Metastore table

2017-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226944#comment-16226944 ] Apache Spark commented on SPARK-22306: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-11215) Add multiple columns support to StringIndexer

2017-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226915#comment-16226915 ] Apache Spark commented on SPARK-11215: -- User 'WeichenXu123' has created a pull request for this

[jira] [Commented] (SPARK-22405) Enrich the event information and add new event of ExternalCatalogEvent

2017-10-31 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226724#comment-16226724 ] Saisai Shao commented on SPARK-22405: - Thanks [~cloud_fan] for your comments. Using fake

[jira] [Commented] (SPARK-21725) spark thriftserver insert overwrite table partition select

2017-10-31 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226720#comment-16226720 ] Marco Gaido commented on SPARK-21725: - [~zhangxin0112zx] I am sorry but I am still unable to

[jira] [Commented] (SPARK-22306) INFER_AND_SAVE overwrites important metadata in Parquet Metastore table

2017-10-31 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226640#comment-16226640 ] Wenchen Fan commented on SPARK-22306: - This is a known issue before Spark 2.3: ALTER TABLE at Spark

[jira] [Issue Comment Deleted] (SPARK-19039) UDF ClosureCleaner bug when UDF, col applied in paste mode in REPL

2017-10-31 Thread Jen-Ming Chung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jen-Ming Chung updated SPARK-19039: --- Comment: was deleted (was: It's weird..you will not get error messages if you paste the code

[jira] [Commented] (SPARK-22405) Enrich the event information and add new event of ExternalCatalogEvent

2017-10-31 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226551#comment-16226551 ] Wenchen Fan commented on SPARK-22405: - If you really want very detailed information about metadata

[jira] [Commented] (SPARK-19628) Duplicate Spark jobs in 2.1.0

2017-10-31 Thread Jork Zijlstra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226559#comment-16226559 ] Jork Zijlstra commented on SPARK-19628: --- Hello [~guilhermeslucas], I'm currently no longer

[jira] [Resolved] (SPARK-22310) Refactor join estimation to incorporate estimation logic for different kinds of statistics

2017-10-31 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-22310. - Resolution: Fixed Fix Version/s: 2.3.0 > Refactor join estimation to incorporate

[jira] [Assigned] (SPARK-22310) Refactor join estimation to incorporate estimation logic for different kinds of statistics

2017-10-31 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-22310: --- Assignee: Zhenhua Wang > Refactor join estimation to incorporate estimation logic for

[jira] [Commented] (SPARK-19039) UDF ClosureCleaner bug when UDF, col applied in paste mode in REPL

2017-10-31 Thread Jen-Ming Chung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226528#comment-16226528 ] Jen-Ming Chung commented on SPARK-19039: It's weird..you will not get error messages if you paste

[jira] [Commented] (SPARK-22405) Enrich the event information and add new event of ExternalCatalogEvent

2017-10-31 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226474#comment-16226474 ] Saisai Shao commented on SPARK-22405: - CC [~smilegator], what's your opinion about this proposal?

[jira] [Commented] (SPARK-22405) Enrich the event information and add new event of ExternalCatalogEvent

2017-10-31 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226468#comment-16226468 ] Saisai Shao commented on SPARK-22405: - This is the WIP branch

[jira] [Comment Edited] (SPARK-21918) HiveClient shouldn't share Hive object between different thread

2017-10-31 Thread junzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226457#comment-16226457 ] junzhang edited comment on SPARK-21918 at 10/31/17 8:36 AM: [~huLiu] how

[jira] [Commented] (SPARK-21918) HiveClient shouldn't share Hive object between different thread

2017-10-31 Thread junzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226457#comment-16226457 ] junzhang commented on SPARK-21918: -- [~huLiu] how about the patch? we are suffering DDL and DML problems

[jira] [Created] (SPARK-22405) Enrich the event information of ExternalCatalogEvent

2017-10-31 Thread Saisai Shao (JIRA)
Saisai Shao created SPARK-22405: --- Summary: Enrich the event information of ExternalCatalogEvent Key: SPARK-22405 URL: https://issues.apache.org/jira/browse/SPARK-22405 Project: Spark Issue

[jira] [Updated] (SPARK-22405) Enrich the event information and add new event of ExternalCatalogEvent

2017-10-31 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-22405: Summary: Enrich the event information and add new event of ExternalCatalogEvent (was: Enrich the

[jira] [Commented] (SPARK-13030) Change OneHotEncoder to Estimator

2017-10-31 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226448#comment-16226448 ] Nick Pentreath commented on SPARK-13030: I just think it makes sense for OHE to be an Estimator

[jira] [Commented] (SPARK-22401) Missing 2.1.2 tag in git

2017-10-31 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226445#comment-16226445 ] Sean Owen commented on SPARK-22401: --- Normally the release plugin would tag and push the tag. I think

[jira] [Assigned] (SPARK-22399) reference in mllib-clustering.html is out of date

2017-10-31 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-22399: - Assignee: Bo Meng > reference in mllib-clustering.html is out of date >

[jira] [Resolved] (SPARK-22399) reference in mllib-clustering.html is out of date

2017-10-31 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22399. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19614

[jira] [Updated] (SPARK-20077) Documentation for ml.stats.Correlation

2017-10-31 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-20077: -- Priority: Minor (was: Major) I don't see any docs at

[jira] [Comment Edited] (SPARK-21725) spark thriftserver insert overwrite table partition select

2017-10-31 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226337#comment-16226337 ] xinzhang edited comment on SPARK-21725 at 10/31/17 7:09 AM: [~mgaido]

[jira] [Comment Edited] (SPARK-21725) spark thriftserver insert overwrite table partition select

2017-10-31 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226337#comment-16226337 ] xinzhang edited comment on SPARK-21725 at 10/31/17 7:07 AM: [~mgaido]

[jira] [Comment Edited] (SPARK-21725) spark thriftserver insert overwrite table partition select

2017-10-31 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226337#comment-16226337 ] xinzhang edited comment on SPARK-21725 at 10/31/17 7:07 AM: [~mgaido]

[jira] [Comment Edited] (SPARK-21725) spark thriftserver insert overwrite table partition select

2017-10-31 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226337#comment-16226337 ] xinzhang edited comment on SPARK-21725 at 10/31/17 7:03 AM: [~mgaido]

[jira] [Comment Edited] (SPARK-21725) spark thriftserver insert overwrite table partition select

2017-10-31 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226337#comment-16226337 ] xinzhang edited comment on SPARK-21725 at 10/31/17 7:01 AM: [~mgaido]

[jira] [Comment Edited] (SPARK-21725) spark thriftserver insert overwrite table partition select

2017-10-31 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226337#comment-16226337 ] xinzhang edited comment on SPARK-21725 at 10/31/17 6:55 AM: [~mgaido]

[jira] [Comment Edited] (SPARK-21725) spark thriftserver insert overwrite table partition select

2017-10-31 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226337#comment-16226337 ] xinzhang edited comment on SPARK-21725 at 10/31/17 6:43 AM: [~mgaido]

[jira] [Commented] (SPARK-21725) spark thriftserver insert overwrite table partition select

2017-10-31 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226337#comment-16226337 ] xinzhang commented on SPARK-21725: -- Now I try with the master branch. The problem is still here. Steps:

[jira] [Commented] (SPARK-13030) Change OneHotEncoder to Estimator

2017-10-31 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226307#comment-16226307 ] yuhao yang commented on SPARK-13030: Sorry to jumping in so late. I can see there's been a lot of