[jira] [Commented] (SPARK-18379) Make the parallelism of parallelPartitionDiscovery configurable.

2017-01-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15790962#comment-15790962 ] Sean Owen commented on SPARK-18379: --- I mean that this change does not change the curren

[jira] [Created] (SPARK-19045) irrelevant warning when creating a checkpoint dir

2017-01-01 Thread Assaf Mendelson (JIRA)
Assaf Mendelson created SPARK-19045: --- Summary: irrelevant warning when creating a checkpoint dir Key: SPARK-19045 URL: https://issues.apache.org/jira/browse/SPARK-19045 Project: Spark Issue

[jira] [Created] (SPARK-19046) Dataset checkpoint consumes too much disk space

2017-01-01 Thread Assaf Mendelson (JIRA)
Assaf Mendelson created SPARK-19046: --- Summary: Dataset checkpoint consumes too much disk space Key: SPARK-19046 URL: https://issues.apache.org/jira/browse/SPARK-19046 Project: Spark Issue T

[jira] [Commented] (SPARK-19042) Remove query string from jar url for executor

2017-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15791038#comment-15791038 ] Apache Spark commented on SPARK-19042: -- User 'hustfxj' has created a pull request fo

[jira] [Assigned] (SPARK-19042) Remove query string from jar url for executor

2017-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19042: Assignee: (was: Apache Spark) > Remove query string from jar url for executor > --

[jira] [Assigned] (SPARK-19042) Remove query string from jar url for executor

2017-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19042: Assignee: Apache Spark > Remove query string from jar url for executor > -

[jira] [Assigned] (SPARK-18959) invalid resource statistics for standalone cluster

2017-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18959: Assignee: Apache Spark > invalid resource statistics for standalone cluster >

[jira] [Commented] (SPARK-18959) invalid resource statistics for standalone cluster

2017-01-01 Thread hustfxj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15791056#comment-15791056 ] hustfxj commented on SPARK-18959: - Yes, I have a fix. You can see the link https://githu

[jira] [Commented] (SPARK-18959) invalid resource statistics for standalone cluster

2017-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15791055#comment-15791055 ] Apache Spark commented on SPARK-18959: -- User 'hustfxj' has created a pull request fo

[jira] [Assigned] (SPARK-18959) invalid resource statistics for standalone cluster

2017-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18959: Assignee: (was: Apache Spark) > invalid resource statistics for standalone cluster > -

[jira] [Commented] (SPARK-19046) Dataset checkpoint consumes too much disk space

2017-01-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15791128#comment-15791128 ] Sean Owen commented on SPARK-19046: --- I don't think that's a bug, because you're storing

[jira] [Commented] (SPARK-18997) Recommended upgrade libthrift to 0.9.3

2017-01-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15791139#comment-15791139 ] Sean Owen commented on SPARK-18997: --- Found the reference for this (this should be part

[jira] [Commented] (SPARK-19045) irrelevant warning when creating a checkpoint dir

2017-01-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15791135#comment-15791135 ] Sean Owen commented on SPARK-19045: --- I disagree; the check looks correct: {code} i

[jira] [Created] (SPARK-19047) Invalid correlated column may not be reported as an error

2017-01-01 Thread Nattavut Sutyanyong (JIRA)
Nattavut Sutyanyong created SPARK-19047: --- Summary: Invalid correlated column may not be reported as an error Key: SPARK-19047 URL: https://issues.apache.org/jira/browse/SPARK-19047 Project: Spark

[jira] [Commented] (SPARK-18863) Output non-aggregate expressions without GROUP BY in a subquery does not yield an error

2017-01-01 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15791310#comment-15791310 ] Nattavut Sutyanyong commented on SPARK-18863: - My further investigation concl

[jira] [Updated] (SPARK-19047) Invalid correlated column may not be reported as an error

2017-01-01 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nattavut Sutyanyong updated SPARK-19047: Description: [subquery/in-subquery/in-group-by.sql TC 01.12] {code} Seq((1,1,1)).t

[jira] [Commented] (SPARK-18857) SparkSQL ThriftServer hangs while extracting huge data volumes in incremental collect mode

2017-01-01 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15791397#comment-15791397 ] Dongjoon Hyun commented on SPARK-18857: --- Hi [~vishalagrwal]. Could you test your ca

[jira] [Created] (SPARK-19048) Managed Partitioned Table in InMemoryCatalog: the user specified partition location is not deleted after table dropping

2017-01-01 Thread Xiao Li (JIRA)
Xiao Li created SPARK-19048: --- Summary: Managed Partitioned Table in InMemoryCatalog: the user specified partition location is not deleted after table dropping Key: SPARK-19048 URL: https://issues.apache.org/jira/browse/

[jira] [Commented] (SPARK-19048) Managed Partitioned Table in InMemoryCatalog: the user specified partition location is not deleted after table dropping

2017-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15791418#comment-15791418 ] Apache Spark commented on SPARK-19048: -- User 'gatorsmile' has created a pull request

[jira] [Assigned] (SPARK-19048) Managed Partitioned Table in InMemoryCatalog: the user specified partition location is not deleted after table dropping

2017-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19048: Assignee: Apache Spark > Managed Partitioned Table in InMemoryCatalog: the user specified

[jira] [Assigned] (SPARK-19048) Managed Partitioned Table in InMemoryCatalog: the user specified partition location is not deleted after table dropping

2017-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19048: Assignee: (was: Apache Spark) > Managed Partitioned Table in InMemoryCatalog: the user

[jira] [Updated] (SPARK-19048) Managed Partitioned Table in InMemoryCatalog: the user specified partition location is not deleted after table dropping

2017-01-01 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-19048: Description: The data in the managed table should be deleted after table is dropped. However, if the partit

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2017-01-01 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15791469#comment-15791469 ] Sunil Rangwani commented on SPARK-17463: Hi [~zsxwing] Below is how I am using t

[jira] [Commented] (SPARK-19046) Dataset checkpoint consumes too much disk space

2017-01-01 Thread Assaf Mendelson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15791468#comment-15791468 ] Assaf Mendelson commented on SPARK-19046: - This is an easily created example. I s

[jira] [Commented] (SPARK-19045) irrelevant warning when creating a checkpoint dir

2017-01-01 Thread Assaf Mendelson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15791511#comment-15791511 ] Assaf Mendelson commented on SPARK-19045: - I 100% agree standalone is not local,

[jira] [Created] (SPARK-19049) Failed in `delay in months and years handled correctly`

2017-01-01 Thread Xiao Li (JIRA)
Xiao Li created SPARK-19049: --- Summary: Failed in `delay in months and years handled correctly` Key: SPARK-19049 URL: https://issues.apache.org/jira/browse/SPARK-19049 Project: Spark Issue Type: Tes

[jira] [Updated] (SPARK-19049) Failed in `delay in months and years handled correctly`

2017-01-01 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-19049: Description: The master build failed in the following test cases. It blocks all the PRs. For example: http

[jira] [Commented] (SPARK-19049) Failed in `delay in months and years handled correctly`

2017-01-01 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15791566#comment-15791566 ] Xiao Li commented on SPARK-19049: - cc [~rxin] [~zsxwing] [~tdas] [~marmbrus] > Failed in

[jira] [Created] (SPARK-19050) Fix EventTimeWatermarkSuite 'delay in months and years handled correctly'

2017-01-01 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-19050: Summary: Fix EventTimeWatermarkSuite 'delay in months and years handled correctly' Key: SPARK-19050 URL: https://issues.apache.org/jira/browse/SPARK-19050 Project: Sp

[jira] [Commented] (SPARK-19050) Fix EventTimeWatermarkSuite 'delay in months and years handled correctly'

2017-01-01 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15791581#comment-15791581 ] Shixiong Zhu commented on SPARK-19050: -- monthsSinceEpoch in this test is like math.f

[jira] [Assigned] (SPARK-19050) Fix EventTimeWatermarkSuite 'delay in months and years handled correctly'

2017-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19050: Assignee: Apache Spark (was: Shixiong Zhu) > Fix EventTimeWatermarkSuite 'delay in months

[jira] [Commented] (SPARK-19050) Fix EventTimeWatermarkSuite 'delay in months and years handled correctly'

2017-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15791583#comment-15791583 ] Apache Spark commented on SPARK-19050: -- User 'zsxwing' has created a pull request fo

[jira] [Assigned] (SPARK-19050) Fix EventTimeWatermarkSuite 'delay in months and years handled correctly'

2017-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19050: Assignee: Shixiong Zhu (was: Apache Spark) > Fix EventTimeWatermarkSuite 'delay in months

[jira] [Updated] (SPARK-19050) Fix EventTimeWatermarkSuite 'delay in months and years handled correctly'

2017-01-01 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19050: - Component/s: Structured Streaming > Fix EventTimeWatermarkSuite 'delay in months and years handle

[jira] [Commented] (SPARK-18379) Make the parallelism of parallelPartitionDiscovery configurable.

2017-01-01 Thread Adam Budde (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15791600#comment-15791600 ] Adam Budde commented on SPARK-18379: You're right, the default is still 1. I was

[jira] [Resolved] (SPARK-19049) Failed in `delay in months and years handled correctly`

2017-01-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19049. --- Resolution: Duplicate > Failed in `delay in months and years handled correctly` > ---

[jira] [Resolved] (SPARK-19050) Fix EventTimeWatermarkSuite 'delay in months and years handled correctly'

2017-01-01 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19050. -- Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 > Fix EventTimeWatermark

[jira] [Commented] (SPARK-17204) Spark 2.0 off heap RDD persistence with replication factor 2 leads to in-memory data corruption

2017-01-01 Thread Michael Allman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15791884#comment-15791884 ] Michael Allman commented on SPARK-17204: I'm 99% sure I've fixed this. I'll submi

[jira] [Commented] (SPARK-19038) Can't find keytab file when using Hive catalog

2017-01-01 Thread Peter Parente (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15791995#comment-15791995 ] Peter Parente commented on SPARK-19038: --- Also, since the keytab file name in the st

[jira] [Commented] (SPARK-18857) SparkSQL ThriftServer hangs while extracting huge data volumes in incremental collect mode

2017-01-01 Thread vishal agrawal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15792152#comment-15792152 ] vishal agrawal commented on SPARK-18857: thanks. we will test it and confirm. >

[jira] [Created] (SPARK-19051) test_hivecontext (pyspark.sql.tests.HiveSparkSubmitTests) fails in python/run-tests

2017-01-01 Thread Nirman Narang (JIRA)
Nirman Narang created SPARK-19051: - Summary: test_hivecontext (pyspark.sql.tests.HiveSparkSubmitTests) fails in python/run-tests Key: SPARK-19051 URL: https://issues.apache.org/jira/browse/SPARK-19051