[jira] [Assigned] (SPARK-22644) Make ML testsuite support StructuredStreaming test

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22644: Assignee: (was: Apache Spark) > Make ML testsuite support StructuredStreaming test > -

[jira] [Assigned] (SPARK-22644) Make ML testsuite support StructuredStreaming test

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22644: Assignee: Apache Spark > Make ML testsuite support StructuredStreaming test >

[jira] [Commented] (SPARK-22644) Make ML testsuite support StructuredStreaming test

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16270325#comment-16270325 ] Apache Spark commented on SPARK-22644: -- User 'WeichenXu123' has created a pull reque

[jira] [Created] (SPARK-22644) Make ML testsuite support StructuredStreaming test

2017-11-28 Thread Weichen Xu (JIRA)
Weichen Xu created SPARK-22644: -- Summary: Make ML testsuite support StructuredStreaming test Key: SPARK-22644 URL: https://issues.apache.org/jira/browse/SPARK-22644 Project: Spark Issue Type: Te

[jira] [Commented] (SPARK-22643) ColumnarArray should be an immutable view

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16270307#comment-16270307 ] Apache Spark commented on SPARK-22643: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-22643) ColumnarArray should be an immutable view

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22643: Assignee: Apache Spark (was: Wenchen Fan) > ColumnarArray should be an immutable view > -

[jira] [Assigned] (SPARK-22643) ColumnarArray should be an immutable view

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22643: Assignee: Wenchen Fan (was: Apache Spark) > ColumnarArray should be an immutable view > -

[jira] [Created] (SPARK-22643) ColumnarArray should be an immutable view

2017-11-28 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-22643: --- Summary: ColumnarArray should be an immutable view Key: SPARK-22643 URL: https://issues.apache.org/jira/browse/SPARK-22643 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-22472) Datasets generate random values for null primitive types

2017-11-28 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16270259#comment-16270259 ] Wenchen Fan commented on SPARK-22472: - yea I think we should, this may turn a runnabl

[jira] [Assigned] (SPARK-22642) the createdTempDir will not be deleted if an exception occurs

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22642: Assignee: (was: Apache Spark) > the createdTempDir will not be deleted if an exception

[jira] [Commented] (SPARK-22642) the createdTempDir will not be deleted if an exception occurs

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16270228#comment-16270228 ] Apache Spark commented on SPARK-22642: -- User 'zuotingbing' has created a pull reques

[jira] [Assigned] (SPARK-22642) the createdTempDir will not be deleted if an exception occurs

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22642: Assignee: Apache Spark > the createdTempDir will not be deleted if an exception occurs > -

[jira] [Created] (SPARK-22642) the createdTempDir will not be deleted if an exception occurs

2017-11-28 Thread zuotingbing (JIRA)
zuotingbing created SPARK-22642: --- Summary: the createdTempDir will not be deleted if an exception occurs Key: SPARK-22642 URL: https://issues.apache.org/jira/browse/SPARK-22642 Project: Spark

[jira] [Commented] (SPARK-22640) Can not switch python exec in executor side

2017-11-28 Thread Kent Yao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16270208#comment-16270208 ] Kent Yao commented on SPARK-22640: -- [~Jeff_Z] spark.yarn.appMasterEnv specified > Can n

[jira] [Updated] (SPARK-22637) CatalogImpl.refresh() has quadratic complexity for a view

2017-11-28 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22637: - Target Version/s: 2.2.2, 2.3.0 (was: 2.3.0) > CatalogImpl.refresh() has quadratic complexity for

[jira] [Commented] (SPARK-22472) Datasets generate random values for null primitive types

2017-11-28 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16270178#comment-16270178 ] Felix Cheung commented on SPARK-22472: -- This is going out in 2.2.1 - do we need a re

[jira] [Commented] (SPARK-22627) Fix formatting of headers in configuration.html page

2017-11-28 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16270169#comment-16270169 ] Felix Cheung commented on SPARK-22627: -- quite possibly this is related to jekyll cha

[jira] [Commented] (SPARK-22640) Can not switch python exec in executor side

2017-11-28 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16270139#comment-16270139 ] Jeff Zhang commented on SPARK-22640: you need to use spark.yarn.appMasterEnv since yo

[jira] [Commented] (SPARK-22641) Pyspark UDF relying on column added with withColumn after distinct

2017-11-28 Thread Andrew Duffy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16270132#comment-16270132 ] Andrew Duffy commented on SPARK-22641: -- Query plan with the literal: {code} == Pars

[jira] [Commented] (SPARK-22641) Pyspark UDF relying on column added with withColumn after distinct

2017-11-28 Thread Andrew Duffy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16270128#comment-16270128 ] Andrew Duffy commented on SPARK-22641: -- So it seems this is only a problem when usin

[jira] [Comment Edited] (SPARK-22641) Pyspark UDF relying on column added with withColumn after distinct

2017-11-28 Thread Andrew Duffy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16270128#comment-16270128 ] Andrew Duffy edited comment on SPARK-22641 at 11/29/17 4:44 AM: ---

[jira] [Updated] (SPARK-22641) Pyspark UDF relying on column added with withColumn after distinct

2017-11-28 Thread Andrew Duffy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Duffy updated SPARK-22641: - Description: We seem to have found an issue with PySpark UDFs interacting with {{withColumn}} wh

[jira] [Updated] (SPARK-22641) Pyspark UDF relying on column added with withColumn after distinct

2017-11-28 Thread Andrew Duffy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Duffy updated SPARK-22641: - Description: We seem to have found an issue with PySpark UDFs interacting with {{withColumn}} wh

[jira] [Updated] (SPARK-22641) Pyspark UDF relying on column added with withColumn after distinct

2017-11-28 Thread Andrew Duffy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Duffy updated SPARK-22641: - Description: We seem to have found an issue with PySpark UDFs interacting with {{withColumn}} wh

[jira] [Updated] (SPARK-22641) Pyspark UDF relying on column added with withColumn after distinct

2017-11-28 Thread Andrew Duffy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Duffy updated SPARK-22641: - Description: We seem to have found an issue with PySpark UDFs interacting with {{withColumn}} wh

[jira] [Created] (SPARK-22641) Pyspark UDF relying on column added with withColumn after distinct

2017-11-28 Thread Andrew Duffy (JIRA)
Andrew Duffy created SPARK-22641: Summary: Pyspark UDF relying on column added with withColumn after distinct Key: SPARK-22641 URL: https://issues.apache.org/jira/browse/SPARK-22641 Project: Spark

[jira] [Assigned] (SPARK-22640) Can not switch python exec in executor side

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22640: Assignee: Apache Spark > Can not switch python exec in executor side > ---

[jira] [Commented] (SPARK-22640) Can not switch python exec in executor side

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16270067#comment-16270067 ] Apache Spark commented on SPARK-22640: -- User 'yaooqinn' has created a pull request f

[jira] [Assigned] (SPARK-22640) Can not switch python exec in executor side

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22640: Assignee: (was: Apache Spark) > Can not switch python exec in executor side >

[jira] [Created] (SPARK-22640) Can not switch python exec in executor side

2017-11-28 Thread Kent Yao (JIRA)
Kent Yao created SPARK-22640: Summary: Can not switch python exec in executor side Key: SPARK-22640 URL: https://issues.apache.org/jira/browse/SPARK-22640 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-22373) Intermittent NullPointerException in org.codehaus.janino.IClass.isAssignableFrom

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22373: Assignee: (was: Apache Spark) > Intermittent NullPointerException in > org.codehaus.j

[jira] [Assigned] (SPARK-22373) Intermittent NullPointerException in org.codehaus.janino.IClass.isAssignableFrom

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22373: Assignee: Apache Spark > Intermittent NullPointerException in > org.codehaus.janino.IClas

[jira] [Commented] (SPARK-22373) Intermittent NullPointerException in org.codehaus.janino.IClass.isAssignableFrom

2017-11-28 Thread Min Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16270055#comment-16270055 ] Min Shen commented on SPARK-22373: -- Created PR https://github.com/apache/spark/pull/1983

[jira] [Commented] (SPARK-22373) Intermittent NullPointerException in org.codehaus.janino.IClass.isAssignableFrom

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16270053#comment-16270053 ] Apache Spark commented on SPARK-22373: -- User 'Victsm' has created a pull request for

[jira] [Commented] (SPARK-21918) HiveClient shouldn't share Hive object between different thread

2017-11-28 Thread Reid Chan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16270040#comment-16270040 ] Reid Chan commented on SPARK-21918: --- Expecting > HiveClient shouldn't share Hive objec

[jira] [Updated] (SPARK-22373) Intermittent NullPointerException in org.codehaus.janino.IClass.isAssignableFrom

2017-11-28 Thread Min Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Shen updated SPARK-22373: - Attachment: generated.java CodeGeneratorTester.scala Attach the standalone testing applic

[jira] [Commented] (SPARK-19250) In security cluster, spark beeline connect to hive metastore failed

2017-11-28 Thread Reid Chan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16270005#comment-16270005 ] Reid Chan commented on SPARK-19250: --- Hi [~meiyoula], may i ask how do you solve it? >

[jira] [Comment Edited] (SPARK-22622) OutOfMemory thrown by Closure Serializer without proper failure propagation

2017-11-28 Thread Raghavendra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269974#comment-16269974 ] Raghavendra edited comment on SPARK-22622 at 11/29/17 2:55 AM:

[jira] [Comment Edited] (SPARK-22622) OutOfMemory thrown by Closure Serializer without proper failure propagation

2017-11-28 Thread Raghavendra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269974#comment-16269974 ] Raghavendra edited comment on SPARK-22622 at 11/29/17 2:37 AM:

[jira] [Commented] (SPARK-22622) OutOfMemory thrown by Closure Serializer without proper failure propagation

2017-11-28 Thread Raghavendra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269974#comment-16269974 ] Raghavendra commented on SPARK-22622: - [~srowen] I am broadcasting all my large data

[jira] [Comment Edited] (SPARK-22622) OutOfMemory thrown by Closure Serializer without proper failure propagation

2017-11-28 Thread Raghavendra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269974#comment-16269974 ] Raghavendra edited comment on SPARK-22622 at 11/29/17 2:36 AM:

[jira] [Commented] (SPARK-22622) OutOfMemory thrown by Closure Serializer without proper failure propagation

2017-11-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269935#comment-16269935 ] Sean Owen commented on SPARK-22622: --- The error isn't out of memory. It's that an array

[jira] [Commented] (SPARK-22373) Intermittent NullPointerException in org.codehaus.janino.IClass.isAssignableFrom

2017-11-28 Thread Min Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269924#comment-16269924 ] Min Shen commented on SPARK-22373: -- [~leigjklotz], I think bumping up Janino version to

[jira] [Commented] (SPARK-22622) OutOfMemory thrown by Closure Serializer without proper failure propagation

2017-11-28 Thread Raghavendra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269917#comment-16269917 ] Raghavendra commented on SPARK-22622: - The output of previous Stage is around 70GB an

[jira] [Comment Edited] (SPARK-22622) OutOfMemory thrown by Closure Serializer without proper failure propagation

2017-11-28 Thread Raghavendra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269917#comment-16269917 ] Raghavendra edited comment on SPARK-22622 at 11/29/17 1:50 AM:

[jira] [Issue Comment Deleted] (SPARK-22622) OutOfMemory thrown by Closure Serializer without proper failure propagation

2017-11-28 Thread Raghavendra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghavendra updated SPARK-22622: Comment: was deleted (was: The output of previous Stage is around 70GB and my driver memory is 225

[jira] [Updated] (SPARK-22622) OutOfMemory thrown by Closure Serializer without proper failure propagation

2017-11-28 Thread Raghavendra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghavendra updated SPARK-22622: Summary: OutOfMemory thrown by Closure Serializer without proper failure propagation (was: OutOfMe

[jira] [Updated] (SPARK-22622) OutOfMemory thrown by Closure Serializer without proper failure propagation

2017-11-28 Thread Raghavendra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghavendra updated SPARK-22622: Description: While moving from a Stage to another, the Closure serializer is trying to Serialize t

[jira] [Updated] (SPARK-22622) OutOfMemory thrown by Closure Serializer

2017-11-28 Thread Raghavendra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghavendra updated SPARK-22622: Description: While moving from a Stage to another, the Closure serializer is trying to Serialize t

[jira] [Comment Edited] (SPARK-22622) OutOfMemory thrown by Closure Serializer

2017-11-28 Thread Raghavendra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269894#comment-16269894 ] Raghavendra edited comment on SPARK-22622 at 11/29/17 1:39 AM:

[jira] [Reopened] (SPARK-22622) OutOfMemory thrown by Closure Serializer

2017-11-28 Thread Raghavendra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghavendra reopened SPARK-22622: - The output of previous Stage is around 70GB and my driver memory is 225GB. Why am i getting an OOM w

[jira] [Issue Comment Deleted] (SPARK-22622) OutOfMemory thrown by Closure Serializer

2017-11-28 Thread Raghavendra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghavendra updated SPARK-22622: Comment: was deleted (was: The output of previous Stage is around 70GB and my driver memory is 225

[jira] [Commented] (SPARK-22622) OutOfMemory thrown by Closure Serializer

2017-11-28 Thread Raghavendra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269894#comment-16269894 ] Raghavendra commented on SPARK-22622: - The output of previous Stage is around 70GB an

[jira] [Created] (SPARK-22639) no rowcount estimation returned if groupby clause involves substring

2017-11-28 Thread ey-chih chow (JIRA)
ey-chih chow created SPARK-22639: Summary: no rowcount estimation returned if groupby clause involves substring Key: SPARK-22639 URL: https://issues.apache.org/jira/browse/SPARK-22639 Project: Spark

[jira] [Resolved] (SPARK-22637) CatalogImpl.refresh() has quadratic complexity for a view

2017-11-28 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-22637. - Resolution: Fixed Assignee: Herman van Hovell Fix Version/s: 2.3.0 2.2.

[jira] [Commented] (SPARK-19552) Upgrade Netty version to 4.1.x final

2017-11-28 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269711#comment-16269711 ] Shixiong Zhu commented on SPARK-19552: -- [~wesmckinn] I will try. Looks like not a lo

[jira] [Updated] (SPARK-22636) row count not being set correctly (always 0) after Statement.executeUpdate()

2017-11-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-22636: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) That doesn't look like a bug, a

[jira] [Assigned] (SPARK-22638) Use a separate query for StreamingQueryListenerBus

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22638: Assignee: Apache Spark (was: Shixiong Zhu) > Use a separate query for StreamingQueryListe

[jira] [Assigned] (SPARK-22638) Use a separate query for StreamingQueryListenerBus

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22638: Assignee: Shixiong Zhu (was: Apache Spark) > Use a separate query for StreamingQueryListe

[jira] [Commented] (SPARK-22638) Use a separate query for StreamingQueryListenerBus

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269557#comment-16269557 ] Apache Spark commented on SPARK-22638: -- User 'zsxwing' has created a pull request fo

[jira] [Created] (SPARK-22638) Use a separate query for StreamingQueryListenerBus

2017-11-28 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-22638: Summary: Use a separate query for StreamingQueryListenerBus Key: SPARK-22638 URL: https://issues.apache.org/jira/browse/SPARK-22638 Project: Spark Issue Type

[jira] [Commented] (SPARK-22620) Deadlock in blacklisting when executor is removed due to a heartbeat timeout

2017-11-28 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269517#comment-16269517 ] Thomas Graves commented on SPARK-22620: --- Looking at this some more it seems that no

[jira] [Commented] (SPARK-22637) CatalogImpl.refresh() has quadratic complexity for a view

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269472#comment-16269472 ] Apache Spark commented on SPARK-22637: -- User 'hvanhovell' has created a pull request

[jira] [Assigned] (SPARK-22637) CatalogImpl.refresh() has quadratic complexity for a view

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22637: Assignee: (was: Apache Spark) > CatalogImpl.refresh() has quadratic complexity for a v

[jira] [Updated] (SPARK-22637) CatalogImpl.refresh() has quadratic complexity for a view

2017-11-28 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-22637: -- Description: {{org.apache.spark.sql.internal.CatalogImpl.refreshTable}} uses {{foreach(

[jira] [Assigned] (SPARK-22637) CatalogImpl.refresh() has quadratic complexity for a view

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22637: Assignee: Apache Spark > CatalogImpl.refresh() has quadratic complexity for a view > -

[jira] [Resolved] (SPARK-22431) Creating Permanent view with illegal type

2017-11-28 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-22431. --- Resolution: Fixed Assignee: Sunitha Kambhampati Fix Version/s: 2.3.0

[jira] [Created] (SPARK-22637) CatalogImpl.refresh() has quadratic complexity for a view

2017-11-28 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-22637: - Summary: CatalogImpl.refresh() has quadratic complexity for a view Key: SPARK-22637 URL: https://issues.apache.org/jira/browse/SPARK-22637 Project: Spark

[jira] [Commented] (SPARK-22634) Update Bouncy castle dependency

2017-11-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269419#comment-16269419 ] Sean Owen commented on SPARK-22634: --- OK, so Spark pulls it in even with {{-Phadoop-prov

[jira] [Assigned] (SPARK-22618) RDD.unpersist can cause fatal exception when used with dynamic allocation

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22618: Assignee: Apache Spark > RDD.unpersist can cause fatal exception when used with dynamic al

[jira] [Commented] (SPARK-22618) RDD.unpersist can cause fatal exception when used with dynamic allocation

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269374#comment-16269374 ] Apache Spark commented on SPARK-22618: -- User 'brad-kaiser' has created a pull reques

[jira] [Assigned] (SPARK-22618) RDD.unpersist can cause fatal exception when used with dynamic allocation

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22618: Assignee: (was: Apache Spark) > RDD.unpersist can cause fatal exception when used with

[jira] [Commented] (SPARK-19552) Upgrade Netty version to 4.1.x final

2017-11-28 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269357#comment-16269357 ] Wes McKinney commented on SPARK-19552: -- Is there anyone who might be able to help wi

[jira] [Comment Edited] (SPARK-22373) Intermittent NullPointerException in org.codehaus.janino.IClass.isAssignableFrom

2017-11-28 Thread Leigh Klotz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269354#comment-16269354 ] Leigh Klotz edited comment on SPARK-22373 at 11/28/17 7:51 PM: ---

[jira] [Comment Edited] (SPARK-22373) Intermittent NullPointerException in org.codehaus.janino.IClass.isAssignableFrom

2017-11-28 Thread Leigh Klotz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269354#comment-16269354 ] Leigh Klotz edited comment on SPARK-22373 at 11/28/17 7:51 PM: ---

[jira] [Comment Edited] (SPARK-22373) Intermittent NullPointerException in org.codehaus.janino.IClass.isAssignableFrom

2017-11-28 Thread Leigh Klotz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269354#comment-16269354 ] Leigh Klotz edited comment on SPARK-22373 at 11/28/17 7:50 PM: ---

[jira] [Commented] (SPARK-22373) Intermittent NullPointerException in org.codehaus.janino.IClass.isAssignableFrom

2017-11-28 Thread Leigh Klotz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269354#comment-16269354 ] Leigh Klotz commented on SPARK-22373: -- This happens to me regularly enough using 2.

[jira] [Resolved] (SPARK-22515) Estimation relation size based on numRows * rowSize

2017-11-28 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-22515. - Resolution: Fixed Assignee: Zhenhua Wang Fix Version/s: 2.3.0 > Estimation relation size

[jira] [Created] (SPARK-22636) row count not being set correctly (always 0) after Statement.executeUpdate()

2017-11-28 Thread Sergio Lob (JIRA)
Sergio Lob created SPARK-22636: -- Summary: row count not being set correctly (always 0) after Statement.executeUpdate() Key: SPARK-22636 URL: https://issues.apache.org/jira/browse/SPARK-22636 Project: Spa

[jira] [Commented] (SPARK-22634) Update Bouncy castle dependency

2017-11-28 Thread Lior Regev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269308#comment-16269308 ] Lior Regev commented on SPARK-22634: Well, mainly there are 2 issues: # The included

[jira] [Updated] (SPARK-19552) Upgrade Netty version to 4.1.x final

2017-11-28 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19552: - Summary: Upgrade Netty version to 4.1.x final (was: Upgrade Netty version to 4.1.8 final) > Upg

[jira] [Updated] (SPARK-19552) Upgrade Netty version to 4.1.8 final

2017-11-28 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19552: - Priority: Major (was: Minor) > Upgrade Netty version to 4.1.8 final > --

[jira] [Reopened] (SPARK-19552) Upgrade Netty version to 4.1.8 final

2017-11-28 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reopened SPARK-19552: -- Netty is deprecating in https://github.com/netty/netty/issues/7439 Reopened this one to discuss ou

[jira] [Commented] (SPARK-22331) Make MLlib string params case-insensitive

2017-11-28 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269169#comment-16269169 ] yuhao yang commented on SPARK-22331: Thanks for the interests [~smurakozi]. I tried t

[jira] [Commented] (SPARK-15573) Backwards-compatible persistence for spark.ml

2017-11-28 Thread Ming Ma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269140#comment-16269140 ] Ming Ma commented on SPARK-15573: - This is probably off topic as it isn't related to Spar

[jira] [Commented] (SPARK-22373) Intermittent NullPointerException in org.codehaus.janino.IClass.isAssignableFrom

2017-11-28 Thread Min Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269128#comment-16269128 ] Min Shen commented on SPARK-22373: -- Tried running the test application using 10 concurre

[jira] [Created] (SPARK-22635) FileNotFoundException again while reading ORC files containing special characters

2017-11-28 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-22635: --- Summary: FileNotFoundException again while reading ORC files containing special characters Key: SPARK-22635 URL: https://issues.apache.org/jira/browse/SPARK-22635 Proje

[jira] [Commented] (SPARK-21866) SPIP: Image support in Spark

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269026#comment-16269026 ] Apache Spark commented on SPARK-21866: -- User 'HyukjinKwon' has created a pull reques

[jira] [Comment Edited] (SPARK-22629) incorrect handling of calls to random in UDFs

2017-11-28 Thread Michael H (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16268960#comment-16268960 ] Michael H edited comment on SPARK-22629 at 11/28/17 4:06 PM: -

[jira] [Commented] (SPARK-22629) incorrect handling of calls to random in UDFs

2017-11-28 Thread Michael H (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16268960#comment-16268960 ] Michael H commented on SPARK-22629: --- Hi Sean, Thanks for looking into this! I was worr

[jira] [Commented] (SPARK-22585) Url encoding of jar path expected?

2017-11-28 Thread Jakub Dubovsky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16268892#comment-16268892 ] Jakub Dubovsky commented on SPARK-22585: I have created a PR where we can agree o

[jira] [Commented] (SPARK-22585) Url encoding of jar path expected?

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1626#comment-1626 ] Apache Spark commented on SPARK-22585: -- User 'james64' has created a pull request fo

[jira] [Assigned] (SPARK-22585) Url encoding of jar path expected?

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22585: Assignee: (was: Apache Spark) > Url encoding of jar path expected? > -

[jira] [Assigned] (SPARK-22585) Url encoding of jar path expected?

2017-11-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22585: Assignee: Apache Spark > Url encoding of jar path expected? >

[jira] [Updated] (SPARK-19106) Styling for the configuration docs is broken

2017-11-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-19106: -- Fix Version/s: (was: 2.2.1,2.1.1) 2.1.1 2.2.1 > Styling for t

[jira] [Resolved] (SPARK-22627) Fix formatting of headers in configuration.html page

2017-11-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22627. --- Resolution: Fixed Assignee: Felix Cheung Fix Version/s: 2.2.1 Hm, I wonder if the mar

[jira] [Updated] (SPARK-19106) Styling for the configuration docs is broken

2017-11-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-19106: -- Fix Version/s: (was: 2.1.1) (was: 2.2.0) 2.2.1,2.1.1 > St

[jira] [Commented] (SPARK-22630) Consolidate all configuration properties into one page

2017-11-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16268880#comment-16268880 ] Sean Owen commented on SPARK-22630: --- The flip-side is that you end up duplicating docum

[jira] [Commented] (SPARK-22629) incorrect handling of calls to random in UDFs

2017-11-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16268875#comment-16268875 ] Sean Owen commented on SPARK-22629: --- This is expected behavior. If your source of data

[jira] [Commented] (SPARK-22627) Fix formatting of headers in configuration.html page

2017-11-28 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16268869#comment-16268869 ] Marco Gaido commented on SPARK-22627: - [~srowen] the issue seems not present anymore

  1   2   >