[jira] [Commented] (SPARK-21661) SparkSQL can't merge load table from Hadoop

[ https://issues.apache.org/jira/browse/SPARK-21661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453591#comment-16453591 ] Li Yuanjian commented on SPARK-21661: - [~hyukjin.kwon] Hi Hyukjin, do you think we sh

[jira] [Updated] (SPARK-21661) SparkSQL can't merge load table from Hadoop

[ https://issues.apache.org/jira/browse/SPARK-21661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-21661: - Fix Version/s: 2.3.0 > SparkSQL can't merge load table from Hadoop >

[jira] [Commented] (SPARK-21661) SparkSQL can't merge load table from Hadoop

[ https://issues.apache.org/jira/browse/SPARK-21661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453623#comment-16453623 ] Hyukjin Kwon commented on SPARK-21661: -- I think it's relatively risky for backportin

[jira] [Commented] (SPARK-21109) union two dataset[A] don't work as expected if one of the datasets is originated from a dataframe

[ https://issues.apache.org/jira/browse/SPARK-21109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453625#comment-16453625 ] Eyal Farago commented on SPARK-21109: - I've also encountered this issue in two separa

[jira] [Created] (SPARK-24096) file format issue using create table as select

StephenZou created SPARK-24096: -- Summary: file format issue using create table as select Key: SPARK-24096 URL: https://issues.apache.org/jira/browse/SPARK-24096 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-24096) create table as select not using hive.default.fileformat

[ https://issues.apache.org/jira/browse/SPARK-24096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] StephenZou updated SPARK-24096: --- Summary: create table as select not using hive.default.fileformat (was: file format issue using crea

[jira] [Updated] (SPARK-24096) create table as select not using hive.default.fileformat

[ https://issues.apache.org/jira/browse/SPARK-24096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] StephenZou updated SPARK-24096: --- Description: In my spark conf directory, hive-site.xml have an item indicating orc is the default fi

[jira] [Created] (SPARK-24097) Instruments improvements - RandomForest and GradientBoostedTree

Weichen Xu created SPARK-24097: -- Summary: Instruments improvements - RandomForest and GradientBoostedTree Key: SPARK-24097 URL: https://issues.apache.org/jira/browse/SPARK-24097 Project: Spark

[jira] [Commented] (SPARK-24097) Instruments improvements - RandomForest and GradientBoostedTree

[ https://issues.apache.org/jira/browse/SPARK-24097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453633#comment-16453633 ] Apache Spark commented on SPARK-24097: -- User 'WeichenXu123' has created a pull reque

[jira] [Assigned] (SPARK-24097) Instruments improvements - RandomForest and GradientBoostedTree

[ https://issues.apache.org/jira/browse/SPARK-24097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24097: Assignee: (was: Apache Spark) > Instruments improvements - RandomForest and GradientBo

[jira] [Assigned] (SPARK-24097) Instruments improvements - RandomForest and GradientBoostedTree

[ https://issues.apache.org/jira/browse/SPARK-24097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24097: Assignee: Apache Spark > Instruments improvements - RandomForest and GradientBoostedTree >

[jira] [Commented] (SPARK-23929) pandas_udf schema mapped by position and not by name

[ https://issues.apache.org/jira/browse/SPARK-23929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453646#comment-16453646 ] Hyukjin Kwon commented on SPARK-23929: -- How about we document the current behaviour

[jira] [Created] (SPARK-24098) ScriptTransformationExec should wait process exiting before output iterator finish

Lijia Liu created SPARK-24098: - Summary: ScriptTransformationExec should wait process exiting before output iterator finish Key: SPARK-24098 URL: https://issues.apache.org/jira/browse/SPARK-24098 Project:

[jira] [Assigned] (SPARK-24098) ScriptTransformationExec should wait process exiting before output iterator finish

[ https://issues.apache.org/jira/browse/SPARK-24098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24098: Assignee: Apache Spark > ScriptTransformationExec should wait process exiting before outpu

[jira] [Assigned] (SPARK-24098) ScriptTransformationExec should wait process exiting before output iterator finish

[ https://issues.apache.org/jira/browse/SPARK-24098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24098: Assignee: (was: Apache Spark) > ScriptTransformationExec should wait process exiting b

[jira] [Commented] (SPARK-24098) ScriptTransformationExec should wait process exiting before output iterator finish

[ https://issues.apache.org/jira/browse/SPARK-24098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453716#comment-16453716 ] Apache Spark commented on SPARK-24098: -- User 'liutang123' has created a pull request

[jira] [Created] (SPARK-24099) java.io.CharConversionException: Invalid UTF-32 character prevents me from querying my data in JSON

ABHISHEK KUMAR GUPTA created SPARK-24099: Summary: java.io.CharConversionException: Invalid UTF-32 character prevents me from querying my data in JSON Key: SPARK-24099 URL: https://issues.apache.org/jira/b

[jira] [Commented] (SPARK-23885) trying to spark submit 2.3.0 on minikube

[ https://issues.apache.org/jira/browse/SPARK-23885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453803#comment-16453803 ] anant pukale commented on SPARK-23885: -- HI Anirudh, Kindly suggest any work around

[jira] [Commented] (SPARK-13446) Spark need to support reading data from Hive 2.0.0 metastore

[ https://issues.apache.org/jira/browse/SPARK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453825#comment-16453825 ] Steve Loughran commented on SPARK-13446: [~Tavis]: can you paste in the stack you

[jira] [Commented] (SPARK-11334) numRunningTasks can't be less than 0, or it will affect executor allocation

[ https://issues.apache.org/jira/browse/SPARK-11334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453838#comment-16453838 ] Apache Spark commented on SPARK-11334: -- User 'sadhen' has created a pull request for

[jira] [Commented] (SPARK-11748) Result is null after alter column name of table stored as Parquet

[ https://issues.apache.org/jira/browse/SPARK-11748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453858#comment-16453858 ] Peter Simon commented on SPARK-11748: - Possible workaround can be: {code:java} scala>

[jira] [Commented] (SPARK-4781) Column values become all NULL after doing ALTER TABLE CHANGE for renaming column names (Parquet external table in HiveContext)

[ https://issues.apache.org/jira/browse/SPARK-4781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453861#comment-16453861 ] Peter Simon commented on SPARK-4781: As commented under SPARK-11748,  Possible workar

[jira] [Commented] (SPARK-23929) pandas_udf schema mapped by position and not by name

[ https://issues.apache.org/jira/browse/SPARK-23929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453880#comment-16453880 ] Tr3wory commented on SPARK-23929: - Yes, the documentation is a must, even for 2.3 if poss

[jira] [Commented] (SPARK-23856) Spark jdbc setQueryTimeout option

[ https://issues.apache.org/jira/browse/SPARK-23856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453948#comment-16453948 ] Takeshi Yamamuro commented on SPARK-23856: -- [~dmitrymikhailov] You don't have ti

[jira] [Created] (SPARK-24100) Add the CompressionCodec to the saveAsTextFiles interface.

caijie created SPARK-24100: -- Summary: Add the CompressionCodec to the saveAsTextFiles interface. Key: SPARK-24100 URL: https://issues.apache.org/jira/browse/SPARK-24100 Project: Spark Issue Type: Im

[jira] [Assigned] (SPARK-24100) Add the CompressionCodec to the saveAsTextFiles interface.

[ https://issues.apache.org/jira/browse/SPARK-24100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24100: Assignee: Apache Spark > Add the CompressionCodec to the saveAsTextFiles interface. >

[jira] [Commented] (SPARK-24100) Add the CompressionCodec to the saveAsTextFiles interface.

[ https://issues.apache.org/jira/browse/SPARK-24100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453952#comment-16453952 ] Apache Spark commented on SPARK-24100: -- User 'WzRaCai' has created a pull request fo

[jira] [Assigned] (SPARK-24100) Add the CompressionCodec to the saveAsTextFiles interface.

[ https://issues.apache.org/jira/browse/SPARK-24100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24100: Assignee: (was: Apache Spark) > Add the CompressionCodec to the saveAsTextFiles interf

[jira] [Commented] (SPARK-23151) Provide a distribution of Spark with Hadoop 3.0

[ https://issues.apache.org/jira/browse/SPARK-23151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453991#comment-16453991 ] Steve Loughran commented on SPARK-23151: well, there's "have everything work on H

[jira] [Commented] (SPARK-21661) SparkSQL can't merge load table from Hadoop

[ https://issues.apache.org/jira/browse/SPARK-21661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454070#comment-16454070 ] Li Yuanjian commented on SPARK-21661: - Got it. > SparkSQL can't merge load table fro

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454221#comment-16454221 ] Wenchen Fan commented on SPARK-23715: - It seems the `from_utc_timestamp` doesn't make

[jira] [Comment Edited] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454221#comment-16454221 ] Wenchen Fan edited comment on SPARK-23715 at 4/26/18 1:53 PM: -

[jira] [Commented] (SPARK-13446) Spark need to support reading data from Hive 2.0.0 metastore

[ https://issues.apache.org/jira/browse/SPARK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454278#comment-16454278 ] Tavis Barr commented on SPARK-13446: Enjoy!   This is with Spark 2.3.0 and Hive 2.2

[jira] [Commented] (SPARK-23830) Spark on YARN in cluster deploy mode fail with NullPointerException when a Spark application is a Scala class not object

[ https://issues.apache.org/jira/browse/SPARK-23830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454358#comment-16454358 ] Apache Spark commented on SPARK-23830: -- User 'eric-maynard' has created a pull reque

[jira] [Assigned] (SPARK-23830) Spark on YARN in cluster deploy mode fail with NullPointerException when a Spark application is a Scala class not object

[ https://issues.apache.org/jira/browse/SPARK-23830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23830: Assignee: (was: Apache Spark) > Spark on YARN in cluster deploy mode fail with NullPoi

[jira] [Assigned] (SPARK-23830) Spark on YARN in cluster deploy mode fail with NullPointerException when a Spark application is a Scala class not object

[ https://issues.apache.org/jira/browse/SPARK-23830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23830: Assignee: Apache Spark > Spark on YARN in cluster deploy mode fail with NullPointerExcepti

[jira] [Commented] (SPARK-23830) Spark on YARN in cluster deploy mode fail with NullPointerException when a Spark application is a Scala class not object

[ https://issues.apache.org/jira/browse/SPARK-23830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454365#comment-16454365 ] Eric Maynard commented on SPARK-23830: -- [~jerryshao] I agree we should not support u

[jira] [Commented] (SPARK-23933) High-order function: map(array, array) → map

[ https://issues.apache.org/jira/browse/SPARK-23933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454393#comment-16454393 ] Alex Wajda commented on SPARK-23933: Oh, now I got it, thanks. I overlooked the case

[jira] [Created] (SPARK-24101) MulticlassClassificationEvaluator should use sample weight data

Ilya Matiach created SPARK-24101: Summary: MulticlassClassificationEvaluator should use sample weight data Key: SPARK-24101 URL: https://issues.apache.org/jira/browse/SPARK-24101 Project: Spark

[jira] [Created] (SPARK-24102) RegressionEvaluator should use sample weight data

Ilya Matiach created SPARK-24102: Summary: RegressionEvaluator should use sample weight data Key: SPARK-24102 URL: https://issues.apache.org/jira/browse/SPARK-24102 Project: Spark Issue Type:

[jira] [Updated] (SPARK-24101) MulticlassClassificationEvaluator should use sample weight data

[ https://issues.apache.org/jira/browse/SPARK-24101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Matiach updated SPARK-24101: - Issue Type: Improvement (was: Bug) > MulticlassClassificationEvaluator should use sample weight

[jira] [Updated] (SPARK-24102) RegressionEvaluator should use sample weight data

[ https://issues.apache.org/jira/browse/SPARK-24102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Matiach updated SPARK-24102: - Issue Type: Improvement (was: Bug) > RegressionEvaluator should use sample weight data > ---

[jira] [Created] (SPARK-24103) BinaryClassificationEvaluator should use sample weight data

Ilya Matiach created SPARK-24103: Summary: BinaryClassificationEvaluator should use sample weight data Key: SPARK-24103 URL: https://issues.apache.org/jira/browse/SPARK-24103 Project: Spark

[jira] [Commented] (SPARK-18693) BinaryClassificationEvaluator, RegressionEvaluator, and MulticlassClassificationEvaluator should use sample weight data

[ https://issues.apache.org/jira/browse/SPARK-18693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454435#comment-16454435 ] Ilya Matiach commented on SPARK-18693: -- [~josephkb] sure, I've added 3 JIRAs for tra

[jira] [Updated] (SPARK-24103) BinaryClassificationEvaluator should use sample weight data

[ https://issues.apache.org/jira/browse/SPARK-24103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Matiach updated SPARK-24103: - Description: The LogisticRegression and LinearRegression models support training with a weight co

[jira] [Updated] (SPARK-24102) RegressionEvaluator should use sample weight data

[ https://issues.apache.org/jira/browse/SPARK-24102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Matiach updated SPARK-24102: - Description: The LogisticRegression and LinearRegression models support training with a weight co

[jira] [Updated] (SPARK-24101) MulticlassClassificationEvaluator should use sample weight data

[ https://issues.apache.org/jira/browse/SPARK-24101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Matiach updated SPARK-24101: - Description: The LogisticRegression and LinearRegression models support training with a weight co

[jira] [Assigned] (SPARK-24101) MulticlassClassificationEvaluator should use sample weight data

[ https://issues.apache.org/jira/browse/SPARK-24101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24101: Assignee: Apache Spark > MulticlassClassificationEvaluator should use sample weight data >

[jira] [Assigned] (SPARK-24101) MulticlassClassificationEvaluator should use sample weight data

[ https://issues.apache.org/jira/browse/SPARK-24101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24101: Assignee: (was: Apache Spark) > MulticlassClassificationEvaluator should use sample we

[jira] [Commented] (SPARK-24101) MulticlassClassificationEvaluator should use sample weight data

[ https://issues.apache.org/jira/browse/SPARK-24101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454445#comment-16454445 ] Apache Spark commented on SPARK-24101: -- User 'imatiach-msft' has created a pull requ

[jira] [Commented] (SPARK-20087) Include accumulators / taskMetrics when sending TaskKilled to onTaskEnd listeners

[ https://issues.apache.org/jira/browse/SPARK-20087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454454#comment-16454454 ] Apache Spark commented on SPARK-20087: -- User 'advancedxy' has created a pull request

[jira] [Resolved] (SPARK-24096) create table as select not using hive.default.fileformat

[ https://issues.apache.org/jira/browse/SPARK-24096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-24096. - Resolution: Duplicate > create table as select not using hive.default.fileformat > --

[jira] [Assigned] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23715: Assignee: (was: Apache Spark) > from_utc_timestamp returns incorrect results for some

[jira] [Commented] (SPARK-24096) create table as select not using hive.default.fileformat

[ https://issues.apache.org/jira/browse/SPARK-24096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454463#comment-16454463 ] Yuming Wang commented on SPARK-24096: - Another related PR: https://github.com/apache/

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454462#comment-16454462 ] Apache Spark commented on SPARK-23715: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23715: Assignee: Apache Spark > from_utc_timestamp returns incorrect results for some UTC date/ti

[jira] [Commented] (SPARK-22732) Add DataSourceV2 streaming APIs

[ https://issues.apache.org/jira/browse/SPARK-22732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454501#comment-16454501 ] Apache Spark commented on SPARK-22732: -- User 'wangyum' has created a pull request fo

[jira] [Commented] (SPARK-23925) High-order function: repeat(element, count) → array

[ https://issues.apache.org/jira/browse/SPARK-23925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454526#comment-16454526 ] Marek Novotny commented on SPARK-23925: --- @pepinoflo Any joy? I can take this one or

[jira] [Comment Edited] (SPARK-23925) High-order function: repeat(element, count) → array

[ https://issues.apache.org/jira/browse/SPARK-23925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454526#comment-16454526 ] Marek Novotny edited comment on SPARK-23925 at 4/26/18 4:47 PM: ---

[jira] [Created] (SPARK-24104) SQLAppStatusListener overwrites metrics onDriverAccumUpdates instead of updating them

Juliusz Sompolski created SPARK-24104: - Summary: SQLAppStatusListener overwrites metrics onDriverAccumUpdates instead of updating them Key: SPARK-24104 URL: https://issues.apache.org/jira/browse/SPARK-24104

[jira] [Commented] (SPARK-23842) accessing java from PySpark lambda functions

[ https://issues.apache.org/jira/browse/SPARK-23842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454585#comment-16454585 ] holdenk commented on SPARK-23842: - So the py4j gateway only exists on the driver program,

[jira] [Resolved] (SPARK-23842) accessing java from PySpark lambda functions

[ https://issues.apache.org/jira/browse/SPARK-23842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk resolved SPARK-23842. - Resolution: Won't Fix Not supported by the current design, alternatives do exist though. > accessing jav

[jira] [Commented] (SPARK-23962) Flaky tests from SQLMetricsTestUtils.currentExecutionIds

[ https://issues.apache.org/jira/browse/SPARK-23962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454594#comment-16454594 ] Dongjoon Hyun commented on SPARK-23962: --- [~irashid], [~cloud_fan], Please check the

[jira] [Created] (SPARK-24105) Spark 2.3.0 on kubernetes

Lenin created SPARK-24105: - Summary: Spark 2.3.0 on kubernetes Key: SPARK-24105 URL: https://issues.apache.org/jira/browse/SPARK-24105 Project: Spark Issue Type: Improvement Components: Kub

[jira] [Assigned] (SPARK-24104) SQLAppStatusListener overwrites metrics onDriverAccumUpdates instead of updating them

[ https://issues.apache.org/jira/browse/SPARK-24104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24104: Assignee: Apache Spark > SQLAppStatusListener overwrites metrics onDriverAccumUpdates inst

[jira] [Commented] (SPARK-24104) SQLAppStatusListener overwrites metrics onDriverAccumUpdates instead of updating them

[ https://issues.apache.org/jira/browse/SPARK-24104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454630#comment-16454630 ] Apache Spark commented on SPARK-24104: -- User 'juliuszsompolski' has created a pull r

[jira] [Assigned] (SPARK-24104) SQLAppStatusListener overwrites metrics onDriverAccumUpdates instead of updating them

[ https://issues.apache.org/jira/browse/SPARK-24104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24104: Assignee: (was: Apache Spark) > SQLAppStatusListener overwrites metrics onDriverAccumU

[jira] [Updated] (SPARK-24105) Spark 2.3.0 on kubernetes

[ https://issues.apache.org/jira/browse/SPARK-24105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lenin updated SPARK-24105: -- Description: Right now its only possible to define node selector configurations  thruspark.kubernetes.node.sele

[jira] [Commented] (SPARK-24085) Scalar subquery error

[ https://issues.apache.org/jira/browse/SPARK-24085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454707#comment-16454707 ] Dilip Biswal commented on SPARK-24085: -- Working on a fix for this. > Scalar subquer

[jira] [Comment Edited] (SPARK-24085) Scalar subquery error

[ https://issues.apache.org/jira/browse/SPARK-24085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454707#comment-16454707 ] Dilip Biswal edited comment on SPARK-24085 at 4/26/18 7:09 PM:

[jira] [Assigned] (SPARK-23120) Add PMML pipeline export support to PySpark

[ https://issues.apache.org/jira/browse/SPARK-23120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23120: Assignee: Apache Spark (was: holdenk) > Add PMML pipeline export support to PySpark > ---

[jira] [Assigned] (SPARK-23120) Add PMML pipeline export support to PySpark

[ https://issues.apache.org/jira/browse/SPARK-23120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23120: Assignee: holdenk (was: Apache Spark) > Add PMML pipeline export support to PySpark > ---

[jira] [Commented] (SPARK-23120) Add PMML pipeline export support to PySpark

[ https://issues.apache.org/jira/browse/SPARK-23120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454716#comment-16454716 ] Apache Spark commented on SPARK-23120: -- User 'holdenk' has created a pull request fo

[jira] [Commented] (SPARK-24068) CSV schema inferring doesn't work for compressed files

[ https://issues.apache.org/jira/browse/SPARK-24068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454761#comment-16454761 ] Maxim Gekk commented on SPARK-24068: The same issue exists in JSON datasource. [~hyuk

[jira] [Created] (SPARK-24106) Spark Structure Streaming with RF model taking long time in processing probability for each mini batch

Tamilselvan Veeramani created SPARK-24106: - Summary: Spark Structure Streaming with RF model taking long time in processing probability for each mini batch Key: SPARK-24106 URL: https://issues.apache.org/j

[jira] [Commented] (SPARK-24106) Spark Structure Streaming with RF model taking long time in processing probability for each mini batch

[ https://issues.apache.org/jira/browse/SPARK-24106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454774#comment-16454774 ] Tamilselvan Veeramani commented on SPARK-24106: --- I have the code change rea

[jira] [Updated] (SPARK-24106) Spark Structure Streaming with RF model taking long time in processing probability for each mini batch

[ https://issues.apache.org/jira/browse/SPARK-24106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tamilselvan Veeramani updated SPARK-24106: -- Target Version/s: 2.3.0, 2.2.1 (was: 2.2.1, 2.3.0) Component/s: (w

[jira] [Commented] (SPARK-24036) Stateful operators in continuous processing

[ https://issues.apache.org/jira/browse/SPARK-24036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454788#comment-16454788 ] Jose Torres commented on SPARK-24036: - https://docs.google.com/document/d/1IL4kJoKrZW

[jira] [Updated] (SPARK-24106) Spark Structure Streaming with RF model taking long time in processing probability for each mini batch

[ https://issues.apache.org/jira/browse/SPARK-24106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tamilselvan Veeramani updated SPARK-24106: -- Target Version/s: 2.3.0, 2.2.1 (was: 2.2.1, 2.3.0) Issue Type: Improv

[jira] [Commented] (SPARK-24105) Spark 2.3.0 on kubernetes

[ https://issues.apache.org/jira/browse/SPARK-24105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454847#comment-16454847 ] Anirudh Ramanathan commented on SPARK-24105: > To avoid this deadlock, its re

[jira] [Assigned] (SPARK-24057) put the real data type in the AssertionError message

[ https://issues.apache.org/jira/browse/SPARK-24057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-24057: Assignee: Huaxin Gao > put the real data type in the AssertionError message >

[jira] [Resolved] (SPARK-24057) put the real data type in the AssertionError message

[ https://issues.apache.org/jira/browse/SPARK-24057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-24057. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21159 [https://git

[jira] [Commented] (SPARK-23580) Interpreted mode fallback should be implemented for all expressions & projections

[ https://issues.apache.org/jira/browse/SPARK-23580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454918#comment-16454918 ] Bruce Robbins commented on SPARK-23580: --- Should SortPrefix also get this treatment?

[jira] [Assigned] (SPARK-23856) Spark jdbc setQueryTimeout option

[ https://issues.apache.org/jira/browse/SPARK-23856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23856: Assignee: Apache Spark > Spark jdbc setQueryTimeout option > -

[jira] [Commented] (SPARK-23856) Spark jdbc setQueryTimeout option

[ https://issues.apache.org/jira/browse/SPARK-23856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16455342#comment-16455342 ] Apache Spark commented on SPARK-23856: -- User 'maropu' has created a pull request for

[jira] [Assigned] (SPARK-23856) Spark jdbc setQueryTimeout option

[ https://issues.apache.org/jira/browse/SPARK-23856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23856: Assignee: (was: Apache Spark) > Spark jdbc setQueryTimeout option > --

[jira] [Assigned] (SPARK-24044) Explicitly print out skipped tests from unittest module

[ https://issues.apache.org/jira/browse/SPARK-24044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-24044: Assignee: Hyukjin Kwon > Explicitly print out skipped tests from unittest module > ---

[jira] [Resolved] (SPARK-24044) Explicitly print out skipped tests from unittest module

[ https://issues.apache.org/jira/browse/SPARK-24044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-24044. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21107 [https://git

[jira] [Assigned] (SPARK-24085) Scalar subquery error

[ https://issues.apache.org/jira/browse/SPARK-24085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24085: Assignee: Apache Spark > Scalar subquery error > - > >

[jira] [Commented] (SPARK-24085) Scalar subquery error

[ https://issues.apache.org/jira/browse/SPARK-24085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16455468#comment-16455468 ] Apache Spark commented on SPARK-24085: -- User 'dilipbiswal' has created a pull reques

[jira] [Assigned] (SPARK-24085) Scalar subquery error

[ https://issues.apache.org/jira/browse/SPARK-24085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24085: Assignee: (was: Apache Spark) > Scalar subquery error > - > >

[jira] [Commented] (SPARK-21274) Implement EXCEPT ALL and INTERSECT ALL

[ https://issues.apache.org/jira/browse/SPARK-21274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16455470#comment-16455470 ] Takeshi Yamamuro commented on SPARK-21274: -- Looks great to me. I checked the que

[jira] [Resolved] (SPARK-24083) Diagnostics message for uncaught exceptions should include the stacktrace

[ https://issues.apache.org/jira/browse/SPARK-24083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-24083. Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21151 [https:/

[jira] [Assigned] (SPARK-24083) Diagnostics message for uncaught exceptions should include the stacktrace

[ https://issues.apache.org/jira/browse/SPARK-24083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-24083: -- Assignee: zhoukang > Diagnostics message for uncaught exceptions should include the st

[jira] [Commented] (SPARK-23925) High-order function: repeat(element, count) → array

[ https://issues.apache.org/jira/browse/SPARK-23925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16455506#comment-16455506 ] Florent Pepin commented on SPARK-23925: --- Hey, sorry I didn't realise the complexity

[jira] [Commented] (SPARK-24068) CSV schema inferring doesn't work for compressed files

[ https://issues.apache.org/jira/browse/SPARK-24068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16455611#comment-16455611 ] Hyukjin Kwon commented on SPARK-24068: -- I roughly assume the fix will be small, simi

[jira] [Comment Edited] (SPARK-24068) CSV schema inferring doesn't work for compressed files

[ https://issues.apache.org/jira/browse/SPARK-24068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16455611#comment-16455611 ] Hyukjin Kwon edited comment on SPARK-24068 at 4/27/18 12:51 AM: ---

[jira] [Resolved] (SPARK-24099) java.io.CharConversionException: Invalid UTF-32 character prevents me from querying my data in JSON

[ https://issues.apache.org/jira/browse/SPARK-24099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24099. -- Resolution: Duplicate It will be fixed in SPARK-23723 soon. > java.io.CharConversionException:

[jira] [Commented] (SPARK-23830) Spark on YARN in cluster deploy mode fail with NullPointerException when a Spark application is a Scala class not object

[ https://issues.apache.org/jira/browse/SPARK-23830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16455621#comment-16455621 ] Saisai Shao commented on SPARK-23830: - I agree [~emaynard]. > Spark on YARN in clust

[jira] [Updated] (SPARK-24100) Add the CompressionCodec to the saveAsTextFiles interface.

[ https://issues.apache.org/jira/browse/SPARK-24100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijie updated SPARK-24100: --- Component/s: (was: DStreams) > Add the CompressionCodec to the saveAsTextFiles interface. > -

  1   2   >