[jira] [Resolved] (SPARK-30700) NaiveBayesModel predict optimization

2020-01-31 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-30700. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27427 [https://gi

[jira] [Assigned] (SPARK-30700) NaiveBayesModel predict optimization

2020-01-31 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng reassigned SPARK-30700: Assignee: zhengruifeng > NaiveBayesModel predict optimization > -

[jira] [Resolved] (SPARK-29138) Flaky test: pyspark.mllib.tests.test_streaming_algorithms.StreamingLogisticRegressionWithSGDTests.test_parameter_accuracy

2020-01-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29138. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27424 [https://gi

[jira] [Assigned] (SPARK-29138) Flaky test: pyspark.mllib.tests.test_streaming_algorithms.StreamingLogisticRegressionWithSGDTests.test_parameter_accuracy

2020-01-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-29138: Assignee: Dongjoon Hyun > Flaky test: > pyspark.mllib.tests.test_streaming_algorithms.St

[jira] [Created] (SPARK-30700) NaiveBayesModel predict optimization

2020-01-31 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-30700: Summary: NaiveBayesModel predict optimization Key: SPARK-30700 URL: https://issues.apache.org/jira/browse/SPARK-30700 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-30699) GMM blockify input vectors

2020-01-31 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-30699: Summary: GMM blockify input vectors Key: SPARK-30699 URL: https://issues.apache.org/jira/browse/SPARK-30699 Project: Spark Issue Type: Sub-task Com

[jira] [Comment Edited] (SPARK-30696) Wrong result of the combination of from_utc_timestamp and to_utc_timestamp

2020-01-31 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027975#comment-17027975 ] Maxim Gekk edited comment on SPARK-30696 at 2/1/20 5:28 AM:

[jira] [Commented] (SPARK-30696) Wrong result of the combination of from_utc_timestamp and to_utc_timestamp

2020-01-31 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027975#comment-17027975 ] Maxim Gekk commented on SPARK-30696: [~dongjoon] We have different default time zone

[jira] [Commented] (SPARK-28344) fail the query if detect ambiguous self join

2020-01-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027973#comment-17027973 ] Dongjoon Hyun commented on SPARK-28344: --- Please refer the discussion on our backpo

[jira] [Updated] (SPARK-28344) fail the query if detect ambiguous self join

2020-01-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28344: -- Target Version/s: 3.0.0 (was: 2.4.5, 3.0.0) > fail the query if detect ambiguous self join >

[jira] [Assigned] (SPARK-30698) Bumps checkstyle from 8.25 to 8.29.

2020-01-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-30698: - Assignee: jiaan.geng > Bumps checkstyle from 8.25 to 8.29. > --

[jira] [Resolved] (SPARK-30698) Bumps checkstyle from 8.25 to 8.29.

2020-01-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30698. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27426 [https://

[jira] [Resolved] (SPARK-30689) Allow custom resource scheduling to work with YARN versions that don't support custom resource scheduling

2020-01-31 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-30689. --- Fix Version/s: 3.0.0 Assignee: Thomas Graves Resolution: Fixed > Allow custo

[jira] [Updated] (SPARK-27946) Hive DDL to Spark DDL conversion USING "show create table"

2020-01-31 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-27946: Description: This patch adds a DDL command SHOW CREATE TABLE AS SERDE. It is used to generate Hive DDL fo

[jira] [Resolved] (SPARK-27946) Hive DDL to Spark DDL conversion USING "show create table"

2020-01-31 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-27946. - Fix Version/s: 3.0.0 Assignee: L. C. Hsieh Resolution: Fixed > Hive DDL to Spark DDL con

[jira] [Resolved] (SPARK-30660) LinearRegression blockify input vectors

2020-01-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-30660. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27396 [https://gi

[jira] [Assigned] (SPARK-30660) LinearRegression blockify input vectors

2020-01-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-30660: Assignee: zhengruifeng > LinearRegression blockify input vectors > --

[jira] [Created] (SPARK-30698) Bumps checkstyle from 8.25 to 8.29.

2020-01-31 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-30698: -- Summary: Bumps checkstyle from 8.25 to 8.29. Key: SPARK-30698 URL: https://issues.apache.org/jira/browse/SPARK-30698 Project: Spark Issue Type: Dependency upgrad

[jira] [Commented] (SPARK-29138) Flaky test: pyspark.mllib.tests.test_streaming_algorithms.StreamingLogisticRegressionWithSGDTests.test_parameter_accuracy

2020-01-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027930#comment-17027930 ] Dongjoon Hyun commented on SPARK-29138: --- I saw this in three places (2 independent

[jira] [Resolved] (SPARK-30695) Upgrade Apache ORC to 1.5.9

2020-01-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30695. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27421 [https://

[jira] [Commented] (SPARK-30658) Limit after on streaming dataframe before streaming agg returns wrong results

2020-01-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027913#comment-17027913 ] Dongjoon Hyun commented on SPARK-30658: --- Got it. Thank you for confirmation, [~tda

[jira] [Commented] (SPARK-30657) Streaming limit after streaming dropDuplicates can throw error

2020-01-31 Thread Tathagata Das (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027910#comment-17027910 ] Tathagata Das commented on SPARK-30657: --- This fix by itself (separate from the fix

[jira] [Commented] (SPARK-30658) Limit after on streaming dataframe before streaming agg returns wrong results

2020-01-31 Thread Tathagata Das (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027906#comment-17027906 ] Tathagata Das commented on SPARK-30658: --- I am a little afraid to backport this bec

[jira] [Commented] (SPARK-30658) Limit after on streaming dataframe before streaming agg returns wrong results

2020-01-31 Thread Tathagata Das (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027907#comment-17027907 ] Tathagata Das commented on SPARK-30658: --- Fixed in this PR https://github.com/apach

[jira] [Resolved] (SPARK-30657) Streaming limit after streaming dropDuplicates can throw error

2020-01-31 Thread Tathagata Das (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-30657. --- Resolution: Fixed > Streaming limit after streaming dropDuplicates can throw error > ---

[jira] [Resolved] (SPARK-30658) Limit after on streaming dataframe before streaming agg returns wrong results

2020-01-31 Thread Tathagata Das (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-30658. --- Resolution: Fixed > Limit after on streaming dataframe before streaming agg returns wrong re

[jira] [Assigned] (SPARK-30658) Limit after on streaming dataframe before streaming agg returns wrong results

2020-01-31 Thread Tathagata Das (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reassigned SPARK-30658: - Assignee: Tathagata Das > Limit after on streaming dataframe before streaming agg retur

[jira] [Updated] (SPARK-30658) Limit after on streaming dataframe before streaming agg returns wrong results

2020-01-31 Thread Tathagata Das (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-30658: -- Description: Limit before a streaming aggregate (i.e. {{df.limit(5).groupBy().count()}}) in co

[jira] [Updated] (SPARK-30688) Spark SQL Unix Timestamp produces incorrect result with unix_timestamp UDF

2020-01-31 Thread Rajkumar Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh updated SPARK-30688: --- Description:   {code:java} scala> spark.sql("select unix_timestamp('20201', 'ww')").show

[jira] [Created] (SPARK-30697) Handle database and namespace exceptions in catalog.isView

2020-01-31 Thread Burak Yavuz (Jira)
Burak Yavuz created SPARK-30697: --- Summary: Handle database and namespace exceptions in catalog.isView Key: SPARK-30697 URL: https://issues.apache.org/jira/browse/SPARK-30697 Project: Spark Issu

[jira] [Commented] (SPARK-30696) Wrong result of the combination of from_utc_timestamp and to_utc_timestamp

2020-01-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027885#comment-17027885 ] Dongjoon Hyun commented on SPARK-30696: --- I got the following numbers from the exam

[jira] [Commented] (SPARK-30696) Wrong result of the combination of from_utc_timestamp and to_utc_timestamp

2020-01-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027883#comment-17027883 ] Dongjoon Hyun commented on SPARK-30696: --- Thank you for pinging me. > Wrong result

[jira] [Resolved] (SPARK-30508) Add DataFrameReader.executeCommand API for external datasource

2020-01-31 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-30508. - Fix Version/s: 3.0.0 Resolution: Fixed > Add DataFrameReader.executeCommand API for external data

[jira] [Assigned] (SPARK-30508) Add DataFrameReader.executeCommand API for external datasource

2020-01-31 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-30508: --- Assignee: wuyi > Add DataFrameReader.executeCommand API for external datasource > -

[jira] [Commented] (SPARK-30614) The native ALTER COLUMN syntax should change one thing at a time

2020-01-31 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027878#comment-17027878 ] Terry Kim commented on SPARK-30614: --- Currently, ALTER COLUMN requires TYPE property fo

[jira] [Assigned] (SPARK-30695) Upgrade Apache ORC to 1.5.9

2020-01-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-30695: - Assignee: Dongjoon Hyun > Upgrade Apache ORC to 1.5.9 > --- > >

[jira] [Commented] (SPARK-30696) Wrong result of the combination of from_utc_timestamp and to_utc_timestamp

2020-01-31 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027834#comment-17027834 ] Maxim Gekk commented on SPARK-30696: The issue can be reproduced when DateTimeUtils

[jira] [Commented] (SPARK-30696) Wrong result of the combination of from_utc_timestamp and to_utc_timestamp

2020-01-31 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027830#comment-17027830 ] Maxim Gekk commented on SPARK-30696: [~dongjoon] FYI > Wrong result of the combinat

[jira] [Created] (SPARK-30696) Wrong result of the combination of from_utc_timestamp and to_utc_timestamp

2020-01-31 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-30696: -- Summary: Wrong result of the combination of from_utc_timestamp and to_utc_timestamp Key: SPARK-30696 URL: https://issues.apache.org/jira/browse/SPARK-30696 Project: Spark

[jira] [Resolved] (SPARK-30676) Eliminate warnings from deprecated constructors of java.lang.Integer and java.lang.Double

2020-01-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-30676. -- Fix Version/s: 3.0.0 Assignee: Maxim Gekk Resolution: Fixed Resolved by https:

[jira] [Resolved] (SPARK-27324) document configurations related to executor metrics

2020-01-31 Thread Imran Rashid (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-27324. -- Fix Version/s: 3.0.0 Resolution: Fixed > document configurations related to executor me

[jira] [Assigned] (SPARK-27324) document configurations related to executor metrics

2020-01-31 Thread Imran Rashid (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid reassigned SPARK-27324: Assignee: Wing Yew Poon > document configurations related to executor metrics > -

[jira] [Commented] (SPARK-27324) document configurations related to executor metrics

2020-01-31 Thread Imran Rashid (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027807#comment-17027807 ] Imran Rashid commented on SPARK-27324: -- Fixed by https://github.com/apache/spark/pu

[jira] [Commented] (SPARK-25355) Support --proxy-user for Spark on K8s

2020-01-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027801#comment-17027801 ] Dongjoon Hyun commented on SPARK-25355: --- [~pedro.rossi]. 2.4.x is maintained, but

[jira] [Resolved] (SPARK-30684) Show the descripton of metrics for WholeStageCodegen in DAG viz

2020-01-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30684. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27405 [https://

[jira] [Updated] (SPARK-30036) REPARTITION hint dose not work with order by

2020-01-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30036: -- Fix Version/s: (was: 3.0.0) > REPARTITION hint dose not work with order by > -

[jira] [Commented] (SPARK-30036) REPARTITION hint dose not work with order by

2020-01-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027779#comment-17027779 ] Dongjoon Hyun commented on SPARK-30036: --- This is reverted via [https://github.com

[jira] [Reopened] (SPARK-30036) REPARTITION hint dose not work with order by

2020-01-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-30036: --- Assignee: (was: Jackey Lee) > REPARTITION hint dose not work with order by > -

[jira] [Resolved] (SPARK-30691) Add a few main pages

2020-01-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-30691. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27416 [https://gi

[jira] [Assigned] (SPARK-30691) Add a few main pages

2020-01-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-30691: Assignee: Huaxin Gao > Add a few main pages > > > Ke

[jira] [Updated] (SPARK-30691) Add a few main pages

2020-01-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-30691: - Priority: Minor (was: Major) > Add a few main pages > > >

[jira] [Created] (SPARK-30695) Upgrade Apache ORC to 1.5.9

2020-01-31 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-30695: - Summary: Upgrade Apache ORC to 1.5.9 Key: SPARK-30695 URL: https://issues.apache.org/jira/browse/SPARK-30695 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-30658) Limit after on streaming dataframe before streaming agg returns wrong results

2020-01-31 Thread Shixiong Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-30658: - Fix Version/s: 3.0.0 > Limit after on streaming dataframe before streaming agg returns wrong res

[jira] [Updated] (SPARK-30657) Streaming limit after streaming dropDuplicates can throw error

2020-01-31 Thread Shixiong Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-30657: - Fix Version/s: 3.0.0 > Streaming limit after streaming dropDuplicates can throw error >

[jira] [Commented] (SPARK-27996) Spark UI redirect will be failed behind the https reverse proxy

2020-01-31 Thread Igor Shikin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027680#comment-17027680 ] Igor Shikin commented on SPARK-27996: - PR to master: [https://github.com/apache/spar

[jira] [Commented] (SPARK-25355) Support --proxy-user for Spark on K8s

2020-01-31 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027672#comment-17027672 ] Pedro Gonçalves Rossi Rodrigues commented on SPARK-25355: - This

[jira] [Commented] (SPARK-25355) Support --proxy-user for Spark on K8s

2020-01-31 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027665#comment-17027665 ] Pedro Gonçalves Rossi Rodrigues commented on SPARK-25355: - [~van

[jira] [Updated] (SPARK-30510) Publicly document options under spark.sql.*

2020-01-31 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-30510: - Description: SPARK-20236 added a new option, {{spark.sql.sources.partitionOverwriteMode}

[jira] [Updated] (SPARK-30510) Publicly document options under spark.sql.*

2020-01-31 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-30510: - Summary: Publicly document options under spark.sql.* (was: Document spark.sql.sources.p

[jira] [Commented] (SPARK-30686) Spark 2.4.4 metrics endpoint throwing error

2020-01-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027622#comment-17027622 ] Dongjoon Hyun commented on SPARK-30686: --- Thanks for investigation. Please ping on

[jira] [Assigned] (SPARK-30511) Spark marks intentionally killed speculative tasks as pending leads to holding idle executors

2020-01-31 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-30511: - Assignee: Zebing Lin > Spark marks intentionally killed speculative tasks as pending le

[jira] [Resolved] (SPARK-30511) Spark marks intentionally killed speculative tasks as pending leads to holding idle executors

2020-01-31 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-30511. --- Fix Version/s: 3.0.0 Resolution: Fixed > Spark marks intentionally killed speculative

[jira] [Resolved] (SPARK-30638) add resources as parameter to the PluginContext

2020-01-31 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-30638. --- Fix Version/s: 3.0.0 Assignee: Thomas Graves Resolution: Fixed > add resourc

[jira] [Created] (SPARK-30694) If exception occured while fetching blocks by ExternalBlockClient, fail early when External Shuffle Service is not alive

2020-01-31 Thread angerszhu (Jira)
angerszhu created SPARK-30694: - Summary: If exception occured while fetching blocks by ExternalBlockClient, fail early when External Shuffle Service is not alive Key: SPARK-30694 URL: https://issues.apache.org/jira/br

[jira] [Assigned] (SPARK-30690) Expose the documentation of CalendarInternval in API documentation

2020-01-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-30690: Assignee: Hyukjin Kwon > Expose the documentation of CalendarInternval in API documentati

[jira] [Resolved] (SPARK-30690) Expose the documentation of CalendarInternval in API documentation

2020-01-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30690. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27412 [https://gi

[jira] [Commented] (SPARK-30693) Document STORED AS Clause of CREATE statement in SQL Reference

2020-01-31 Thread jobit mathew (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027441#comment-17027441 ] jobit mathew commented on SPARK-30693: -- I will work on this > Document STORED AS C

[jira] [Created] (SPARK-30693) Document STORED AS Clause of CREATE statement in SQL Reference

2020-01-31 Thread jobit mathew (Jira)
jobit mathew created SPARK-30693: Summary: Document STORED AS Clause of CREATE statement in SQL Reference Key: SPARK-30693 URL: https://issues.apache.org/jira/browse/SPARK-30693 Project: Spark

[jira] [Created] (SPARK-30692) Mechanism to check that all queries of spark structured-streaming are started in case of multiple sink actions.

2020-01-31 Thread Amit (Jira)
Amit created SPARK-30692: - Summary: Mechanism to check that all queries of spark structured-streaming are started in case of multiple sink actions. Key: SPARK-30692 URL: https://issues.apache.org/jira/browse/SPARK-30692

[jira] [Commented] (SPARK-30687) When reading from a file with pre-defined schema and encountering a single value that is not the same type as that of its column , Spark nullifies the entire row

2020-01-31 Thread pavithra ramachandran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027333#comment-17027333 ] pavithra ramachandran commented on SPARK-30687: --- l would like to work on t

[jira] [Resolved] (SPARK-27747) add a logical plan link in the physical plan

2020-01-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-27747. - Fix Version/s: 3.0.0 Resolution: Fixed > add a logical plan link in the physical plan > -

[jira] [Commented] (SPARK-30688) Spark SQL Unix Timestamp produces incorrect result with unix_timestamp UDF

2020-01-31 Thread Rakesh Raushan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027330#comment-17027330 ] Rakesh Raushan commented on SPARK-30688: I will check this issue   > Spark SQL

[jira] [Created] (SPARK-30691) Add a few main pages

2020-01-31 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-30691: -- Summary: Add a few main pages Key: SPARK-30691 URL: https://issues.apache.org/jira/browse/SPARK-30691 Project: Spark Issue Type: Sub-task Components: D

[jira] [Assigned] (SPARK-30615) normalize the column name in AlterTable

2020-01-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-30615: --- Assignee: Burak Yavuz > normalize the column name in AlterTable > -

[jira] [Resolved] (SPARK-30615) normalize the column name in AlterTable

2020-01-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30615. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27350 [https://gith

[jira] [Commented] (SPARK-30686) Spark 2.4.4 metrics endpoint throwing error

2020-01-31 Thread Behroz Sikander (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027291#comment-17027291 ] Behroz Sikander commented on SPARK-30686: - Could it be linked to [https://github

[jira] [Commented] (SPARK-26448) retain the difference between 0.0 and -0.0

2020-01-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027282#comment-17027282 ] Dongjoon Hyun commented on SPARK-26448: --- Do we need to mark this as a correctness

[jira] [Commented] (SPARK-26021) -0.0 and 0.0 not treated consistently, doesn't match Hive

2020-01-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027280#comment-17027280 ] Dongjoon Hyun commented on SPARK-26021: --- Is that backported or marked as correctne