[jira] [Commented] (SPARK-30049) SQL fails to parse when comment contains an unmatched quote character

[ https://issues.apache.org/jira/browse/SPARK-30049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021832#comment-17021832 ] Oleg Bonar commented on SPARK-30049: [~tgraves], no, i'm not.   > SQL fails to parse when comment

[jira] [Resolved] (SPARK-30607) overlay wrappers for SparkR and PySpark

[ https://issues.apache.org/jira/browse/SPARK-30607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30607. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27325

[jira] [Assigned] (SPARK-30607) overlay wrappers for SparkR and PySpark

[ https://issues.apache.org/jira/browse/SPARK-30607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-30607: Assignee: Maciej Szymkiewicz > overlay wrappers for SparkR and PySpark >

[jira] [Commented] (SPARK-30590) can't use more than five type-safe user-defined aggregation in select statement

[ https://issues.apache.org/jira/browse/SPARK-30590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021823#comment-17021823 ] Hyukjin Kwon commented on SPARK-30590: -- Ah, thanks. I rushed to read. This issue still persists in

[jira] [Updated] (SPARK-30590) can't use more than five type-safe user-defined aggregation in select statement

[ https://issues.apache.org/jira/browse/SPARK-30590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30590: - Affects Version/s: 3.0.0 > can't use more than five type-safe user-defined aggregation in

[jira] [Assigned] (SPARK-30601) Add a Google Maven Central as a primary repository

[ https://issues.apache.org/jira/browse/SPARK-30601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-30601: Assignee: Hyukjin Kwon > Add a Google Maven Central as a primary repository >

[jira] [Resolved] (SPARK-30601) Add a Google Maven Central as a primary repository

[ https://issues.apache.org/jira/browse/SPARK-30601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30601. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27307

[jira] [Reopened] (SPARK-30535) Migrate ALTER TABLE commands to the new resolution framework

[ https://issues.apache.org/jira/browse/SPARK-30535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reopened SPARK-30535: - Assignee: (was: Terry Kim) > Migrate ALTER TABLE commands to the new resolution framework >

[jira] [Commented] (SPARK-30546) Make interval type more future-proofing

[ https://issues.apache.org/jira/browse/SPARK-30546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021806#comment-17021806 ] Kent Yao commented on SPARK-30546: -- Thanks [~dongjoon] > Make interval type more future-proofing >

[jira] [Created] (SPARK-30616) Introduce TTL config option for SQL Parquet Metadata Cache

Yaroslav Tkachenko created SPARK-30616: -- Summary: Introduce TTL config option for SQL Parquet Metadata Cache Key: SPARK-30616 URL: https://issues.apache.org/jira/browse/SPARK-30616 Project: Spark

[jira] [Comment Edited] (SPARK-30590) can't use more than five type-safe user-defined aggregation in select statement

[ https://issues.apache.org/jira/browse/SPARK-30590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021792#comment-17021792 ] Daniel Mantovani edited comment on SPARK-30590 at 1/23/20 5:48 AM: ---

[jira] [Reopened] (SPARK-30590) can't use more than five type-safe user-defined aggregation in select statement

[ https://issues.apache.org/jira/browse/SPARK-30590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Mantovani reopened SPARK-30590: -- [~hyukjin.kwon] You tried with 5 parameters which works, you should try with 6 to get

[jira] [Issue Comment Deleted] (SPARK-30590) can't use more than five type-safe user-defined aggregation in select statement

[ https://issues.apache.org/jira/browse/SPARK-30590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Mantovani updated SPARK-30590: - Comment: was deleted (was: [~hyukjin.kwon] You tried the wrong thing, with 5 parameters

[jira] [Commented] (SPARK-30590) can't use more than five type-safe user-defined aggregation in select statement

[ https://issues.apache.org/jira/browse/SPARK-30590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021791#comment-17021791 ] Daniel Mantovani commented on SPARK-30590: -- [~hyukjin.kwon] You tried the wrong thing, with 5

[jira] [Commented] (SPARK-30615) normalize the column name in AlterTable

[ https://issues.apache.org/jira/browse/SPARK-30615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021783#comment-17021783 ] Terry Kim commented on SPARK-30615: --- [~cloud_fan] Yes, I will work on this. Thanks! > normalize the

[jira] [Commented] (SPARK-30614) The native ALTER COLUMN syntax should change one thing at a time

[ https://issues.apache.org/jira/browse/SPARK-30614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021781#comment-17021781 ] Terry Kim commented on SPARK-30614: --- [~cloud_fan] Yes, I will work on this. Thanks! > The native

[jira] [Updated] (SPARK-30360) Avoid Redact classpath entries in History Server UI

[ https://issues.apache.org/jira/browse/SPARK-30360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-30360: Description: Currently SPARK history server display the classpath entries in the Environment tab with

[jira] [Updated] (SPARK-30360) Avoid Redact classpath entries in History Server UI

[ https://issues.apache.org/jira/browse/SPARK-30360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-30360: Summary: Avoid Redact classpath entries in History Server UI (was: Redact classpath entries in Spark UI)

[jira] [Commented] (SPARK-30275) Add gitlab-ci.yml file for reproducible builds

[ https://issues.apache.org/jira/browse/SPARK-30275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021766#comment-17021766 ] Jim Kleckner commented on SPARK-30275: -- I sent a message to

[jira] [Commented] (SPARK-27871) LambdaVariable should use per-query unique IDs instead of globally unique IDs

[ https://issues.apache.org/jira/browse/SPARK-27871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021765#comment-17021765 ] Wenchen Fan commented on SPARK-27871: - This is done with an optimizer rule so we added a public

[jira] [Updated] (SPARK-30615) normalize the column name in AlterTable

[ https://issues.apache.org/jira/browse/SPARK-30615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-30615: Description: Because of case insensitive resolution, the column name in AlterTable may match the

[jira] [Commented] (SPARK-30615) normalize the column name in AlterTable

[ https://issues.apache.org/jira/browse/SPARK-30615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021763#comment-17021763 ] Wenchen Fan commented on SPARK-30615: - Hi [~imback82], do you have time to work on it? Thanks! >

[jira] [Created] (SPARK-30615) normalize the column name in AlterTable

Wenchen Fan created SPARK-30615: --- Summary: normalize the column name in AlterTable Key: SPARK-30615 URL: https://issues.apache.org/jira/browse/SPARK-30615 Project: Spark Issue Type: New

[jira] [Comment Edited] (SPARK-30614) The native ALTER COLUMN syntax should change one thing at a time

[ https://issues.apache.org/jira/browse/SPARK-30614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021761#comment-17021761 ] Wenchen Fan edited comment on SPARK-30614 at 1/23/20 4:22 AM: -- Hi

[jira] [Commented] (SPARK-30614) The native ALTER COLUMN syntax should change one thing at a time

[ https://issues.apache.org/jira/browse/SPARK-30614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021761#comment-17021761 ] Wenchen Fan commented on SPARK-30614: - Hi [~imback82], do you have time to work in it? thanks! >

[jira] [Created] (SPARK-30614) The native ALTER COLUMN syntax should change one thing at a time

Wenchen Fan created SPARK-30614: --- Summary: The native ALTER COLUMN syntax should change one thing at a time Key: SPARK-30614 URL: https://issues.apache.org/jira/browse/SPARK-30614 Project: Spark

[jira] [Updated] (SPARK-30614) The native ALTER COLUMN syntax should change one thing at a time

[ https://issues.apache.org/jira/browse/SPARK-30614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-30614: Description: Our native ALTER COLUMN syntax is newly added in 3.0 and almost follows the SQL

[jira] [Commented] (SPARK-30613) support hive style REPLACE COLUMN syntax

[ https://issues.apache.org/jira/browse/SPARK-30613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021753#comment-17021753 ] Terry Kim commented on SPARK-30613: --- [~cloud_fan] Yes, I will work on this. Thanks! > support hive

[jira] [Commented] (SPARK-30612) can't resolve qualified column name with v2 tables

[ https://issues.apache.org/jira/browse/SPARK-30612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021752#comment-17021752 ] Terry Kim commented on SPARK-30612: --- [~cloud_fan] Yes, I will work on this. > can't resolve qualified

[jira] [Commented] (SPARK-30613) support hive style REPLACE COLUMN syntax

[ https://issues.apache.org/jira/browse/SPARK-30613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021737#comment-17021737 ] Wenchen Fan commented on SPARK-30613: - Hi [~imback82] do you have time to work on it? Thanks! >

[jira] [Created] (SPARK-30613) support hive style REPLACE COLUMN syntax

Wenchen Fan created SPARK-30613: --- Summary: support hive style REPLACE COLUMN syntax Key: SPARK-30613 URL: https://issues.apache.org/jira/browse/SPARK-30613 Project: Spark Issue Type: New

[jira] [Commented] (SPARK-30612) can't resolve qualified column name with v2 tables

[ https://issues.apache.org/jira/browse/SPARK-30612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021729#comment-17021729 ] Wenchen Fan commented on SPARK-30612: - Hi [~imback82] do you have time to work on it? Thanks >

[jira] [Created] (SPARK-30612) can't resolve qualified column name with v2 tables

Wenchen Fan created SPARK-30612: --- Summary: can't resolve qualified column name with v2 tables Key: SPARK-30612 URL: https://issues.apache.org/jira/browse/SPARK-30612 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-30609) Allow default merge command resolution to be bypassed by DSv2 sources

[ https://issues.apache.org/jira/browse/SPARK-30609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-30609. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27326

[jira] [Assigned] (SPARK-30609) Allow default merge command resolution to be bypassed by DSv2 sources

[ https://issues.apache.org/jira/browse/SPARK-30609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reassigned SPARK-30609: - Assignee: Tathagata Das > Allow default merge command resolution to be bypassed by

[jira] [Comment Edited] (SPARK-30476) NullPointerException when Insert data to hive mongo external table by spark-sql

[ https://issues.apache.org/jira/browse/SPARK-30476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021709#comment-17021709 ] XiongCheng edited comment on SPARK-30476 at 1/23/20 2:49 AM: -

[jira] [Commented] (SPARK-30476) NullPointerException when Insert data to hive mongo external table by spark-sql

[ https://issues.apache.org/jira/browse/SPARK-30476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021709#comment-17021709 ] XiongCheng commented on SPARK-30476: [~hyukjin.kwon]Thanks for your reply~.However, my doubt is why

[jira] [Commented] (SPARK-30229) java.lang.NullPointerException at org.apache.spark.SparkContext.getPreferredLocs(SparkContext.scala:1783)

[ https://issues.apache.org/jira/browse/SPARK-30229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021704#comment-17021704 ] Hyukjin Kwon commented on SPARK-30229: -- [~Ankitraj] were you able to reproduce? [~SeaAndHill] can

[jira] [Resolved] (SPARK-30239) Creating a dataframe with Pandas rather than Numpy datatypes fails

[ https://issues.apache.org/jira/browse/SPARK-30239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30239. -- Resolution: Incomplete Resolving by no feedback from reporter. > Creating a dataframe with

[jira] [Commented] (SPARK-30275) Add gitlab-ci.yml file for reproducible builds

[ https://issues.apache.org/jira/browse/SPARK-30275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021701#comment-17021701 ] Hyukjin Kwon commented on SPARK-30275: -- Can you send an email to the dev list and ask some

[jira] [Commented] (SPARK-30327) Caused by: java.lang.ArrayIndexOutOfBoundsException: -1

[ https://issues.apache.org/jira/browse/SPARK-30327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021700#comment-17021700 ] Hyukjin Kwon commented on SPARK-30327: -- Can you show full, self-contained reproducer? > Caused by:

[jira] [Resolved] (SPARK-30328) Fail to write local files with RDD.saveTextFile when setting the incorrect Hadoop configuration files

[ https://issues.apache.org/jira/browse/SPARK-30328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30328. -- Resolution: Invalid > Fail to write local files with RDD.saveTextFile when setting the

[jira] [Commented] (SPARK-30328) Fail to write local files with RDD.saveTextFile when setting the incorrect Hadoop configuration files

[ https://issues.apache.org/jira/browse/SPARK-30328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021699#comment-17021699 ] Hyukjin Kwon commented on SPARK-30328: -- If Hadoop configuration path is set, it should be correct.

[jira] [Resolved] (SPARK-30332) When running sql query with limit catalyst throw StackOverFlow exception

[ https://issues.apache.org/jira/browse/SPARK-30332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30332. -- Resolution: Incomplete > When running sql query with limit catalyst throw StackOverFlow

[jira] [Resolved] (SPARK-30442) Write mode ignored when using CodecStreams

[ https://issues.apache.org/jira/browse/SPARK-30442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30442. -- Resolution: Incomplete Resolving due to no feedback from the reporter. > Write mode ignored

[jira] [Commented] (SPARK-30444) The same job will be computated for many times when using Dataset.show()

[ https://issues.apache.org/jira/browse/SPARK-30444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021697#comment-17021697 ] Hyukjin Kwon commented on SPARK-30444: -- [~aman_omer] have you made some progresses on this? > The

[jira] [Updated] (SPARK-30462) Structured Streaming _spark_metadata fills up Spark Driver memory when having lots of objects

[ https://issues.apache.org/jira/browse/SPARK-30462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30462: - Priority: Major (was: Critical) > Structured Streaming _spark_metadata fills up Spark Driver

[jira] [Resolved] (SPARK-30473) PySpark enum subclass crashes when used inside UDF

[ https://issues.apache.org/jira/browse/SPARK-30473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30473. -- Resolution: Cannot Reproduce > PySpark enum subclass crashes when used inside UDF >

[jira] [Commented] (SPARK-30473) PySpark enum subclass crashes when used inside UDF

[ https://issues.apache.org/jira/browse/SPARK-30473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021695#comment-17021695 ] Hyukjin Kwon commented on SPARK-30473: -- This was fixed in the upstream master by upgrading

[jira] [Resolved] (SPARK-30476) NullPointerException when Insert data to hive mongo external table by spark-sql

[ https://issues.apache.org/jira/browse/SPARK-30476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30476. -- Resolution: Not A Problem So, it's an issue in MongoDB connector, as you described. It seems

[jira] [Resolved] (SPARK-30483) Job History does not show pool properties table

[ https://issues.apache.org/jira/browse/SPARK-30483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30483. -- Resolution: Duplicate > Job History does not show pool properties table >

[jira] [Updated] (SPARK-30556) Copy sparkContext.localproperties to child thread inSubqueryExec.executionContext

[ https://issues.apache.org/jira/browse/SPARK-30556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30556: -- Summary: Copy sparkContext.localproperties to child thread inSubqueryExec.executionContext

[jira] [Resolved] (SPARK-30487) Hive MetaException

[ https://issues.apache.org/jira/browse/SPARK-30487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30487. -- Resolution: Incomplete > Hive MetaException > --- > > Key:

[jira] [Resolved] (SPARK-30484) Job History Storage Tab does not display RDD Table

[ https://issues.apache.org/jira/browse/SPARK-30484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30484. -- Resolution: Not A Problem > Job History Storage Tab does not display RDD Table >

[jira] [Commented] (SPARK-30488) Deadlock between block-manager-slave-async-thread-pool and spark context cleaner

[ https://issues.apache.org/jira/browse/SPARK-30488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021692#comment-17021692 ] Hyukjin Kwon commented on SPARK-30488: -- Is that the only place to create? Can you show full

[jira] [Assigned] (SPARK-30556) SubqueryExec passes local properties to SubqueryExec.executionContext

[ https://issues.apache.org/jira/browse/SPARK-30556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-30556: - Assignee: Ajith S > SubqueryExec passes local properties to

[jira] [Resolved] (SPARK-30513) Question about spark on k8s

[ https://issues.apache.org/jira/browse/SPARK-30513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30513. -- Resolution: Invalid Please ask qeustions at dev mailing list or stackoverflow. See

[jira] [Resolved] (SPARK-30556) SubqueryExec passes local properties to SubqueryExec.executionContext

[ https://issues.apache.org/jira/browse/SPARK-30556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30556. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27267

[jira] [Resolved] (SPARK-30526) Can I translate Spark documents into Chinese ？

[ https://issues.apache.org/jira/browse/SPARK-30526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30526. -- Resolution: Won't Fix I think it's better to do it in a separate thridparty repository. Not

[jira] [Updated] (SPARK-30529) Improve error messages when Executor dies before registering with driver

[ https://issues.apache.org/jira/browse/SPARK-30529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30529: - Description: currently when you give a bad configuration for accelerator aware scheduling to

[jira] [Resolved] (SPARK-30542) Two Spark structured streaming jobs cannot write to same base path

[ https://issues.apache.org/jira/browse/SPARK-30542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30542. -- Resolution: Invalid Please ask questions to mailing list. You could have a better answer. See

[jira] [Resolved] (SPARK-30550) Random pyspark-shell applications being generated

[ https://issues.apache.org/jira/browse/SPARK-30550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30550. -- Resolution: Cannot Reproduce Seems like no one can reproduce this. > Random pyspark-shell

[jira] [Commented] (SPARK-30557) Add public documentation for SPARK_SUBMIT_OPTS

[ https://issues.apache.org/jira/browse/SPARK-30557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021684#comment-17021684 ] Hyukjin Kwon commented on SPARK-30557: -- Yup, and {{spark.driver.extraJavaOptions}} and

[jira] [Commented] (SPARK-30561) start spark applications without a 30second startup penalty

[ https://issues.apache.org/jira/browse/SPARK-30561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021682#comment-17021682 ] Hyukjin Kwon commented on SPARK-30561: -- Is any of them safe to remove? > start spark applications

[jira] [Commented] (SPARK-30577) StorageLevel.DISK_ONLY_2 causes the data loss

[ https://issues.apache.org/jira/browse/SPARK-30577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021679#comment-17021679 ] Hyukjin Kwon commented on SPARK-30577: -- Spark 2.3 is EOL. Can you try it in higher versions? Also,

[jira] [Resolved] (SPARK-30577) StorageLevel.DISK_ONLY_2 causes the data loss

[ https://issues.apache.org/jira/browse/SPARK-30577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30577. -- Resolution: Incomplete I am resolving this as incomplete as it targets EOL release. >

[jira] [Commented] (SPARK-30580) Why can PySpark persist data only in serialised format?

[ https://issues.apache.org/jira/browse/SPARK-30580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021677#comment-17021677 ] Hyukjin Kwon commented on SPARK-30580: -- Let's ask questions to mailing list. You could have a

[jira] [Resolved] (SPARK-30580) Why can PySpark persist data only in serialised format?

[ https://issues.apache.org/jira/browse/SPARK-30580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30580. -- Resolution: Invalid > Why can PySpark persist data only in serialised format? >

[jira] [Resolved] (SPARK-30585) scalatest fails for Apache Spark SQL project

[ https://issues.apache.org/jira/browse/SPARK-30585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30585. -- Resolution: Incomplete > scalatest fails for Apache Spark SQL project >

[jira] [Commented] (SPARK-30585) scalatest fails for Apache Spark SQL project

[ https://issues.apache.org/jira/browse/SPARK-30585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021675#comment-17021675 ] Hyukjin Kwon commented on SPARK-30585: -- and please just don't copy and paste the logs. You should

[jira] [Commented] (SPARK-30590) can't use more than five type-safe user-defined aggregation in select statement

[ https://issues.apache.org/jira/browse/SPARK-30590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021674#comment-17021674 ] Hyukjin Kwon commented on SPARK-30590: -- Seems fixed in the master: {code} scala>

[jira] [Resolved] (SPARK-30590) can't use more than five type-safe user-defined aggregation in select statement

[ https://issues.apache.org/jira/browse/SPARK-30590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30590. -- Resolution: Cannot Reproduce > can't use more than five type-safe user-defined aggregation in

[jira] [Updated] (SPARK-30602) SPIP: Support push-based shuffle to improve shuffle efficiency

[ https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Shen updated SPARK-30602: - Description: In a large deployment of a Spark compute infrastructure, Spark shuffle is becoming a

[jira] [Updated] (SPARK-30597) Unable to load properties fine in SparkStandalone HDFS mode

[ https://issues.apache.org/jira/browse/SPARK-30597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30597: - Description: We run the spark application in Yarn HDFS/NFS/WebHDFS and standalone HDFS/NFS

[jira] [Updated] (SPARK-30597) Unable to load properties fine in SparkStandalone HDFS mode

[ https://issues.apache.org/jira/browse/SPARK-30597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30597: - Description: We run the spark application in Yarn HDFS/NFS/WebHDFS and standalone HDFS/NFS

[jira] [Commented] (SPARK-30602) SPIP: Support push-based shuffle to improve shuffle efficiency

[ https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021672#comment-17021672 ] Hyukjin Kwon commented on SPARK-30602: -- Can you send the email to the dev list to discuss? If you

[jira] [Commented] (SPARK-30608) Postgres Column Interval converts to string and cant be written back to postgres

[ https://issues.apache.org/jira/browse/SPARK-30608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021669#comment-17021669 ] Hyukjin Kwon commented on SPARK-30608: -- I don't see interval type conversions are supported between

[jira] [Updated] (SPARK-30602) SPIP: Support push-based shuffle to improve shuffle efficiency

[ https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30602: - Summary: SPIP: Support push-based shuffle to improve shuffle efficiency (was: Support

[jira] [Resolved] (SPARK-30608) Postgres Column Interval converts to string and cant be written back to postgres

[ https://issues.apache.org/jira/browse/SPARK-30608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30608. -- Resolution: Invalid > Postgres Column Interval converts to string and cant be written back to

[jira] [Resolved] (SPARK-30463) Move test cases for 'pandas' sub-package

[ https://issues.apache.org/jira/browse/SPARK-30463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30463. -- Resolution: Later Let me take a look for this later. Fortunately, the tests are grouped in

[jira] [Updated] (SPARK-30531) Duplicate query plan on Spark UI SQL page

[ https://issues.apache.org/jira/browse/SPARK-30531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-30531: - Issue Type: Improvement (was: Bug) Priority: Minor (was: Major) > Duplicate query plan

[jira] [Resolved] (SPARK-30531) Duplicate query plan on Spark UI SQL page

[ https://issues.apache.org/jira/browse/SPARK-30531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-30531. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27238

[jira] [Assigned] (SPARK-30531) Duplicate query plan on Spark UI SQL page

[ https://issues.apache.org/jira/browse/SPARK-30531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-30531: Assignee: Enrico Minack > Duplicate query plan on Spark UI SQL page >

[jira] [Updated] (SPARK-29701) Different answers when empty input given in GROUPING SETS

[ https://issues.apache.org/jira/browse/SPARK-29701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29701: -- Affects Version/s: 2.4.0 2.4.1 2.4.2

[jira] [Updated] (SPARK-29701) Different answers when empty input given in GROUPING SETS

[ https://issues.apache.org/jira/browse/SPARK-29701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29701: -- Affects Version/s: 2.4.4 > Different answers when empty input given in GROUPING SETS >

[jira] [Resolved] (SPARK-30611) Update testthat dependency

[ https://issues.apache.org/jira/browse/SPARK-30611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30611. -- Resolution: Duplicate > Update testthat dependency > -- > >

[jira] [Updated] (SPARK-28801) Document SELECT statement in SQL Reference.

[ https://issues.apache.org/jira/browse/SPARK-28801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-28801: - Priority: Minor (was: Major) > Document SELECT statement in SQL Reference. >

[jira] [Resolved] (SPARK-28801) Document SELECT statement in SQL Reference.

[ https://issues.apache.org/jira/browse/SPARK-28801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-28801. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27216

[jira] [Assigned] (SPARK-28801) Document SELECT statement in SQL Reference.

[ https://issues.apache.org/jira/browse/SPARK-28801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-28801: Assignee: Dilip Biswal > Document SELECT statement in SQL Reference. >

[jira] [Resolved] (SPARK-30574) Document GROUP BY Clause of SELECT statement in SQL Reference.

[ https://issues.apache.org/jira/browse/SPARK-30574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-30574. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27283

[jira] [Updated] (SPARK-30574) Document GROUP BY Clause of SELECT statement in SQL Reference.

[ https://issues.apache.org/jira/browse/SPARK-30574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-30574: - Priority: Minor (was: Major) > Document GROUP BY Clause of SELECT statement in SQL Reference.

[jira] [Assigned] (SPARK-30574) Document GROUP BY Clause of SELECT statement in SQL Reference.

[ https://issues.apache.org/jira/browse/SPARK-30574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-30574: Assignee: Dilip Biswal > Document GROUP BY Clause of SELECT statement in SQL Reference.

[jira] [Updated] (SPARK-26132) Remove support for Scala 2.11 in Spark 3.0.0

[ https://issues.apache.org/jira/browse/SPARK-26132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-26132: - Docs Text: Scala 2.11 support is removed in Apache Spark 3.0.0. > Remove support for Scala 2.11

[jira] [Updated] (SPARK-26132) Remove support for Scala 2.11 in Spark 3.0.0

[ https://issues.apache.org/jira/browse/SPARK-26132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-26132: - Labels: release-notes (was: ) > Remove support for Scala 2.11 in Spark 3.0.0 >

[jira] [Commented] (SPARK-26154) Stream-stream joins - left outer join gives inconsistent output

[ https://issues.apache.org/jira/browse/SPARK-26154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021627#comment-17021627 ] Jungtaek Lim commented on SPARK-26154: -- Leaving information why this issue cannot be easily ported

[jira] [Assigned] (SPARK-30606) Applying the `like` function with 2 parameters fails

[ https://issues.apache.org/jira/browse/SPARK-30606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-30606: - Assignee: Maxim Gekk > Applying the `like` function with 2 parameters fails >

[jira] [Resolved] (SPARK-30606) Applying the `like` function with 2 parameters fails

[ https://issues.apache.org/jira/browse/SPARK-30606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30606. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27323

[jira] [Created] (SPARK-30611) Update testthat dependency

Maciej Szymkiewicz created SPARK-30611: -- Summary: Update testthat dependency Key: SPARK-30611 URL: https://issues.apache.org/jira/browse/SPARK-30611 Project: Spark Issue Type:

[jira] [Created] (SPARK-30610) spark worker graceful shutdown

t oo created SPARK-30610: Summary: spark worker graceful shutdown Key: SPARK-30610 URL: https://issues.apache.org/jira/browse/SPARK-30610 Project: Spark Issue Type: Improvement Components:

[jira] [Commented] (SPARK-29175) Make maven central repository in IsolatedClientLoader configurable

[ https://issues.apache.org/jira/browse/SPARK-29175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021586#comment-17021586 ] Reynold Xin commented on SPARK-29175: - I think the config should be more clear, e.g.

  1   2   3   >