[jira] [Updated] (SPARK-29776) rpad returning invalid value when parameter is empty

[ https://issues.apache.org/jira/browse/SPARK-29776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29776: - Description: As per rpad definition rpad rpad(str, len, pad) - Returns str, right-padded with

[jira] [Resolved] (SPARK-29784) Built in function trim is not compatible in 3.0 with previous version

[ https://issues.apache.org/jira/browse/SPARK-29784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29784. -- Resolution: Invalid > Built in function trim is not compatible in 3.0 with previous version >

[jira] [Updated] (SPARK-29784) Built in function trim is not compatible in 3.0 with previous version

[ https://issues.apache.org/jira/browse/SPARK-29784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29784: - Description: SELECT trim('SL', 'SSparkSQLS'); returns empty in Spark 3.0 where as in 2.4 and

[jira] [Commented] (SPARK-29797) Read key-value metadata in Parquet files written by Apache Arrow

[ https://issues.apache.org/jira/browse/SPARK-29797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969914#comment-16969914 ] Hyukjin Kwon commented on SPARK-29797: -- Not sure. How should they be read into DataFrame? > Read

[jira] [Issue Comment Deleted] (SPARK-29776) rpad returning invalid value when parameter is empty

[ https://issues.apache.org/jira/browse/SPARK-29776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit raj boudh updated SPARK-29776: Comment: was deleted (was: i will start checking this issue.) > rpad returning invalid

[jira] [Commented] (SPARK-29776) rpad returning invalid value when parameter is empty

[ https://issues.apache.org/jira/browse/SPARK-29776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969843#comment-16969843 ] Ankit Raj Boudh commented on SPARK-29776: - i will submit the PR soon > rpad returning invalid

[jira] [Resolved] (SPARK-29772) Add withNamespace method in test

[ https://issues.apache.org/jira/browse/SPARK-29772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29772. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26411

[jira] [Assigned] (SPARK-29772) Add withNamespace method in test

[ https://issues.apache.org/jira/browse/SPARK-29772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-29772: --- Assignee: ulysses you > Add withNamespace method in test >

[jira] [Updated] (SPARK-29672) remove python2 tests and test infra

[ https://issues.apache.org/jira/browse/SPARK-29672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shane Knapp updated SPARK-29672: Summary: remove python2 tests and test infra (was: remove python2 test from python/run-tests.py)

[jira] [Updated] (SPARK-29651) Incorrect parsing of interval seconds fraction

[ https://issues.apache.org/jira/browse/SPARK-29651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29651: -- Labels: correctness (was: ) > Incorrect parsing of interval seconds fraction >

[jira] [Created] (SPARK-29799) Split a kafka partition into multiple KafkaRDD partitions in the kafka external plugin for Spark Streaming

zengrui created SPARK-29799: --- Summary: Split a kafka partition into multiple KafkaRDD partitions in the kafka external plugin for Spark Streaming Key: SPARK-29799 URL: https://issues.apache.org/jira/browse/SPARK-29799

[jira] [Created] (SPARK-29798) Infers bytes as binary type in Python 3 at PySpark

Hyukjin Kwon created SPARK-29798: Summary: Infers bytes as binary type in Python 3 at PySpark Key: SPARK-29798 URL: https://issues.apache.org/jira/browse/SPARK-29798 Project: Spark Issue

[jira] [Resolved] (SPARK-29787) Move method add/subtract/negate from CalendarInterval to IntervalUtils

[ https://issues.apache.org/jira/browse/SPARK-29787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29787. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26423

[jira] [Assigned] (SPARK-29787) Move method add/subtract/negate from CalendarInterval to IntervalUtils

[ https://issues.apache.org/jira/browse/SPARK-29787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-29787: --- Assignee: Kent Yao > Move method add/subtract/negate from CalendarInterval to

[jira] [Updated] (SPARK-29769) Spark SQL cannot handle "exists/not exists" condition when using "JOIN"

[ https://issues.apache.org/jira/browse/SPARK-29769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-29769: -- Description: In origin master, we can'y run sql use `EXISTS/NOT EXISTS` in Join's on condition:

[jira] [Updated] (SPARK-29769) Spark SQL cannot handle "exists/not exists" condition when using "JOIN"

[ https://issues.apache.org/jira/browse/SPARK-29769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-29769: -- Description: In origin master, we can'y run sql use `EXISTS/NOT EXISTS` in Join's on condition:

[jira] [Assigned] (SPARK-21869) A cached Kafka producer should not be closed if any task is using it.

[ https://issues.apache.org/jira/browse/SPARK-21869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-21869: -- Assignee: Gabor Somogyi > A cached Kafka producer should not be

[jira] [Resolved] (SPARK-21869) A cached Kafka producer should not be closed if any task is using it.

[ https://issues.apache.org/jira/browse/SPARK-21869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-21869. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Commented] (SPARK-29797) Read key-value metadata in Parquet files written by Apache Arrow

[ https://issues.apache.org/jira/browse/SPARK-29797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969666#comment-16969666 ] Isaac Myers commented on SPARK-29797: - The "code" block is garbage. I attached the cpp file

[jira] [Updated] (SPARK-29797) Read key-value metadata in Parquet files written by Apache Arrow

[ https://issues.apache.org/jira/browse/SPARK-29797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Isaac Myers updated SPARK-29797: Attachment: minimal_working_example.cpp > Read key-value metadata in Parquet files written by

[jira] [Created] (SPARK-29797) Read key-value metadata in Parquet files written by Apache Arrow

Isaac Myers created SPARK-29797: --- Summary: Read key-value metadata in Parquet files written by Apache Arrow Key: SPARK-29797 URL: https://issues.apache.org/jira/browse/SPARK-29797 Project: Spark

[jira] [Updated] (SPARK-29794) Column level compression

[ https://issues.apache.org/jira/browse/SPARK-29794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Vyas updated SPARK-29794: - Priority: Minor (was: Major) > Column level compression > > >

[jira] [Updated] (SPARK-29794) Column level compression

[ https://issues.apache.org/jira/browse/SPARK-29794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Vyas updated SPARK-29794: - Priority: Major (was: Minor) > Column level compression > > >

[jira] [Commented] (SPARK-29106) Add jenkins arm test for spark

[ https://issues.apache.org/jira/browse/SPARK-29106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969615#comment-16969615 ] Shane Knapp commented on SPARK-29106: - just uploaded the pip requirements.txt file that i used to

[jira] [Updated] (SPARK-29106) Add jenkins arm test for spark

[ https://issues.apache.org/jira/browse/SPARK-29106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shane Knapp updated SPARK-29106: Attachment: arm-python36.txt > Add jenkins arm test for spark > -- >

[jira] [Resolved] (SPARK-22340) pyspark setJobGroup doesn't match java threads

[ https://issues.apache.org/jira/browse/SPARK-22340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22340. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 24898

[jira] [Assigned] (SPARK-22340) pyspark setJobGroup doesn't match java threads

[ https://issues.apache.org/jira/browse/SPARK-22340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-22340: Assignee: Hyukjin Kwon > pyspark setJobGroup doesn't match java threads >

[jira] [Assigned] (SPARK-29781) Override SBT Jackson-databind dependency like Maven

[ https://issues.apache.org/jira/browse/SPARK-29781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29781: - Assignee: Dongjoon Hyun > Override SBT Jackson-databind dependency like Maven >

[jira] [Resolved] (SPARK-29781) Override SBT Jackson-databind dependency like Maven

[ https://issues.apache.org/jira/browse/SPARK-29781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29781. --- Fix Version/s: 2.4.5 Resolution: Fixed Issue resolved by pull request 26417

[jira] [Commented] (SPARK-29335) Cost Based Optimizer stats are not used while evaluating query plans in Spark Sql

[ https://issues.apache.org/jira/browse/SPARK-29335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969494#comment-16969494 ] venkata yerubandi commented on SPARK-29335: --- Srini  Even i am having the issue now , were you

[jira] [Resolved] (SPARK-29796) HiveExternalCatalogVersionsSuite` should ignore preview release

[ https://issues.apache.org/jira/browse/SPARK-29796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29796. --- Fix Version/s: 3.0.0 2.4.5 Resolution: Fixed Issue resolved by

[jira] [Assigned] (SPARK-29796) HiveExternalCatalogVersionsSuite` should ignore preview release

[ https://issues.apache.org/jira/browse/SPARK-29796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29796: - Assignee: Dongjoon Hyun > HiveExternalCatalogVersionsSuite` should ignore preview

[jira] [Created] (SPARK-29796) HiveExternalCatalogVersionsSuite` should ignore preview release

Dongjoon Hyun created SPARK-29796: - Summary: HiveExternalCatalogVersionsSuite` should ignore preview release Key: SPARK-29796 URL: https://issues.apache.org/jira/browse/SPARK-29796 Project: Spark

[jira] [Comment Edited] (SPARK-27990) Provide a way to recursively load data from datasource

[ https://issues.apache.org/jira/browse/SPARK-27990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969462#comment-16969462 ] Nicholas Chammas edited comment on SPARK-27990 at 11/7/19 5:54 PM: ---

[jira] [Commented] (SPARK-27990) Provide a way to recursively load data from datasource

[ https://issues.apache.org/jira/browse/SPARK-27990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969462#comment-16969462 ] Nicholas Chammas commented on SPARK-27990: -- Are there any docs for this new option? I can't

[jira] [Created] (SPARK-29795) Possible 'leak' of Metrics with dropwizard metrics 4.x

Sean R. Owen created SPARK-29795: Summary: Possible 'leak' of Metrics with dropwizard metrics 4.x Key: SPARK-29795 URL: https://issues.apache.org/jira/browse/SPARK-29795 Project: Spark Issue

[jira] [Created] (SPARK-29794) Column level compression

Anirudh Vyas created SPARK-29794: Summary: Column level compression Key: SPARK-29794 URL: https://issues.apache.org/jira/browse/SPARK-29794 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-29793) Display plan evolve history in AQE

Wei Xue created SPARK-29793: --- Summary: Display plan evolve history in AQE Key: SPARK-29793 URL: https://issues.apache.org/jira/browse/SPARK-29793 Project: Spark Issue Type: Task

[jira] [Assigned] (SPARK-29792) SQL metrics cannot be updated to subqueries in AQE

[ https://issues.apache.org/jira/browse/SPARK-29792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-29792: --- Assignee: Ke Jia > SQL metrics cannot be updated to subqueries in AQE >

[jira] [Created] (SPARK-29792) SQL metrics cannot be updated to subqueries in AQE

Wei Xue created SPARK-29792: --- Summary: SQL metrics cannot be updated to subqueries in AQE Key: SPARK-29792 URL: https://issues.apache.org/jira/browse/SPARK-29792 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-29791) Add a spark config to allow user to use executor cores virtually.

[ https://issues.apache.org/jira/browse/SPARK-29791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zengrui updated SPARK-29791: Description: We can config the executor cores by "spark.executor.cores". For example, if we config 8

[jira] [Created] (SPARK-29791) Add a spark config to allow user to use executor cores virtually.

zengrui created SPARK-29791: --- Summary: Add a spark config to allow user to use executor cores virtually. Key: SPARK-29791 URL: https://issues.apache.org/jira/browse/SPARK-29791 Project: Spark

[jira] [Commented] (SPARK-29764) Error on Serializing POJO with java datetime property to a Parquet file

[ https://issues.apache.org/jira/browse/SPARK-29764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969301#comment-16969301 ] Felix Kizhakkel Jose commented on SPARK-29764: -- [~hyukjin.kwon] Could you please help me

[jira] [Commented] (SPARK-29784) Built in function trim is not compatible in 3.0 with previous version

[ https://issues.apache.org/jira/browse/SPARK-29784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969279#comment-16969279 ] Yuming Wang commented on SPARK-29784: - !image-2019-11-07-22-06-48-156.png! > Built in function trim

[jira] [Updated] (SPARK-29784) Built in function trim is not compatible in 3.0 with previous version

[ https://issues.apache.org/jira/browse/SPARK-29784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-29784: Attachment: image-2019-11-07-22-06-48-156.png > Built in function trim is not compatible in 3.0

[jira] [Commented] (SPARK-29784) Built in function trim is not compatible in 3.0 with previous version

[ https://issues.apache.org/jira/browse/SPARK-29784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969272#comment-16969272 ] Yuming Wang commented on SPARK-29784: - Thank you [~pavithraramachandran]. We changed our

[jira] [Created] (SPARK-29790) Add notes about port being required for Kubernetes API URL when set as master

Emil Sandstø created SPARK-29790: Summary: Add notes about port being required for Kubernetes API URL when set as master Key: SPARK-29790 URL: https://issues.apache.org/jira/browse/SPARK-29790

[jira] [Commented] (SPARK-25466) Documentation does not specify how to set Kafka consumer cache capacity for SS

[ https://issues.apache.org/jira/browse/SPARK-25466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969262#comment-16969262 ] Gabor Somogyi commented on SPARK-25466: --- Well, if you think there is a bug in Spark please collect

[jira] [Created] (SPARK-29789) should not parse the bucket column name again when creating v2 tables

Wenchen Fan created SPARK-29789: --- Summary: should not parse the bucket column name again when creating v2 tables Key: SPARK-29789 URL: https://issues.apache.org/jira/browse/SPARK-29789 Project: Spark

[jira] [Commented] (SPARK-29788) Fix the typo errors in the SQL reference document merges

[ https://issues.apache.org/jira/browse/SPARK-29788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969226#comment-16969226 ] jobit mathew commented on SPARK-29788: -- I will work on this. > Fix the typo errors in the SQL

[jira] [Created] (SPARK-29788) Fix the typo errors in the SQL reference document merges

jobit mathew created SPARK-29788: Summary: Fix the typo errors in the SQL reference document merges Key: SPARK-29788 URL: https://issues.apache.org/jira/browse/SPARK-29788 Project: Spark

[jira] [Resolved] (SPARK-29757) Move calendar interval constants together

[ https://issues.apache.org/jira/browse/SPARK-29757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29757. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26399

[jira] [Assigned] (SPARK-29757) Move calendar interval constants together

[ https://issues.apache.org/jira/browse/SPARK-29757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-29757: --- Assignee: Kent Yao > Move calendar interval constants together >

[jira] [Commented] (SPARK-29760) Document VALUES statement in SQL Reference.

[ https://issues.apache.org/jira/browse/SPARK-29760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969167#comment-16969167 ] Ankit Raj Boudh commented on SPARK-29760: - @Sean R. Owen , i will raise PR for this. > Document

[jira] [Created] (SPARK-29787) Move method add/subtract/negate from CalendarInterval to IntervalUtils

Kent Yao created SPARK-29787: Summary: Move method add/subtract/negate from CalendarInterval to IntervalUtils Key: SPARK-29787 URL: https://issues.apache.org/jira/browse/SPARK-29787 Project: Spark