[jira] [Updated] (SPARK-30162) Filter is not being pushed down for Parquet files

[ https://issues.apache.org/jira/browse/SPARK-30162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nasir Ali updated SPARK-30162: -- Description: Filters are not pushed down in Spark 3.0 preview. Also the output of "explain" method

[jira] [Updated] (SPARK-30162) Filter is not being pushed down for Parquet files

[ https://issues.apache.org/jira/browse/SPARK-30162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nasir Ali updated SPARK-30162: -- Description: Filters are not pushed down in Spark 3.0 preview. Also the output of "explain" method

[jira] [Updated] (SPARK-30162) Filter is not being pushed down for Parquet files

[ https://issues.apache.org/jira/browse/SPARK-30162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nasir Ali updated SPARK-30162: -- Summary: Filter is not being pushed down for Parquet files (was: Filters is not being pushed down

[jira] [Created] (SPARK-30162) Filters is not being pushed down for Parquet files

Nasir Ali created SPARK-30162: - Summary: Filters is not being pushed down for Parquet files Key: SPARK-30162 URL: https://issues.apache.org/jira/browse/SPARK-30162 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-30141) Fix QueryTest.checkAnswer usage

[ https://issues.apache.org/jira/browse/SPARK-30141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-30141. - Resolution: Duplicate > Fix QueryTest.checkAnswer usage > --- > >

[jira] [Created] (SPARK-30161) Uncaught Analysis Exception in Spark-Shell

Aman Omer created SPARK-30161: - Summary: Uncaught Analysis Exception in Spark-Shell Key: SPARK-30161 URL: https://issues.apache.org/jira/browse/SPARK-30161 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-30147) Trim the string when cast string type to booleans

[ https://issues.apache.org/jira/browse/SPARK-30147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-30147. -- Fix Version/s: 3.0.0 Assignee: Kent Yao Resolution: Fixed Resolved by

[jira] [Commented] (SPARK-30148) Optimize writing plans if there is an analysis exception

[ https://issues.apache.org/jira/browse/SPARK-30148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990309#comment-16990309 ] Aman Omer commented on SPARK-30148: --- [~hyukjin.kwon] can you assign this JIRA to me. ID: aman_omer >

[jira] [Assigned] (SPARK-30156) Upgrade Jersey from 2.29 to 2.29.1

[ https://issues.apache.org/jira/browse/SPARK-30156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-30156: - Assignee: Dongjoon Hyun > Upgrade Jersey from 2.29 to 2.29.1 >

[jira] [Resolved] (SPARK-30156) Upgrade Jersey from 2.29 to 2.29.1

[ https://issues.apache.org/jira/browse/SPARK-30156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30156. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26785

[jira] [Resolved] (SPARK-30157) Upgrade Apache HttpCore from 4.4.10 to 4.4.12

[ https://issues.apache.org/jira/browse/SPARK-30157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30157. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26786

[jira] [Assigned] (SPARK-30157) Upgrade Apache HttpCore from 4.4.10 to 4.4.12

[ https://issues.apache.org/jira/browse/SPARK-30157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-30157: Assignee: Dongjoon Hyun > Upgrade Apache HttpCore from 4.4.10 to 4.4.12 >

[jira] [Resolved] (SPARK-30148) Optimize writing plans if there is an analysis exception

[ https://issues.apache.org/jira/browse/SPARK-30148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30148. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26778

[jira] [Assigned] (SPARK-30148) Optimize writing plans if there is an analysis exception

[ https://issues.apache.org/jira/browse/SPARK-30148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-30148: Assignee: jobit mathew > Optimize writing plans if there is an analysis exception >

[jira] [Created] (SPARK-30160) Extend ExecutorPodAllocator for arbitrary CRDs

Ilan Filonenko created SPARK-30160: -- Summary: Extend ExecutorPodAllocator for arbitrary CRDs Key: SPARK-30160 URL: https://issues.apache.org/jira/browse/SPARK-30160 Project: Spark Issue

[jira] [Updated] (SPARK-30160) Extend ExecutorPodAllocator for arbitrary CRDs

[ https://issues.apache.org/jira/browse/SPARK-30160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilan Filonenko updated SPARK-30160: --- Priority: Minor (was: Major) > Extend ExecutorPodAllocator for arbitrary CRDs >

[jira] [Resolved] (SPARK-30155) Resolve ambiguous overload from RegexParsers in 2.13

[ https://issues.apache.org/jira/browse/SPARK-30155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30155. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26784

[jira] [Created] (SPARK-30159) Fix the method calls of QueryTest.checkAnswer

Gengliang Wang created SPARK-30159: -- Summary: Fix the method calls of QueryTest.checkAnswer Key: SPARK-30159 URL: https://issues.apache.org/jira/browse/SPARK-30159 Project: Spark Issue

[jira] [Commented] (SPARK-28817) Support standard Javadoc packaging to allow automatic javadoc location settings

[ https://issues.apache.org/jira/browse/SPARK-28817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990168#comment-16990168 ] Marc Le Bihan commented on SPARK-28817: --- I think that the main _pom.xml_ has the correct plugin,

[jira] [Created] (SPARK-30158) Resolve Array + reference type compile problems in 2.13, with sc.parallelize

Sean R. Owen created SPARK-30158: Summary: Resolve Array + reference type compile problems in 2.13, with sc.parallelize Key: SPARK-30158 URL: https://issues.apache.org/jira/browse/SPARK-30158

[jira] [Created] (SPARK-30157) Upgrade Apache HttpCore from 4.4.10 to 4.4.12

Dongjoon Hyun created SPARK-30157: - Summary: Upgrade Apache HttpCore from 4.4.10 to 4.4.12 Key: SPARK-30157 URL: https://issues.apache.org/jira/browse/SPARK-30157 Project: Spark Issue Type:

[jira] [Created] (SPARK-30156) Upgrade Jersey from 2.29 to 2.29.1

Dongjoon Hyun created SPARK-30156: - Summary: Upgrade Jersey from 2.29 to 2.29.1 Key: SPARK-30156 URL: https://issues.apache.org/jira/browse/SPARK-30156 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-30051) Clean up hadoop-3.2 transitive dependencies

[ https://issues.apache.org/jira/browse/SPARK-30051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30051: -- Parent: SPARK-29194 Issue Type: Sub-task (was: Improvement) > Clean up hadoop-3.2

[jira] [Updated] (SPARK-29957) Reset MiniKDC's default enctypes to fit jdk8/jdk11

[ https://issues.apache.org/jira/browse/SPARK-29957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29957: -- Parent: SPARK-29194 Issue Type: Sub-task (was: Improvement) > Reset MiniKDC's

[jira] [Assigned] (SPARK-30035) Upgrade to Apache Commons Lang 3.9

[ https://issues.apache.org/jira/browse/SPARK-30035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-30035: - Assignee: Dongjoon Hyun > Upgrade to Apache Commons Lang 3.9 >

[jira] [Updated] (SPARK-30152) Enable Hadoop-2.7/JDK11 build at GitHub Action

[ https://issues.apache.org/jira/browse/SPARK-30152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30152: -- Parent: SPARK-29194 Issue Type: Sub-task (was: Improvement) > Enable

[jira] [Resolved] (SPARK-30152) Enable Hadoop-2.7/JDK11 build at GitHub Action

[ https://issues.apache.org/jira/browse/SPARK-30152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30152. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26782

[jira] [Assigned] (SPARK-30152) Enable Hadoop-2.7/JDK11 build at GitHub Action

[ https://issues.apache.org/jira/browse/SPARK-30152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-30152: - Assignee: Dongjoon Hyun > Enable Hadoop-2.7/JDK11 build at GitHub Action >

[jira] [Created] (SPARK-30155) Resolve ambiguous overload from RegexParsers in 2.13

Sean R. Owen created SPARK-30155: Summary: Resolve ambiguous overload from RegexParsers in 2.13 Key: SPARK-30155 URL: https://issues.apache.org/jira/browse/SPARK-30155 Project: Spark Issue

[jira] [Commented] (SPARK-26091) Upgrade to 2.3.4 for Hive Metastore Client 2.3

[ https://issues.apache.org/jira/browse/SPARK-26091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990074#comment-16990074 ] Dongjoon Hyun commented on SPARK-26091: --- It seems that you misunderstand this feature. > but I

[jira] [Comment Edited] (SPARK-26091) Upgrade to 2.3.4 for Hive Metastore Client 2.3

[ https://issues.apache.org/jira/browse/SPARK-26091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990074#comment-16990074 ] Dongjoon Hyun edited comment on SPARK-26091 at 12/6/19 6:41 PM: It seems

[jira] [Resolved] (SPARK-30098) Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-30098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30098. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26736

[jira] [Assigned] (SPARK-30098) Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-30098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-30098: --- Assignee: wuyi > Use default datasource as provider for CREATE TABLE syntax >

[jira] [Commented] (SPARK-27300) Create the new graph projects in Spark and set up build/test

[ https://issues.apache.org/jira/browse/SPARK-27300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990048#comment-16990048 ] Sean R. Owen commented on SPARK-27300: -- So, I have the impression that the follow-up work in the

[jira] [Updated] (SPARK-30154) PySpark UDF to convert MLlib vectors to dense arrays

[ https://issues.apache.org/jira/browse/SPARK-30154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-30154: -- Description: If a PySpark user wants to convert MLlib sparse/dense vectors in a DataFrame

[jira] [Updated] (SPARK-30154) PySpark UDF to convert MLlib vectors to dense arrays

[ https://issues.apache.org/jira/browse/SPARK-30154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-30154: -- Summary: PySpark UDF to convert MLlib vectors to dense arrays (was: Allow PySpark code

[jira] [Created] (SPARK-30154) Allow PySpark code efficiently convert MLlib vectors to dense arrays

Xiangrui Meng created SPARK-30154: - Summary: Allow PySpark code efficiently convert MLlib vectors to dense arrays Key: SPARK-30154 URL: https://issues.apache.org/jira/browse/SPARK-30154 Project:

[jira] [Resolved] (SPARK-30112) Insert overwrite should be able to overwrite to same table under dynamic partition overwrite

[ https://issues.apache.org/jira/browse/SPARK-30112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30112. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26752

[jira] [Updated] (SPARK-30153) Extend data exchange options for vectorized UDF functions with vanilla Arrow serialization

[ https://issues.apache.org/jira/browse/SPARK-30153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luca Canali updated SPARK-30153: Attachment: Flamegraph_test_pandas_udf_SCALAR.png > Extend data exchange options for vectorized

[jira] [Updated] (SPARK-30153) Extend data exchange options for vectorized UDF functions with vanilla Arrow serialization

[ https://issues.apache.org/jira/browse/SPARK-30153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luca Canali updated SPARK-30153: Attachment: Flamegraph_test_pandas_udf_SCALAR_ARROW.png > Extend data exchange options for

[jira] [Resolved] (SPARK-30011) Use DoubleAsIfIntegral / FloatAsIfIntegral workalike in 2.13

[ https://issues.apache.org/jira/browse/SPARK-30011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30011. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26769

[jira] [Assigned] (SPARK-30011) Use DoubleAsIfIntegral / FloatAsIfIntegral workalike in 2.13

[ https://issues.apache.org/jira/browse/SPARK-30011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-30011: - Assignee: Sean R. Owen > Use DoubleAsIfIntegral / FloatAsIfIntegral workalike in 2.13

[jira] [Created] (SPARK-30153) Extend data exchange options for vectorized UDF functions with vanilla Arrow serialization

Luca Canali created SPARK-30153: --- Summary: Extend data exchange options for vectorized UDF functions with vanilla Arrow serialization Key: SPARK-30153 URL: https://issues.apache.org/jira/browse/SPARK-30153

[jira] [Created] (SPARK-30152) Enable Hadoop-2.7/JDK11 build at GitHub Action

Dongjoon Hyun created SPARK-30152: - Summary: Enable Hadoop-2.7/JDK11 build at GitHub Action Key: SPARK-30152 URL: https://issues.apache.org/jira/browse/SPARK-30152 Project: Spark Issue Type:

[jira] [Updated] (SPARK-30144) MLP param map missing

[ https://issues.apache.org/jira/browse/SPARK-30144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen-Erik Cortes updated SPARK-30144: - Description: Param maps for fitted classifiers are available with all classifiers

[jira] [Updated] (SPARK-30144) MLP param map missing

[ https://issues.apache.org/jira/browse/SPARK-30144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen-Erik Cortes updated SPARK-30144: - Attachment: data_banknote_authentication.csv MLP_params_missing.ipynb >

[jira] [Created] (SPARK-30151) Issue better error message when user-specified schema not match relation schema

wuyi created SPARK-30151: Summary: Issue better error message when user-specified schema not match relation schema Key: SPARK-30151 URL: https://issues.apache.org/jira/browse/SPARK-30151 Project: Spark

[jira] [Assigned] (SPARK-30142) Upgrade Maven to 3.6.3

[ https://issues.apache.org/jira/browse/SPARK-30142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-30142: Assignee: Dongjoon Hyun > Upgrade Maven to 3.6.3 > -- > >

[jira] [Resolved] (SPARK-30142) Upgrade Maven to 3.6.3

[ https://issues.apache.org/jira/browse/SPARK-30142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30142. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26770

[jira] [Created] (SPARK-30150) Manage resources (ADD/LIST) does not support quoted path

jobit mathew created SPARK-30150: Summary: Manage resources (ADD/LIST) does not support quoted path Key: SPARK-30150 URL: https://issues.apache.org/jira/browse/SPARK-30150 Project: Spark

[jira] [Updated] (SPARK-30149) Schema Definition Spark Read

[ https://issues.apache.org/jira/browse/SPARK-30149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joby Joje updated SPARK-30149: -- Attachment: Schema.py Output.txt Test1.csv Test.csv >

[jira] [Updated] (SPARK-30149) Schema Definition Spark Read

[ https://issues.apache.org/jira/browse/SPARK-30149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joby Joje updated SPARK-30149: -- Attachment: (was: Test1.SDL) > Schema Definition Spark Read > > >

[jira] [Updated] (SPARK-30149) Schema Definition Spark Read

[ https://issues.apache.org/jira/browse/SPARK-30149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joby Joje updated SPARK-30149: -- Attachment: (was: Test.SDL) > Schema Definition Spark Read > > >

[jira] [Updated] (SPARK-30149) Schema Definition Spark Read

[ https://issues.apache.org/jira/browse/SPARK-30149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joby Joje updated SPARK-30149: -- Attachment: (was: Schema.py) > Schema Definition Spark Read > > >

[jira] [Updated] (SPARK-30149) Schema Definition Spark Read

[ https://issues.apache.org/jira/browse/SPARK-30149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joby Joje updated SPARK-30149: -- Attachment: (was: Output.txt) > Schema Definition Spark Read > > >

[jira] [Updated] (SPARK-30149) Schema Definition Spark Read

[ https://issues.apache.org/jira/browse/SPARK-30149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joby Joje updated SPARK-30149: -- Description: Reading a CSV file with defined schema I am able to load the files and do the

[jira] [Updated] (SPARK-30149) Schema Definition Spark Read

[ https://issues.apache.org/jira/browse/SPARK-30149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joby Joje updated SPARK-30149: -- Attachment: Schema.py Output.txt > Schema Definition Spark Read >

[jira] [Updated] (SPARK-30149) Schema Definition Spark Read

[ https://issues.apache.org/jira/browse/SPARK-30149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joby Joje updated SPARK-30149: -- Attachment: Test1.SDL Test.SDL > Schema Definition Spark Read >

[jira] [Created] (SPARK-30149) Schema Definition Spark Read

Joby Joje created SPARK-30149: - Summary: Schema Definition Spark Read Key: SPARK-30149 URL: https://issues.apache.org/jira/browse/SPARK-30149 Project: Spark Issue Type: Bug Components:

[jira] [Created] (SPARK-30148) Optimize writing plans if there is an analysis exception

jobit mathew created SPARK-30148: Summary: Optimize writing plans if there is an analysis exception Key: SPARK-30148 URL: https://issues.apache.org/jira/browse/SPARK-30148 Project: Spark

[jira] [Commented] (SPARK-26091) Upgrade to 2.3.4 for Hive Metastore Client 2.3

[ https://issues.apache.org/jira/browse/SPARK-26091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989616#comment-16989616 ] t oo commented on SPARK-26091: -- but i dont use 2.3.3 > Upgrade to 2.3.4 for Hive Metastore Client 2.3 >

[jira] [Reopened] (SPARK-29152) Spark Executor Plugin API shutdown is not proper when dynamic allocation enabled

[ https://issues.apache.org/jira/browse/SPARK-29152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jobit mathew reopened SPARK-29152: -- Reopening the Jira, as the issue exists in the master branch also. > Spark Executor Plugin API

[jira] [Resolved] (SPARK-28083) Support ANSI SQL: LIKE ... ESCAPE syntax

[ https://issues.apache.org/jira/browse/SPARK-28083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-28083. Resolution: Fixed This issue is resolved in https://github.com/apache/spark/pull/25001 >

[jira] [Assigned] (SPARK-28083) Support ANSI SQL: LIKE ... ESCAPE syntax

[ https://issues.apache.org/jira/browse/SPARK-28083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-28083: -- Assignee: jiaan.geng > Support ANSI SQL: LIKE ... ESCAPE syntax >

[jira] [Created] (SPARK-30147) Trim the string when cast string type to booleans

Kent Yao created SPARK-30147: Summary: Trim the string when cast string type to booleans Key: SPARK-30147 URL: https://issues.apache.org/jira/browse/SPARK-30147 Project: Spark Issue Type: