[jira] [Commented] (SPARK-32166) Metastore problem on Spark3.0 with Hive3.0

2020-10-12 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212172#comment-17212172 ] angerszhu commented on SPARK-32166: --- Spark Hive client version too low? > Metastore

[jira] [Comment Edited] (SPARK-24930) Exception information is not accurate when using `LOAD DATA LOCAL INPATH`

2020-10-12 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212188#comment-17212188 ] angerszhu edited comment on SPARK-24930 at 10/12/20, 7:25 AM:

[jira] [Commented] (SPARK-24930) Exception information is not accurate when using `LOAD DATA LOCAL INPATH`

2020-10-12 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212188#comment-17212188 ] angerszhu commented on SPARK-24930: --- In current version, error message  {code:java} sp

[jira] [Commented] (SPARK-24930) Exception information is not accurate when using `LOAD DATA LOCAL INPATH`

2020-10-12 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212189#comment-17212189 ] angerszhu commented on SPARK-24930: --- cc [~maropu] Hey, for this jira, I check again a

[jira] [Resolved] (SPARK-33117) Update zstd-jni to 1.4.5-6

2020-10-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33117. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30010 [https://

[jira] [Assigned] (SPARK-33117) Update zstd-jni to 1.4.5-6

2020-10-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33117: - Assignee: Dongjoon Hyun > Update zstd-jni to 1.4.5-6 > -- > >

[jira] [Commented] (SPARK-24930) Exception information is not accurate when using `LOAD DATA LOCAL INPATH`

2020-10-12 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212196#comment-17212196 ] Takeshi Yamamuro commented on SPARK-24930: -- Do you know which pr resolves this?

[jira] [Commented] (SPARK-31294) Benchmark the performance regression

2020-10-12 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212199#comment-17212199 ] Maxim Gekk commented on SPARK-31294: [~cloud_fan] Could you close this ticket since

[jira] [Commented] (SPARK-24930) Exception information is not accurate when using `LOAD DATA LOCAL INPATH`

2020-10-12 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212201#comment-17212201 ] angerszhu commented on SPARK-24930: --- No, form message I put in comment, seems Hadoop m

[jira] [Commented] (SPARK-24930) Exception information is not accurate when using `LOAD DATA LOCAL INPATH`

2020-10-12 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212205#comment-17212205 ] Takeshi Yamamuro commented on SPARK-24930: -- hm, I see. Could you check the reso

[jira] [Resolved] (SPARK-33092) Support subexpression elimination in ProjectExec

2020-10-12 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-33092. -- Fix Version/s: 3.1.0 Resolution: Fixed Resolved by https://github.com/apache/sp

[jira] [Commented] (SPARK-32455) LogisticRegressionModel prediction optimization

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212226#comment-17212226 ] Apache Spark commented on SPARK-32455: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-32455) LogisticRegressionModel prediction optimization

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212227#comment-17212227 ] Apache Spark commented on SPARK-32455: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-33090) Upgrade Google Guava

2020-10-12 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212240#comment-17212240 ] angerszhu commented on SPARK-33090: --- Meet this problem these days too and spark build

[jira] [Created] (SPARK-33118) CREATE TEMPORARY TABLE fails with location

2020-10-12 Thread Pablo Langa Blanco (Jira)
Pablo Langa Blanco created SPARK-33118: -- Summary: CREATE TEMPORARY TABLE fails with location Key: SPARK-33118 URL: https://issues.apache.org/jira/browse/SPARK-33118 Project: Spark Issue

[jira] [Updated] (SPARK-33118) CREATE TEMPORARY TABLE fails with location

2020-10-12 Thread Pablo Langa Blanco (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Langa Blanco updated SPARK-33118: --- Description: The problem is produced when you use CREATE TEMPORARY TABLE with LOCATI

[jira] [Commented] (SPARK-33118) CREATE TEMPORARY TABLE fails with location

2020-10-12 Thread Pablo Langa Blanco (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212242#comment-17212242 ] Pablo Langa Blanco commented on SPARK-33118: I'm working on it > CREATE TEM

[jira] [Assigned] (SPARK-33118) CREATE TEMPORARY TABLE fails with location

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33118: Assignee: Apache Spark > CREATE TEMPORARY TABLE fails with location > ---

[jira] [Commented] (SPARK-33118) CREATE TEMPORARY TABLE fails with location

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212253#comment-17212253 ] Apache Spark commented on SPARK-33118: -- User 'planga82' has created a pull request

[jira] [Assigned] (SPARK-33118) CREATE TEMPORARY TABLE fails with location

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33118: Assignee: Apache Spark > CREATE TEMPORARY TABLE fails with location > ---

[jira] [Assigned] (SPARK-33118) CREATE TEMPORARY TABLE fails with location

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33118: Assignee: (was: Apache Spark) > CREATE TEMPORARY TABLE fails with location >

[jira] [Resolved] (SPARK-32989) Performance regression when selecting from str_to_map

2020-10-12 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-32989. -- Fix Version/s: 3.1.0 Assignee: L. C. Hsieh Resolution: Fixed > Perform

[jira] [Commented] (SPARK-32704) Logging plan changes for execution

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212273#comment-17212273 ] Apache Spark commented on SPARK-32704: -- User 'maropu' has created a pull request fo

[jira] [Commented] (SPARK-28099) Assertion when querying unpartitioned Hive table with partition-like naming

2020-10-12 Thread Zsombor Fedor (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212281#comment-17212281 ] Zsombor Fedor commented on SPARK-28099: --- It is not ORC specific: val testData =

[jira] [Created] (SPARK-33119) Only return the first two rows to avoid Driver OOM

2020-10-12 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33119: --- Summary: Only return the first two rows to avoid Driver OOM Key: SPARK-33119 URL: https://issues.apache.org/jira/browse/SPARK-33119 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-33119) Only return the first two rows to avoid Driver OOM

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33119: Assignee: (was: Apache Spark) > Only return the first two rows to avoid Driver OOM >

[jira] [Commented] (SPARK-33119) Only return the first two rows to avoid Driver OOM

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212299#comment-17212299 ] Apache Spark commented on SPARK-33119: -- User 'wangyum' has created a pull request f

[jira] [Assigned] (SPARK-33119) Only return the first two rows to avoid Driver OOM

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33119: Assignee: Apache Spark > Only return the first two rows to avoid Driver OOM > ---

[jira] [Commented] (SPARK-33119) Only return the first two rows to avoid Driver OOM

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212300#comment-17212300 ] Apache Spark commented on SPARK-33119: -- User 'wangyum' has created a pull request f

[jira] [Updated] (SPARK-33119) ScalarSubquery should returns the first two rows to avoid Driver OOM

2020-10-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33119: Summary: ScalarSubquery should returns the first two rows to avoid Driver OOM (was: Only return

[jira] [Commented] (SPARK-32229) Application entry parsing fails because DriverWrapper registered instead of the normal driver

2020-10-12 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212327#comment-17212327 ] Gabor Somogyi commented on SPARK-32229: --- Started to work on this. > Application e

[jira] [Commented] (SPARK-33103) Custom Schema with Custom RDD reorders columns when more than 4 added

2020-10-12 Thread Justin Mays (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212356#comment-17212356 ] Justin Mays commented on SPARK-33103: - I tried running against Spark 2.4.7 and got t

[jira] [Commented] (SPARK-33116) Spark SQL window function with order by cause result incorrect

2020-10-12 Thread Will Du (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212359#comment-17212359 ] Will Du commented on SPARK-33116: - [~maropu], the statement is comparing query with PART

[jira] [Comment Edited] (SPARK-33116) Spark SQL window function with order by cause result incorrect

2020-10-12 Thread Will Du (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212359#comment-17212359 ] Will Du edited comment on SPARK-33116 at 10/12/20, 1:18 PM:

[jira] [Comment Edited] (SPARK-33116) Spark SQL window function with order by cause result incorrect

2020-10-12 Thread Will Du (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212359#comment-17212359 ] Will Du edited comment on SPARK-33116 at 10/12/20, 1:19 PM:

[jira] [Commented] (SPARK-31338) Spark SQL JDBC Data Source partitioned read : Spark SQL does not honor for NOT NULL table definition of partition key.

2020-10-12 Thread Ankush Chatterjee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212373#comment-17212373 ] Ankush Chatterjee commented on SPARK-31338: --- When read like this is used : - {

[jira] [Comment Edited] (SPARK-33116) Spark SQL window function with order by cause result incorrect

2020-10-12 Thread Will Du (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212359#comment-17212359 ] Will Du edited comment on SPARK-33116 at 10/12/20, 1:41 PM:

[jira] [Closed] (SPARK-33116) Spark SQL window function with order by cause result incorrect

2020-10-12 Thread Will Du (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Will Du closed SPARK-33116. --- > Spark SQL window function with order by cause result incorrect > -

[jira] [Assigned] (SPARK-33111) aft transform optimization

2020-10-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-33111: Assignee: zhengruifeng > aft transform optimization > -- > >

[jira] [Resolved] (SPARK-33111) aft transform optimization

2020-10-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-33111. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 3 [https://gi

[jira] [Created] (SPARK-33120) Lazy Load of SparkContext.addFiles

2020-10-12 Thread Taylor Smock (Jira)
Taylor Smock created SPARK-33120: Summary: Lazy Load of SparkContext.addFiles Key: SPARK-33120 URL: https://issues.apache.org/jira/browse/SPARK-33120 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-33121) Spark does not stop gracefully

2020-10-12 Thread Dmitry Tverdokhleb (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Tverdokhleb updated SPARK-33121: --- Description: Hi. I have a spark streaming code, like: {code:java} inputStream.foreac

[jira] [Created] (SPARK-33121) Spark does not stop gracefully

2020-10-12 Thread Dmitry Tverdokhleb (Jira)
Dmitry Tverdokhleb created SPARK-33121: -- Summary: Spark does not stop gracefully Key: SPARK-33121 URL: https://issues.apache.org/jira/browse/SPARK-33121 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-33121) Spark does not stop gracefully

2020-10-12 Thread Dmitry Tverdokhleb (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Tverdokhleb updated SPARK-33121: --- Description: Hi. I have a spark streaming code, like: {code:java} inputStream.foreac

[jira] [Commented] (SPARK-33103) Custom Schema with Custom RDD reorders columns when more than 4 added

2020-10-12 Thread Justin Mays (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212428#comment-17212428 ] Justin Mays commented on SPARK-33103: - Ok follow up... I think i figured this out bu

[jira] [Resolved] (SPARK-33103) Custom Schema with Custom RDD reorders columns when more than 4 added

2020-10-12 Thread Justin Mays (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justin Mays resolved SPARK-33103. - Resolution: Not A Problem > Custom Schema with Custom RDD reorders columns when more than 4 adde

[jira] [Updated] (SPARK-33121) Spark does not stop gracefully

2020-10-12 Thread Dmitry Tverdokhleb (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Tverdokhleb updated SPARK-33121: --- Description: Hi. I have a spark streaming code, like: {code:java} inputStream.foreac

[jira] [Assigned] (SPARK-33016) Potential SQLMetrics missed which might cause WEB UI display issue while AQE is on.

2020-10-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33016: --- Assignee: Leanken.Lin > Potential SQLMetrics missed which might cause WEB UI display issue

[jira] [Resolved] (SPARK-33016) Potential SQLMetrics missed which might cause WEB UI display issue while AQE is on.

2020-10-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33016. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29965 [https://gith

[jira] [Resolved] (SPARK-31294) Benchmark the performance regression

2020-10-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31294. - Fix Version/s: 3.0.0 Resolution: Fixed > Benchmark the performance regression > -

[jira] [Commented] (SPARK-27733) Upgrade to Avro 1.10.0

2020-10-12 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-27733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212451#comment-17212451 ] Ismaël Mejía commented on SPARK-27733: -- [~sha...@uber.com] sorry I missed somehow t

[jira] [Commented] (SPARK-25271) Creating parquet table with all the column null throws exception

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212487#comment-17212487 ] Apache Spark commented on SPARK-25271: -- User 'viirya' has created a pull request fo

[jira] [Closed] (SPARK-33089) avro format does not propagate Hadoop config from DS options to underlying HDFS file system

2020-10-12 Thread Yuning Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuning Zhang closed SPARK-33089. > avro format does not propagate Hadoop config from DS options to underlying > HDFS file system > ---

[jira] [Commented] (SPARK-30893) Expressions should not change its data type/nullability after it's created

2020-10-12 Thread Anurag Mantripragada (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212635#comment-17212635 ] Anurag Mantripragada commented on SPARK-30893: -- As mentioned here [#https:

[jira] [Comment Edited] (SPARK-30893) Expressions should not change its data type/nullability after it's created

2020-10-12 Thread Anurag Mantripragada (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212635#comment-17212635 ] Anurag Mantripragada edited comment on SPARK-30893 at 10/12/20, 7:56 PM: -

[jira] [Created] (SPARK-33122) Remove redundant aggregates in the Optimzier

2020-10-12 Thread Tanel Kiis (Jira)
Tanel Kiis created SPARK-33122: -- Summary: Remove redundant aggregates in the Optimzier Key: SPARK-33122 URL: https://issues.apache.org/jira/browse/SPARK-33122 Project: Spark Issue Type: Improvem

[jira] [Assigned] (SPARK-33122) Remove redundant aggregates in the Optimzier

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33122: Assignee: (was: Apache Spark) > Remove redundant aggregates in the Optimzier > --

[jira] [Assigned] (SPARK-33122) Remove redundant aggregates in the Optimzier

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33122: Assignee: Apache Spark > Remove redundant aggregates in the Optimzier > -

[jira] [Commented] (SPARK-33122) Remove redundant aggregates in the Optimzier

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212661#comment-17212661 ] Apache Spark commented on SPARK-33122: -- User 'tanelk' has created a pull request fo

[jira] [Commented] (SPARK-33122) Remove redundant aggregates in the Optimzier

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212660#comment-17212660 ] Apache Spark commented on SPARK-33122: -- User 'tanelk' has created a pull request fo

[jira] [Updated] (SPARK-25271) Creating parquet table with all the column null throws exception

2020-10-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25271: -- Fix Version/s: 2.4.8 > Creating parquet table with all the column null throws exception >

[jira] [Updated] (SPARK-33122) Remove redundant aggregates in the Optimzier

2020-10-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33122: -- Affects Version/s: (was: 3.0.1) 3.1.0 > Remove redundant aggregates

[jira] [Commented] (SPARK-33122) Remove redundant aggregates in the Optimzier

2020-10-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212669#comment-17212669 ] Dongjoon Hyun commented on SPARK-33122: --- Hi, [~tanelk]. Since this is an `Improvem

[jira] [Commented] (SPARK-33120) Lazy Load of SparkContext.addFiles

2020-10-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212670#comment-17212670 ] Dongjoon Hyun commented on SPARK-33120: --- Hi, [~tsmock]. What is the benefit you ne

[jira] [Comment Edited] (SPARK-33120) Lazy Load of SparkContext.addFiles

2020-10-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212670#comment-17212670 ] Dongjoon Hyun edited comment on SPARK-33120 at 10/12/20, 9:00 PM:

[jira] [Commented] (SPARK-33118) CREATE TEMPORARY TABLE fails with location

2020-10-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212672#comment-17212672 ] Dongjoon Hyun commented on SPARK-33118: --- Thank you for your contribution, [~planga

[jira] [Updated] (SPARK-33118) CREATE TEMPORARY TABLE fails with location

2020-10-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33118: -- Affects Version/s: 3.1.0 > CREATE TEMPORARY TABLE fails with location > --

[jira] [Commented] (SPARK-25390) Data source V2 API refactoring

2020-10-12 Thread Jan Berkel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212675#comment-17212675 ] Jan Berkel commented on SPARK-25390: I'm in a similar situation. [~Kyrdan] asked on

[jira] [Resolved] (SPARK-33118) CREATE TEMPORARY TABLE fails with location

2020-10-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33118. --- Fix Version/s: 3.0.2 3.1.0 Resolution: Fixed Issue resolved by pul

[jira] [Assigned] (SPARK-33118) CREATE TEMPORARY TABLE fails with location

2020-10-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33118: - Assignee: Pablo Langa Blanco (was: Apache Spark) > CREATE TEMPORARY TABLE fails with l

[jira] [Commented] (SPARK-33120) Lazy Load of SparkContext.addFiles

2020-10-12 Thread Taylor Smock (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212679#comment-17212679 ] Taylor Smock commented on SPARK-33120: -- I'd like to avoid using excess network reso

[jira] [Commented] (SPARK-32381) Expose the ability for users to use parallel file & avoid location information discovery in RDDs

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212684#comment-17212684 ] Apache Spark commented on SPARK-32381: -- User 'sunchao' has created a pull request f

[jira] [Created] (SPARK-33123) Ignore `GitHub Action file` change in Amplab Jenkins

2020-10-12 Thread William Hyun (Jira)
William Hyun created SPARK-33123: Summary: Ignore `GitHub Action file` change in Amplab Jenkins Key: SPARK-33123 URL: https://issues.apache.org/jira/browse/SPARK-33123 Project: Spark Issue Ty

[jira] [Assigned] (SPARK-33123) Ignore `GitHub Action file` change in Amplab Jenkins

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33123: Assignee: Apache Spark > Ignore `GitHub Action file` change in Amplab Jenkins > -

[jira] [Assigned] (SPARK-33123) Ignore `GitHub Action file` change in Amplab Jenkins

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33123: Assignee: (was: Apache Spark) > Ignore `GitHub Action file` change in Amplab Jenkins

[jira] [Commented] (SPARK-33123) Ignore `GitHub Action file` change in Amplab Jenkins

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212707#comment-17212707 ] Apache Spark commented on SPARK-33123: -- User 'williamhyun' has created a pull reque

[jira] [Commented] (SPARK-33123) Ignore `GitHub Action file` change in Amplab Jenkins

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212708#comment-17212708 ] Apache Spark commented on SPARK-33123: -- User 'williamhyun' has created a pull reque

[jira] [Commented] (SPARK-24434) Support user-specified driver and executor pod templates

2020-10-12 Thread Prakash Rajendran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212711#comment-17212711 ] Prakash Rajendran commented on SPARK-24434: ---   Hello experts, I have requirem

[jira] [Commented] (SPARK-30893) Expressions should not change its data type/nullability after it's created

2020-10-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212715#comment-17212715 ] Dongjoon Hyun commented on SPARK-30893: --- [~anuragmantri]. Which comment are you re

[jira] [Commented] (SPARK-30893) Expressions should not change its data type/nullability after it's created

2020-10-12 Thread Anurag Mantripragada (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212717#comment-17212717 ] Anurag Mantripragada commented on SPARK-30893: -- Sorry for not being clear.

[jira] [Created] (SPARK-33124) Adds a group tag in all the expressions for built-in functions

2020-10-12 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-33124: Summary: Adds a group tag in all the expressions for built-in functions Key: SPARK-33124 URL: https://issues.apache.org/jira/browse/SPARK-33124 Project: Spark

[jira] [Commented] (SPARK-30893) Expressions should not change its data type/nullability after it's created

2020-10-12 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212731#comment-17212731 ] Takeshi Yamamuro commented on SPARK-30893: -- You find any data corruption in bra

[jira] [Commented] (SPARK-27292) Spark Job Fails with Unknown Error writing to S3 from AWS EMR

2020-10-12 Thread Venkata (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212746#comment-17212746 ] Venkata commented on SPARK-27292: - [~hyukjin.kwon] Could you let me know how you resolve

[jira] [Commented] (SPARK-30894) The nullability of Size function should not depend on SQLConf.get

2020-10-12 Thread Anurag Mantripragada (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212748#comment-17212748 ] Anurag Mantripragada commented on SPARK-30894: -- [~maxgekk] - Looking at the

[jira] [Commented] (SPARK-30893) Expressions should not change its data type/nullability after it's created

2020-10-12 Thread Anurag Mantripragada (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212749#comment-17212749 ] Anurag Mantripragada commented on SPARK-30893: -- [~maropu], [~dongjoon] - I

[jira] [Created] (SPARK-33125) Improve the error when Lead and Lag are not allowed to specify window frame

2020-10-12 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-33125: -- Summary: Improve the error when Lead and Lag are not allowed to specify window frame Key: SPARK-33125 URL: https://issues.apache.org/jira/browse/SPARK-33125 Project: Spar

[jira] [Commented] (SPARK-33125) Improve the error when Lead and Lag are not allowed to specify window frame

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212803#comment-17212803 ] Apache Spark commented on SPARK-33125: -- User 'beliefer' has created a pull request

[jira] [Assigned] (SPARK-33125) Improve the error when Lead and Lag are not allowed to specify window frame

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33125: Assignee: (was: Apache Spark) > Improve the error when Lead and Lag are not allowed t

[jira] [Assigned] (SPARK-33125) Improve the error when Lead and Lag are not allowed to specify window frame

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33125: Assignee: Apache Spark > Improve the error when Lead and Lag are not allowed to specify w

[jira] [Commented] (SPARK-27733) Upgrade to Avro 1.10.0

2020-10-12 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212804#comment-17212804 ] Chao Sun commented on SPARK-27733: -- [~iemejia] I can help with Hive releases if you can

[jira] [Commented] (SPARK-33125) Improve the error when Lead and Lag are not allowed to specify window frame

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212805#comment-17212805 ] Apache Spark commented on SPARK-33125: -- User 'beliefer' has created a pull request

[jira] [Commented] (SPARK-33090) Upgrade Google Guava

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212825#comment-17212825 ] Apache Spark commented on SPARK-33090: -- User 'sfcoy' has created a pull request for

[jira] [Assigned] (SPARK-33090) Upgrade Google Guava

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33090: Assignee: (was: Apache Spark) > Upgrade Google Guava > > >

[jira] [Assigned] (SPARK-33090) Upgrade Google Guava

2020-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33090: Assignee: Apache Spark > Upgrade Google Guava > > >

[jira] [Commented] (SPARK-33090) Upgrade Google Guava

2020-10-12 Thread Stephen Coy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212826#comment-17212826 ] Stephen Coy commented on SPARK-33090: - Created PR > Upgrade Google Guava >

[jira] [Updated] (SPARK-33121) Spark does not shutdown gracefully

2020-10-12 Thread Dmitry Tverdokhleb (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Tverdokhleb updated SPARK-33121: --- Summary: Spark does not shutdown gracefully (was: Spark does not stop gracefully)

[jira] [Updated] (SPARK-33121) Spark does not shutdown gracefully

2020-10-12 Thread Dmitry Tverdokhleb (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Tverdokhleb updated SPARK-33121: --- Description: Hi. I am trying to migrate from spark 2.4.5 to 3.0.1 and there is a pro

[jira] [Updated] (SPARK-33121) Spark does not shutdown gracefully

2020-10-12 Thread Dmitry Tverdokhleb (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Tverdokhleb updated SPARK-33121: --- Description: Hi. I am trying to migrate from spark 2.4.5 to 3.0.1 and there is a pro

[jira] [Updated] (SPARK-33121) Spark does not shutdown gracefully

2020-10-12 Thread Dmitry Tverdokhleb (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Tverdokhleb updated SPARK-33121: --- Description: Hi. I am trying to migrate from spark 2.4.5 to 3.0.1 and there is a pro

[jira] [Updated] (SPARK-33121) Spark does not shutdown gracefully

2020-10-12 Thread Dmitry Tverdokhleb (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Tverdokhleb updated SPARK-33121: --- Description: Hi. I am trying to migrate from spark 2.4.5 to 3.0.1 and there is a pro

  1   2   >