[jira] [Updated] (SPARK-27848) AppVeyor change to latest R version (3.6.0)

[ https://issues.apache.org/jira/browse/SPARK-27848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-27848: - Description: R 3.6.0 is released last month. We better test higher versions of R in AppVeyor.

[jira] [Updated] (SPARK-25944) AppVeyor change to latest R version

[ https://issues.apache.org/jira/browse/SPARK-25944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25944: - Description: R 3.5.1 is released few months ago. We better test higher versions of R in

[jira] [Updated] (SPARK-25944) AppVeyor change to latest R version

[ https://issues.apache.org/jira/browse/SPARK-25944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25944: - Description: R 3.5.1 is released few months ago. We better test higher versions of R in

[jira] [Updated] (SPARK-25944) AppVeyor change to latest R version

[ https://issues.apache.org/jira/browse/SPARK-25944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25944: - Description: R 3.5.1 is released few months ago. We better test higher versions of R in

[jira] [Commented] (SPARK-25944) AppVeyor change to latest R version

[ https://issues.apache.org/jira/browse/SPARK-25944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16849354#comment-16849354 ] Hyukjin Kwon commented on SPARK-25944: -- Ah .. I realised that I messed up JIRA and PR links. I

[jira] [Issue Comment Deleted] (SPARK-27848) AppVeyor change to latest R version (3.6.0)

[ https://issues.apache.org/jira/browse/SPARK-27848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-27848: - Comment: was deleted (was: User 'HyukjinKwon' has created a pull request for this issue:

[jira] [Issue Comment Deleted] (SPARK-27848) AppVeyor change to latest R version (3.6.0)

[ https://issues.apache.org/jira/browse/SPARK-27848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-27848: - Comment: was deleted (was: Fixed in https://github.com/apache/spark/pull/24716) > AppVeyor

[jira] [Resolved] (SPARK-27848) AppVeyor change to latest R version (3.6.0)

[ https://issues.apache.org/jira/browse/SPARK-27848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27848. -- Resolution: Duplicate > AppVeyor change to latest R version (3.6.0) >

[jira] [Updated] (SPARK-25944) AppVeyor change to latest R version

[ https://issues.apache.org/jira/browse/SPARK-25944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25944: - Summary: AppVeyor change to latest R version (was: AppVeyor change to latest R version

[jira] [Reopened] (SPARK-27848) AppVeyor change to latest R version (3.6.0)

[ https://issues.apache.org/jira/browse/SPARK-27848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-27848: -- > AppVeyor change to latest R version (3.6.0) > --- > >

[jira] [Commented] (SPARK-27848) AppVeyor change to latest R version (3.6.0)

[ https://issues.apache.org/jira/browse/SPARK-27848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16849353#comment-16849353 ] Apache Spark commented on SPARK-27848: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Resolved] (SPARK-27848) AppVeyor change to latest R version (3.6.0)

[ https://issues.apache.org/jira/browse/SPARK-27848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27848. -- Resolution: Fixed Fixed in https://github.com/apache/spark/pull/24716 > AppVeyor change to

[jira] [Updated] (SPARK-27848) AppVeyor change to latest R version (3.6.0)

[ https://issues.apache.org/jira/browse/SPARK-27848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-27848: - Fix Version/s: 3.0.0 > AppVeyor change to latest R version (3.6.0) >

[jira] [Closed] (SPARK-27860) Use efficient sorting instead of `.sorted.reverse` sequence

[ https://issues.apache.org/jira/browse/SPARK-27860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wenxuanguan closed SPARK-27860. --- > Use efficient sorting instead of `.sorted.reverse` sequence >

[jira] [Resolved] (SPARK-27860) Use efficient sorting instead of `.sorted.reverse` sequence

[ https://issues.apache.org/jira/browse/SPARK-27860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wenxuanguan resolved SPARK-27860. - Resolution: Duplicate > Use efficient sorting instead of `.sorted.reverse` sequence >

[jira] [Closed] (SPARK-341) Added MapPartitionsWithSplitRDD.

[ https://issues.apache.org/jira/browse/SPARK-341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wenxuanguan closed SPARK-341. - > Added MapPartitionsWithSplitRDD. > > > Key: SPARK-341 >

[jira] [Created] (SPARK-27860) Use efficient sorting instead of `.sorted.reverse` sequence

wenxuanguan created SPARK-27860: --- Summary: Use efficient sorting instead of `.sorted.reverse` sequence Key: SPARK-27860 URL: https://issues.apache.org/jira/browse/SPARK-27860 Project: Spark

[jira] [Resolved] (SPARK-27859) Use efficient sorting instead of `.sorted.reverse` sequence

[ https://issues.apache.org/jira/browse/SPARK-27859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27859. --- Resolution: Fixed Assignee: wenxuanguan Fix Version/s: 3.0.0 This is

[jira] [Assigned] (SPARK-27859) Use efficient sorting instead of `.sorted.reverse` sequence

[ https://issues.apache.org/jira/browse/SPARK-27859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27859: Assignee: Apache Spark > Use efficient sorting instead of `.sorted.reverse` sequence >

[jira] [Assigned] (SPARK-27859) Use efficient sorting instead of `.sorted.reverse` sequence

[ https://issues.apache.org/jira/browse/SPARK-27859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27859: Assignee: (was: Apache Spark) > Use efficient sorting instead of `.sorted.reverse`

[jira] [Created] (SPARK-27859) Use efficient sorting instead of `.sorted.reverse` sequence

Dongjoon Hyun created SPARK-27859: - Summary: Use efficient sorting instead of `.sorted.reverse` sequence Key: SPARK-27859 URL: https://issues.apache.org/jira/browse/SPARK-27859 Project: Spark

[jira] [Comment Edited] (SPARK-27833) java.lang.AssertionError: assertion failed: No plan for EventTimeWatermark

[ https://issues.apache.org/jira/browse/SPARK-27833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16849320#comment-16849320 ] Raviteja edited comment on SPARK-27833 at 5/28/19 4:25 AM: --- I have tried step

[jira] [Commented] (SPARK-27833) java.lang.AssertionError: assertion failed: No plan for EventTimeWatermark

[ https://issues.apache.org/jira/browse/SPARK-27833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16849320#comment-16849320 ] Raviteja commented on SPARK-27833: -- I have tried step by step, i faced the issue when i add watermark

[jira] [Assigned] (SPARK-23191) Workers registration failes in case of network drop

[ https://issues.apache.org/jira/browse/SPARK-23191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-23191: --- Assignee: wuyi > Workers registration failes in case of network drop >

[jira] [Resolved] (SPARK-23191) Workers registration failes in case of network drop

[ https://issues.apache.org/jira/browse/SPARK-23191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-23191. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24569

[jira] [Updated] (SPARK-27578) Support INTERVAL ... HOUR TO SECOND syntax

[ https://issues.apache.org/jira/browse/SPARK-27578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27578: -- Summary: Support INTERVAL ... HOUR TO SECOND syntax (was: Add support for "interval

[jira] [Updated] (SPARK-27578) Add support for "interval '23:59:59' hour to second"

[ https://issues.apache.org/jira/browse/SPARK-27578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27578: -- Issue Type: Improvement (was: Bug) > Add support for "interval '23:59:59' hour to second" >

[jira] [Updated] (SPARK-27578) Add support for "interval '23:59:59' hour to second"

[ https://issues.apache.org/jira/browse/SPARK-27578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27578: -- Description: Currently, SparkSQL can support interval format like this.    {code:java}

[jira] [Resolved] (SPARK-27858) Fix for avro deserialization on union types with multiple non-null types

[ https://issues.apache.org/jira/browse/SPARK-27858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27858. --- Resolution: Fixed Fix Version/s: 3.0.0 2.4.4 This is resolved via

[jira] [Assigned] (SPARK-27858) Fix for avro deserialization on union types with multiple non-null types

[ https://issues.apache.org/jira/browse/SPARK-27858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27858: Assignee: (was: Apache Spark) > Fix for avro deserialization on union types with

[jira] [Commented] (SPARK-27858) Fix for avro deserialization on union types with multiple non-null types

[ https://issues.apache.org/jira/browse/SPARK-27858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16849257#comment-16849257 ] Apache Spark commented on SPARK-27858: -- User 'gcmerz' has created a pull request for this issue:

[jira] [Assigned] (SPARK-27858) Fix for avro deserialization on union types with multiple non-null types

[ https://issues.apache.org/jira/browse/SPARK-27858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27858: Assignee: Apache Spark > Fix for avro deserialization on union types with multiple

[jira] [Created] (SPARK-27858) Fix for avro deserialization on union types with multiple non-null types

Dongjoon Hyun created SPARK-27858: - Summary: Fix for avro deserialization on union types with multiple non-null types Key: SPARK-27858 URL: https://issues.apache.org/jira/browse/SPARK-27858 Project:

[jira] [Assigned] (SPARK-27857) DataSourceV2: Support ALTER TABLE statements

[ https://issues.apache.org/jira/browse/SPARK-27857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27857: Assignee: Apache Spark > DataSourceV2: Support ALTER TABLE statements >

[jira] [Assigned] (SPARK-27857) DataSourceV2: Support ALTER TABLE statements

[ https://issues.apache.org/jira/browse/SPARK-27857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27857: Assignee: (was: Apache Spark) > DataSourceV2: Support ALTER TABLE statements >

[jira] [Created] (SPARK-27857) DataSourceV2: Support ALTER TABLE statements

Ryan Blue created SPARK-27857: - Summary: DataSourceV2: Support ALTER TABLE statements Key: SPARK-27857 URL: https://issues.apache.org/jira/browse/SPARK-27857 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-27768) Infinity, -Infinity, NaN should be recognized in a case insensitive manner

[ https://issues.apache.org/jira/browse/SPARK-27768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16849078#comment-16849078 ] Dilip Biswal commented on SPARK-27768: -- [~dongjoon] Thanks for trying out Presto. Just want to

[jira] [Resolved] (SPARK-27071) Expose additional metrics in status.api.v1.StageData

[ https://issues.apache.org/jira/browse/SPARK-27071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-27071. --- Resolution: Fixed Fix Version/s: 3.0.0 > Expose additional metrics in

[jira] [Assigned] (SPARK-27071) Expose additional metrics in status.api.v1.StageData

[ https://issues.apache.org/jira/browse/SPARK-27071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell reassigned SPARK-27071: - Assignee: Tom van Bussel > Expose additional metrics in

[jira] [Assigned] (SPARK-27777) Eliminate uncessary sliding job in AreaUnderCurve

[ https://issues.apache.org/jira/browse/SPARK-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-2: - Assignee: zhengruifeng > Eliminate uncessary sliding job in AreaUnderCurve >

[jira] [Resolved] (SPARK-27777) Eliminate uncessary sliding job in AreaUnderCurve

[ https://issues.apache.org/jira/browse/SPARK-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24648

[jira] [Commented] (SPARK-27855) Union failed between 2 datasets of the same type converted from different dataframes

[ https://issues.apache.org/jira/browse/SPARK-27855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16849016#comment-16849016 ] Liang-Chi Hsieh commented on SPARK-27855: - If you notice, the printed schema of two Datasets is

[jira] [Assigned] (SPARK-27665) Split fetch shuffle blocks protocol from OpenBlocks

[ https://issues.apache.org/jira/browse/SPARK-27665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-27665: --- Assignee: Yuanjian Li > Split fetch shuffle blocks protocol from OpenBlocks >

[jira] [Resolved] (SPARK-27665) Split fetch shuffle blocks protocol from OpenBlocks

[ https://issues.apache.org/jira/browse/SPARK-27665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-27665. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24565

[jira] [Updated] (SPARK-27776) Avoid duplicate Java reflection in DataSource

[ https://issues.apache.org/jira/browse/SPARK-27776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-27776: -- Priority: Trivial (was: Minor) > Avoid duplicate Java reflection in DataSource >

[jira] [Assigned] (SPARK-27856) do not forcibly add cast when inserting table

[ https://issues.apache.org/jira/browse/SPARK-27856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27856: Assignee: Apache Spark (was: Wenchen Fan) > do not forcibly add cast when inserting

[jira] [Assigned] (SPARK-27856) do not forcibly add cast when inserting table

[ https://issues.apache.org/jira/browse/SPARK-27856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27856: Assignee: Wenchen Fan (was: Apache Spark) > do not forcibly add cast when inserting

[jira] [Created] (SPARK-27856) do not forcibly add cast when inserting table

Wenchen Fan created SPARK-27856: --- Summary: do not forcibly add cast when inserting table Key: SPARK-27856 URL: https://issues.apache.org/jira/browse/SPARK-27856 Project: Spark Issue Type:

[jira] [Updated] (SPARK-27855) Union failed between 2 datasets of the same type converted from different dataframes

[ https://issues.apache.org/jira/browse/SPARK-27855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hao Ren updated SPARK-27855: Description: 2 Datasets of the same type converted from different dataframes can not union. Here is the

[jira] [Updated] (SPARK-27855) Union failed between 2 datasets of the same type converted from different dataframes

[ https://issues.apache.org/jira/browse/SPARK-27855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hao Ren updated SPARK-27855: Description: 2 Datasets of the same type converted from different dataframes can not union. Here is the

[jira] [Created] (SPARK-27855) Union failed between 2 datasets of the same type converted from different dataframes

Hao Ren created SPARK-27855: --- Summary: Union failed between 2 datasets of the same type converted from different dataframes Key: SPARK-27855 URL: https://issues.apache.org/jira/browse/SPARK-27855 Project:

[jira] [Commented] (SPARK-13182) Spark Executor retries infinitely

[ https://issues.apache.org/jira/browse/SPARK-13182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16848904#comment-16848904 ] Sean Owen commented on SPARK-13182: --- On its face that sounds like a YARN-related issue. Rescheduling a

[jira] [Resolved] (SPARK-27803) fix column pruning for python UDF

[ https://issues.apache.org/jira/browse/SPARK-27803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27803. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24675

[jira] [Commented] (SPARK-15348) Hive ACID

[ https://issues.apache.org/jira/browse/SPARK-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16848881#comment-16848881 ] Xianyin Xin commented on SPARK-15348: - The starting point (or goal) of Delta Lake is not ACID, but

[jira] [Commented] (SPARK-27833) java.lang.AssertionError: assertion failed: No plan for EventTimeWatermark

[ https://issues.apache.org/jira/browse/SPARK-27833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16848840#comment-16848840 ] Gabor Somogyi commented on SPARK-27833: --- Maybe you could just copy the working sink with a

[jira] [Commented] (SPARK-27553) Operation log is not closed when close session

[ https://issues.apache.org/jira/browse/SPARK-27553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16848823#comment-16848823 ] jinwensc commented on SPARK-27553: -- it was closed in org.apche.hive.service.cli.operation.Operation

[jira] [Commented] (SPARK-27826) saveAsTable() function case table have "HiveFileFormat" "ParquetFileFormat" format issue

[ https://issues.apache.org/jira/browse/SPARK-27826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16848811#comment-16848811 ] fengtlyer commented on SPARK-27826: --- Hi Hyukjin, Our team think this is a compatibility issue. We

[jira] [Commented] (SPARK-27808) Ability to ignore existing files for structured streaming

[ https://issues.apache.org/jira/browse/SPARK-27808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16848769#comment-16848769 ] Gabor Somogyi commented on SPARK-27808: --- What I've suggested is a workaround of course and not

[jira] [Updated] (SPARK-27854) [Spark-SQL] OOM when using unequal join sql

[ https://issues.apache.org/jira/browse/SPARK-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kai zhao updated SPARK-27854: - Environment: Spark Version:1.6.2 HDP Version:2.5 JDK Version:1.8 OS Version:Redhat 7.3   Cluster

[jira] [Updated] (SPARK-27854) [Spark-SQL] OOM when using unequal join sql

[ https://issues.apache.org/jira/browse/SPARK-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kai zhao updated SPARK-27854: - Environment: Spark Version:1.6.2 HDP Version:2.5 JDK Version:1.8 OS Version:Redhat 7.3   Runtime

[jira] [Commented] (SPARK-27837) Running rand() in SQL with seed of column results in error (rand(col1))

[ https://issues.apache.org/jira/browse/SPARK-27837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16848752#comment-16848752 ] Liang-Chi Hsieh commented on SPARK-27837: - I don't see it makes sense. I checked few DBs, and

[jira] [Comment Edited] (SPARK-13182) Spark Executor retries infinitely

[ https://issues.apache.org/jira/browse/SPARK-13182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16848749#comment-16848749 ] Atul Anand edited comment on SPARK-13182 at 5/27/19 9:16 AM: - [~srowen] The

[jira] [Updated] (SPARK-27854) [Spark-SQL] OOM when using unequal join sql

[ https://issues.apache.org/jira/browse/SPARK-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kai zhao updated SPARK-27854: - Description: I am using Spark 1.6.2 which is from hdp package. Reproduce Steps: 1.Start Spark thrift

[jira] [Comment Edited] (SPARK-13182) Spark Executor retries infinitely

[ https://issues.apache.org/jira/browse/SPARK-13182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16848749#comment-16848749 ] Atul Anand edited comment on SPARK-13182 at 5/27/19 9:16 AM: - [~srowen] The

[jira] [Commented] (SPARK-13182) Spark Executor retries infinitely

[ https://issues.apache.org/jira/browse/SPARK-13182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16848749#comment-16848749 ] Atul Anand commented on SPARK-13182: [~srowen] The issue here is spark does not consider this as

[jira] [Updated] (SPARK-27854) [Spark-SQL] OOM when using unequal join sql

[ https://issues.apache.org/jira/browse/SPARK-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kai zhao updated SPARK-27854: - Description: I am using Spark 1.6.2 which is from hdp package. Reproduce Steps: 1.Start Spark thrift

[jira] [Created] (SPARK-27854) [Spark-SQL] OOM when using unequal join sql

kai zhao created SPARK-27854: Summary: [Spark-SQL] OOM when using unequal join sql Key: SPARK-27854 URL: https://issues.apache.org/jira/browse/SPARK-27854 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-27851) Allow for custom BroadcastMode#transform return values

[ https://issues.apache.org/jira/browse/SPARK-27851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marc Arndt updated SPARK-27851: --- Summary: Allow for custom BroadcastMode#transform return values (was: Allow for custom

[jira] [Created] (SPARK-27853) Allow for custom Partitioning implementations

Marc Arndt created SPARK-27853: -- Summary: Allow for custom Partitioning implementations Key: SPARK-27853 URL: https://issues.apache.org/jira/browse/SPARK-27853 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-27852) One updateBytesWritten operaton may be missed in DiskBlockObjectWriter.scala

[ https://issues.apache.org/jira/browse/SPARK-27852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27852: Assignee: (was: Apache Spark) > One updateBytesWritten operaton may be missed in

[jira] [Assigned] (SPARK-27852) One updateBytesWritten operaton may be missed in DiskBlockObjectWriter.scala

[ https://issues.apache.org/jira/browse/SPARK-27852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27852: Assignee: Apache Spark > One updateBytesWritten operaton may be missed in

[jira] [Updated] (SPARK-27852) One updateBytesWritten operaton may be missed in DiskBlockObjectWriter.scala

[ https://issues.apache.org/jira/browse/SPARK-27852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuaiqi Ge updated SPARK-27852: --- Description:  In *_DiskBlockObjectWriter.scala_*, there are 2 overload *_write_* functions, the

[jira] [Updated] (SPARK-27852) One updateBytesWritten operaton may be missed in DiskBlockObjectWriter.scala

[ https://issues.apache.org/jira/browse/SPARK-27852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuaiqi Ge updated SPARK-27852: --- Description:  In *_DiskBlockObjectWriter.scala_*, there are 2 overload *_write_* functions, the

[jira] [Created] (SPARK-27852) One updateBytesWritten operaton may be missed in DiskBlockObjectWriter.scala

Shuaiqi Ge created SPARK-27852: -- Summary: One updateBytesWritten operaton may be missed in DiskBlockObjectWriter.scala Key: SPARK-27852 URL: https://issues.apache.org/jira/browse/SPARK-27852 Project:

[jira] [Updated] (SPARK-27851) Allow for custom BroadcastMode return values

[ https://issues.apache.org/jira/browse/SPARK-27851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marc Arndt updated SPARK-27851: --- Description: According to the BroadcastMode API the BroadcastMode#transform methods are allows to

[jira] [Created] (SPARK-27851) Allow for custom BroadcastMode return values

Marc Arndt created SPARK-27851: -- Summary: Allow for custom BroadcastMode return values Key: SPARK-27851 URL: https://issues.apache.org/jira/browse/SPARK-27851 Project: Spark Issue Type:

[jira] [Commented] (SPARK-27648) In Spark2.4 Structured Streaming：The executor storage memory increasing over time

[ https://issues.apache.org/jira/browse/SPARK-27648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16848702#comment-16848702 ] Gabor Somogyi commented on SPARK-27648: --- OK. Could you please calculate the 2 state graphs like

[jira] [Created] (SPARK-27850) Make SparkPlan#doExecuteBroadcast public

Marc Arndt created SPARK-27850: -- Summary: Make SparkPlan#doExecuteBroadcast public Key: SPARK-27850 URL: https://issues.apache.org/jira/browse/SPARK-27850 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-27849) Redact treeString of FileTable and DataSourceV2ScanExecBase

[ https://issues.apache.org/jira/browse/SPARK-27849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27849: Assignee: Apache Spark > Redact treeString of FileTable and DataSourceV2ScanExecBase >

[jira] [Assigned] (SPARK-27849) Redact treeString of FileTable and DataSourceV2ScanExecBase

[ https://issues.apache.org/jira/browse/SPARK-27849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27849: Assignee: (was: Apache Spark) > Redact treeString of FileTable and

[jira] [Created] (SPARK-27849) Redact treeString of FileTable and DataSourceV2ScanExecBase

Gengliang Wang created SPARK-27849: -- Summary: Redact treeString of FileTable and DataSourceV2ScanExecBase Key: SPARK-27849 URL: https://issues.apache.org/jira/browse/SPARK-27849 Project: Spark

[jira] [Assigned] (SPARK-27322) DataSourceV2: Select from multiple catalogs

[ https://issues.apache.org/jira/browse/SPARK-27322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27322: Assignee: Apache Spark > DataSourceV2: Select from multiple catalogs >

[jira] [Assigned] (SPARK-27322) DataSourceV2: Select from multiple catalogs

[ https://issues.apache.org/jira/browse/SPARK-27322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27322: Assignee: (was: Apache Spark) > DataSourceV2: Select from multiple catalogs >

[jira] [Commented] (SPARK-13283) Spark doesn't escape column names when creating table on JDBC

[ https://issues.apache.org/jira/browse/SPARK-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16848667#comment-16848667 ] Hyukjin Kwon commented on SPARK-13283: -- It's just closed due to EOL affect version set. See

[jira] [Commented] (SPARK-13283) Spark doesn't escape column names when creating table on JDBC

[ https://issues.apache.org/jira/browse/SPARK-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16848660#comment-16848660 ] Maciej Bryński commented on SPARK-13283: [~hyukjin.kwon] What is resolution: incomplete ? >