[jira] [Resolved] (SPARK-28356) Do not reduce the number of partitions for repartition in adaptive execution

[ https://issues.apache.org/jira/browse/SPARK-28356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-28356. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25121

[jira] [Assigned] (SPARK-28356) Do not reduce the number of partitions for repartition in adaptive execution

[ https://issues.apache.org/jira/browse/SPARK-28356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-28356: --- Assignee: Carson Wang > Do not reduce the number of partitions for repartition in adaptive

[jira] [Resolved] (SPARK-27485) Certain query plans fail to run when autoBroadcastJoinThreshold is set to -1

[ https://issues.apache.org/jira/browse/SPARK-27485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-27485. - Resolution: Fixed Assignee: Herman van Hovell Fix Version/s: 3.0.0 > Certain

[jira] [Created] (SPARK-28411) insertInto with overwrite inconsistent behaviour Python/Scala

Maria Rebelka created SPARK-28411: - Summary: insertInto with overwrite inconsistent behaviour Python/Scala Key: SPARK-28411 URL: https://issues.apache.org/jira/browse/SPARK-28411 Project: Spark

[jira] [Updated] (SPARK-28412) ANSI SQL: OVERLAY function support byte array(T312)

[ https://issues.apache.org/jira/browse/SPARK-28412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-28412: --- Description: ||Function||Return Type||Description||Example||Result|| |{{overlay(_{{string_}} 

[jira] [Commented] (SPARK-28412) ANSI SQL: OVERLAY function support byte array(T312)

[ https://issues.apache.org/jira/browse/SPARK-28412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885959#comment-16885959 ] jiaan.geng commented on SPARK-28412: I'm working on. > ANSI SQL: OVERLAY function support byte

[jira] [Updated] (SPARK-28412) ANSI SQL: OVERLAY function support byte array(T312)

[ https://issues.apache.org/jira/browse/SPARK-28412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-28412: --- Description: ||Function||Return Type||Description||Example||Result|| |{{overlay(_{{string_}} 

[jira] [Updated] (SPARK-28412) ANSI SQL: OVERLAY function support byte array(T312)

[ https://issues.apache.org/jira/browse/SPARK-28412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-28412: --- Description: ||Function||Return Type||Description||Example||Result|| |{{overlay(_string_}} placing 

[jira] [Updated] (SPARK-28412) ANSI SQL: OVERLAY function support byte array(T312)

[ https://issues.apache.org/jira/browse/SPARK-28412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-28412: --- Description: ||Function||Return Type||Description||Example||Result|| |{{overlay(_{{string}}_ 

[jira] [Resolved] (SPARK-28129) Add float8.sql

[ https://issues.apache.org/jira/browse/SPARK-28129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28129. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24931

[jira] [Assigned] (SPARK-28129) Add float8.sql

[ https://issues.apache.org/jira/browse/SPARK-28129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-28129: Assignee: Yuming Wang > Add float8.sql > -- > > Key:

[jira] [Created] (SPARK-28412) ANSI SQL: OVERLAY function support byte array(T312)

jiaan.geng created SPARK-28412: -- Summary: ANSI SQL: OVERLAY function support byte array(T312) Key: SPARK-28412 URL: https://issues.apache.org/jira/browse/SPARK-28412 Project: Spark Issue Type:

[jira] [Commented] (SPARK-28343) PostgreSQL test should change some default config

[ https://issues.apache.org/jira/browse/SPARK-28343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885967#comment-16885967 ] Yuming Wang commented on SPARK-28343: - Add {{set spark.sql.function.preferIntegralDivision=true}} by

[jira] [Updated] (SPARK-28411) insertInto with overwrite inconsistent behaviour Python/Scala

[ https://issues.apache.org/jira/browse/SPARK-28411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maria Rebelka updated SPARK-28411: -- Description: The df.write.mode("overwrite").insertInto("table") has inconsistent behaviour

[jira] [Commented] (SPARK-27821) Spark WebUI - show numbers of drivers/apps in waiting/submitted/killed/running state

[ https://issues.apache.org/jira/browse/SPARK-27821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886226#comment-16886226 ] Sean Owen commented on SPARK-27821: --- Likewise I think this is more noise on the UI without a lot of

[jira] [Updated] (SPARK-27822) Spark WebUi - for running applications have a drivername column

[ https://issues.apache.org/jira/browse/SPARK-27822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-27822: -- Priority: Minor (was: Major) You can already get this info from within the app. I don't see much

[jira] [Updated] (SPARK-27033) Add rule to optimize binary comparisons to its push down format

[ https://issues.apache.org/jira/browse/SPARK-27033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27033: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Add rule to optimize

[jira] [Updated] (SPARK-24497) ANSI SQL: Recursive query

[ https://issues.apache.org/jira/browse/SPARK-24497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24497: -- Affects Version/s: (was: 2.4.0) 3.0.0 > ANSI SQL: Recursive query

[jira] [Commented] (SPARK-14948) Exception when joining DataFrames derived form the same DataFrame

[ https://issues.apache.org/jira/browse/SPARK-14948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886213#comment-16886213 ] Abdulhafeth Salah commented on SPARK-14948: --- The work around for this issue is a bit ugly, is

[jira] [Updated] (SPARK-28343) PostgreSQL test should change some default config

[ https://issues.apache.org/jira/browse/SPARK-28343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28343: -- Description: {noformat} set spark.sql.crossJoin.enabled=true; set

[jira] [Commented] (SPARK-28343) PostgreSQL test should change some default config

[ https://issues.apache.org/jira/browse/SPARK-28343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886230#comment-16886230 ] Dongjoon Hyun commented on SPARK-28343: --- https://github.com/apache/spark/pull/25170 is merged,

[jira] [Commented] (SPARK-28366) Logging in driver when loading single large gzipped file via sc.textFile

[ https://issues.apache.org/jira/browse/SPARK-28366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886229#comment-16886229 ] Sean Owen commented on SPARK-28366: --- ... what do you want to log? > Logging in driver when loading

[jira] [Created] (SPARK-28413) sizeInByte is Not updated for parquet datasource on Next Insert.

Babulal created SPARK-28413: --- Summary: sizeInByte is Not updated for parquet datasource on Next Insert. Key: SPARK-28413 URL: https://issues.apache.org/jira/browse/SPARK-28413 Project: Spark

[jira] [Updated] (SPARK-24814) Relationship between catalog and datasources

[ https://issues.apache.org/jira/browse/SPARK-24814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24814: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Relationship between

[jira] [Updated] (SPARK-27457) modify bean encoder to support avro objects

[ https://issues.apache.org/jira/browse/SPARK-27457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27457: -- Affects Version/s: (was: 2.4.1) 3.0.0 > modify bean encoder to

[jira] [Updated] (SPARK-25299) Use remote storage for persisting shuffle data

[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25299: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Use remote storage for

[jira] [Updated] (SPARK-23985) predicate push down doesn't work with simple compound partition spec

[ https://issues.apache.org/jira/browse/SPARK-23985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23985: -- Affects Version/s: (was: 2.4.0) 3.0.0 > predicate push down

[jira] [Updated] (SPARK-26854) Support ANY/SOME subquery

[ https://issues.apache.org/jira/browse/SPARK-26854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26854: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Support ANY/SOME subquery

[jira] [Updated] (SPARK-24818) Ensure all the barrier tasks in the same stage are launched together

[ https://issues.apache.org/jira/browse/SPARK-24818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24818: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Ensure all the barrier

[jira] [Updated] (SPARK-27204) First time Loading application page from History Server is taking time when event log size is huge

[ https://issues.apache.org/jira/browse/SPARK-27204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27204: -- Affects Version/s: (was: 2.3.3) (was: 2.4.0)

[jira] [Updated] (SPARK-27950) Additional DynamoDB and CloudWatch config

[ https://issues.apache.org/jira/browse/SPARK-27950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27950: -- Affects Version/s: (was: 2.4.3) 3.0.0 > Additional DynamoDB and

[jira] [Updated] (SPARK-28292) Enable inject user-defined Hint

[ https://issues.apache.org/jira/browse/SPARK-28292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28292: -- Affects Version/s: (was: 2.4.0) (was: 2.3.0)

[jira] [Updated] (SPARK-25216) Provide better error message when a column contains dot and needs backticks quote

[ https://issues.apache.org/jira/browse/SPARK-25216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25216: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Provide better error

[jira] [Updated] (SPARK-27658) Catalog API to load functions

[ https://issues.apache.org/jira/browse/SPARK-27658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27658: -- Affects Version/s: (was: 2.4.3) 3.0.0 > Catalog API to load

[jira] [Updated] (SPARK-28265) Missing TableCatalog API to rename table

[ https://issues.apache.org/jira/browse/SPARK-28265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28265: -- Affects Version/s: (was: 2.4.3) 3.0.0 > Missing TableCatalog API

[jira] [Updated] (SPARK-24283) Make standard scaler work without legacy MLlib

[ https://issues.apache.org/jira/browse/SPARK-24283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24283: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Make standard scaler work

[jira] [Updated] (SPARK-28303) Support DELETE/UPDATE/MERGE Operations in DataSource V2

[ https://issues.apache.org/jira/browse/SPARK-28303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28303: -- Affects Version/s: (was: 2.4.3) 3.0.0 > Support

[jira] [Updated] (SPARK-28008) Default values & column comments in AVRO schema converters

[ https://issues.apache.org/jira/browse/SPARK-28008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28008: -- Affects Version/s: (was: 2.4.3) 3.0.0 > Default values & column

[jira] [Updated] (SPARK-26544) escape string when serialize map/array to make it a valid json (keep alignment with hive)

[ https://issues.apache.org/jira/browse/SPARK-26544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26544: -- Affects Version/s: (was: 2.4.0) 3.0.0 > escape string when

[jira] [Updated] (SPARK-23758) MLlib 2.4 Roadmap

[ https://issues.apache.org/jira/browse/SPARK-23758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23758: -- Affects Version/s: (was: 2.4.0) 3.0.0 > MLlib 2.4 Roadmap >

[jira] [Updated] (SPARK-26271) remove unuse object SparkPlan

[ https://issues.apache.org/jira/browse/SPARK-26271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26271: -- Affects Version/s: (was: 2.4.1) 3.0.0 > remove unuse object

[jira] [Updated] (SPARK-27093) Honor ParseMode in AvroFileFormat

[ https://issues.apache.org/jira/browse/SPARK-27093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27093: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Honor ParseMode in

[jira] [Updated] (SPARK-26354) Ability to return schema prefix before dataframe column names

[ https://issues.apache.org/jira/browse/SPARK-26354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26354: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Ability to return schema

[jira] [Updated] (SPARK-26376) Skip inputs without tokens by JSON datasource

[ https://issues.apache.org/jira/browse/SPARK-26376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26376: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Skip inputs without tokens

[jira] [Updated] (SPARK-25151) Apply Apache Commons Pool to KafkaDataConsumer

[ https://issues.apache.org/jira/browse/SPARK-25151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25151: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Apply Apache Commons Pool

[jira] [Updated] (SPARK-24634) Add a new metric regarding number of rows later than watermark

[ https://issues.apache.org/jira/browse/SPARK-24634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24634: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Add a new metric regarding

[jira] [Updated] (SPARK-23609) Test EnsureRequirements's test cases to eliminate ShuffleExchange while is not expected

[ https://issues.apache.org/jira/browse/SPARK-23609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23609: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Test EnsureRequirements's

[jira] [Updated] (SPARK-28006) User-defined grouped transform pandas_udf for window operations

[ https://issues.apache.org/jira/browse/SPARK-28006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28006: -- Affects Version/s: (was: 2.4.3) 3.0.0 > User-defined grouped

[jira] [Updated] (SPARK-27319) Filter out dir based on PathFilter before listing them

[ https://issues.apache.org/jira/browse/SPARK-27319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27319: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Filter out dir based on

[jira] [Updated] (SPARK-27750) Standalone scheduler - ability to prioritize applications over drivers, many drivers act like Denial of Service

[ https://issues.apache.org/jira/browse/SPARK-27750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27750: -- Affects Version/s: (was: 2.4.3) (was: 2.3.3)

[jira] [Updated] (SPARK-25360) Parallelized RDDs of Ranges could have known partitioner

[ https://issues.apache.org/jira/browse/SPARK-25360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25360: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Parallelized RDDs of

[jira] [Updated] (SPARK-24707) Enable spark-kafka-streaming to maintain min buffer using async thread to avoid blocking kafka poll

[ https://issues.apache.org/jira/browse/SPARK-24707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24707: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Enable

[jira] [Updated] (SPARK-23443) Spark with Glue as external catalog

[ https://issues.apache.org/jira/browse/SPARK-23443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23443: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Spark with Glue as

[jira] [Updated] (SPARK-27707) Performance issue using explode

[ https://issues.apache.org/jira/browse/SPARK-27707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27707: -- Affects Version/s: (was: 2.4.3) > Performance issue using explode >

[jira] [Updated] (SPARK-23411) Deprecate SparkContext.getRDDStorageInfo

[ https://issues.apache.org/jira/browse/SPARK-23411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23411: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Deprecate

[jira] [Updated] (SPARK-24855) Built-in AVRO support should support specified schema on write

[ https://issues.apache.org/jira/browse/SPARK-24855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24855: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Built-in AVRO support

[jira] [Updated] (SPARK-25802) Use JDBC Oracle Binds from Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-25802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25802: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Use JDBC Oracle Binds from

[jira] [Updated] (SPARK-27661) Add SupportsNamespaces interface for v2 catalogs

[ https://issues.apache.org/jira/browse/SPARK-27661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27661: -- Affects Version/s: (was: 2.4.3) 3.0.0 > Add SupportsNamespaces

[jira] [Updated] (SPARK-26957) Add config properties to configure the default scheduler pool priorities

[ https://issues.apache.org/jira/browse/SPARK-26957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26957: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Add config properties to

[jira] [Updated] (SPARK-27603) Make ShuffleClient pluggable

[ https://issues.apache.org/jira/browse/SPARK-27603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27603: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Make ShuffleClient

[jira] [Updated] (SPARK-28148) repartition after join is not optimized away

[ https://issues.apache.org/jira/browse/SPARK-28148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28148: -- Affects Version/s: (was: 2.4.3) 3.0.0 > repartition after join is

[jira] [Updated] (SPARK-27593) CSV Parser returns 2 DataFrame - Valid and Malformed DFs

[ https://issues.apache.org/jira/browse/SPARK-27593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27593: -- Affects Version/s: (was: 2.4.2) 3.0.0 > CSV Parser returns 2

[jira] [Updated] (SPARK-26321) Split a SQL in a correct way

[ https://issues.apache.org/jira/browse/SPARK-26321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26321: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Split a SQL in a correct

[jira] [Updated] (SPARK-24066) Add new optimization rule to eliminate unnecessary sort by exchanged adjacent Window expressions

[ https://issues.apache.org/jira/browse/SPARK-24066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24066: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Add new optimization rule

[jira] [Updated] (SPARK-23544) Remove redundancy ShuffleExchange in the planner

[ https://issues.apache.org/jira/browse/SPARK-23544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23544: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Remove redundancy

[jira] [Updated] (SPARK-24282) Add support for PMML export for the Standard Scaler Stage

[ https://issues.apache.org/jira/browse/SPARK-24282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24282: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Add support for PMML

[jira] [Updated] (SPARK-24467) VectorAssemblerEstimator

[ https://issues.apache.org/jira/browse/SPARK-24467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24467: -- Affects Version/s: (was: 2.4.0) 3.0.0 > VectorAssemblerEstimator >

[jira] [Updated] (SPARK-28149) Disable negeative DNS caching

[ https://issues.apache.org/jira/browse/SPARK-28149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28149: -- Affects Version/s: (was: 2.4.3) 3.0.0 > Disable negeative DNS

[jira] [Updated] (SPARK-27915) Update logical Filter's output nullability based on IsNotNull conditions

[ https://issues.apache.org/jira/browse/SPARK-27915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27915: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Update logical Filter's

[jira] [Updated] (SPARK-27792) SkewJoin--handle only skewed keys with broadcastjoin and other keys with normal join

[ https://issues.apache.org/jira/browse/SPARK-27792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27792: -- Affects Version/s: (was: 2.4.3) 3.0.0 > SkewJoin--handle only

[jira] [Updated] (SPARK-23661) Implement treeAggregate on Dataset API

[ https://issues.apache.org/jira/browse/SPARK-23661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23661: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Implement treeAggregate on

[jira] [Updated] (SPARK-25556) Predicate Pushdown for Nested fields

[ https://issues.apache.org/jira/browse/SPARK-25556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25556: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Predicate Pushdown for

[jira] [Updated] (SPARK-26875) Add an option on FileStreamSource to include modified files

[ https://issues.apache.org/jira/browse/SPARK-26875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26875: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Add an option on

[jira] [Updated] (SPARK-24393) SQL builtin: isinf

[ https://issues.apache.org/jira/browse/SPARK-24393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24393: -- Affects Version/s: (was: 2.4.0) 3.0.0 > SQL builtin: isinf >

[jira] [Updated] (SPARK-27911) PySpark Packages should automatically choose correct scala version

[ https://issues.apache.org/jira/browse/SPARK-27911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27911: -- Affects Version/s: (was: 2.4.3) 3.0.0 > PySpark Packages should

[jira] [Updated] (SPARK-23894) Flaky Test: BucketedWriteWithoutHiveSupportSuite

[ https://issues.apache.org/jira/browse/SPARK-23894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23894: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Flaky Test:

[jira] [Updated] (SPARK-23798) The CreateArray and ConcatArray should return the default array type when no children provided

[ https://issues.apache.org/jira/browse/SPARK-23798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23798: -- Affects Version/s: (was: 2.4.0) 3.0.0 > The CreateArray and

[jira] [Updated] (SPARK-26524) If the application directory fails to be created on the SPARK_WORKER_DIR on some woker nodes (for example, bad disk or disk has no capacity), the application executor wi

[ https://issues.apache.org/jira/browse/SPARK-26524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26524: -- Affects Version/s: (was: 2.4.0) 3.0.0 > If the application

[jira] [Updated] (SPARK-28098) Native ORC reader doesn't support subdirectories with Hive tables

[ https://issues.apache.org/jira/browse/SPARK-28098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28098: -- Affects Version/s: (was: 2.4.3) 3.0.0 > Native ORC reader doesn't

[jira] [Updated] (SPARK-27739) df.persist should save stats from optimized plan

[ https://issues.apache.org/jira/browse/SPARK-27739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27739: -- Affects Version/s: (was: 2.4.0) (was: 2.3.0)

[jira] [Updated] (SPARK-27753) Support SQL expressions for interval parameter in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-27753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27753: -- Affects Version/s: (was: 2.4.3) 3.0.0 > Support SQL expressions

[jira] [Updated] (SPARK-27214) Upgrading locality level when lots of pending tasks have been waiting more than locality.wait

[ https://issues.apache.org/jira/browse/SPARK-27214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27214: -- Affects Version/s: (was: 2.4.0) (was: 2.1.0)

[jira] [Updated] (SPARK-27295) Provision to provide initial values for each source node in personalised page rank - Graphx

[ https://issues.apache.org/jira/browse/SPARK-27295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27295: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Provision to provide

[jira] [Commented] (SPARK-27781) Tried to access method org.apache.avro.specific.SpecificData.()V

[ https://issues.apache.org/jira/browse/SPARK-27781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886237#comment-16886237 ] Michael Heuer commented on SPARK-27781: --- I believe I saw a fix for this specific issue, where the

[jira] [Commented] (SPARK-19477) [SQL] Datasets created from a Dataframe with extra columns retain the extra columns

[ https://issues.apache.org/jira/browse/SPARK-19477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886192#comment-16886192 ] David Lo commented on SPARK-19477: -- [~cloud_fan] Could you please elaborate on your suggested

[jira] [Resolved] (SPARK-28368) Row.getAs() return different values in scala and java

[ https://issues.apache.org/jira/browse/SPARK-28368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28368. --- Resolution: Not A Problem > Row.getAs() return different values in scala and java >

[jira] [Updated] (SPARK-28413) sizeInByte is Not updated for parquet datasource on Next Insert.

[ https://issues.apache.org/jira/browse/SPARK-28413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Babulal updated SPARK-28413: Description: In  SPARK-21237 (link SPARK-21237)  it is fix when Appending data using  

[jira] [Updated] (SPARK-26912) Allow setting permission for event_log

[ https://issues.apache.org/jira/browse/SPARK-26912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26912: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Allow setting permission

[jira] [Updated] (SPARK-27679) Improve queries with LIKE expression

[ https://issues.apache.org/jira/browse/SPARK-27679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27679: -- Affects Version/s: (was: 2.4.3) 3.0.0 > Improve queries with LIKE

[jira] [Updated] (SPARK-24528) Missing optimization for Aggregations/Windowing on a bucketed table

[ https://issues.apache.org/jira/browse/SPARK-24528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24528: -- Affects Version/s: (was: 2.4.0) (was: 2.3.0)

[jira] [Updated] (SPARK-27790) Support SQL INTERVAL types

[ https://issues.apache.org/jira/browse/SPARK-27790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27790: -- Affects Version/s: (was: 2.4.3) 3.0.0 > Support SQL INTERVAL types

[jira] [Updated] (SPARK-25894) Include a count of the number of physical columns read for a columnar data source in the metadata of FileSourceScanExec

[ https://issues.apache.org/jira/browse/SPARK-25894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25894: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Include a count of the

[jira] [Updated] (SPARK-28070) writeType and writeObject in SparkR should be handled by S3 methods

[ https://issues.apache.org/jira/browse/SPARK-28070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28070: -- Affects Version/s: (was: 2.4.3) 3.0.0 > writeType and writeObject

[jira] [Updated] (SPARK-23678) a more efficient partition strategy

[ https://issues.apache.org/jira/browse/SPARK-23678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23678: -- Affects Version/s: (was: 2.4.0) 3.0.0 > a more efficient partition

[jira] [Updated] (SPARK-24914) totalSize is not a good estimate for broadcast joins

[ https://issues.apache.org/jira/browse/SPARK-24914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24914: -- Affects Version/s: (was: 2.4.0) 3.0.0 > totalSize is not a good

[jira] [Updated] (SPARK-24513) Attribute support in UnaryTransformer

[ https://issues.apache.org/jira/browse/SPARK-24513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24513: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Attribute support in

[jira] [Updated] (SPARK-25083) remove the type erasure hack in data source scan

[ https://issues.apache.org/jira/browse/SPARK-25083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25083: -- Affects Version/s: (was: 2.4.0) 3.0.0 > remove the type erasure

[jira] [Updated] (SPARK-25236) Investigate using a logging library inside of PySpark on the workers instead of print

[ https://issues.apache.org/jira/browse/SPARK-25236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25236: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Investigate using a

[jira] [Updated] (SPARK-27171) Support Full-Partiton limit in the first scan

[ https://issues.apache.org/jira/browse/SPARK-27171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27171: -- Affects Version/s: (was: 2.3.2) (was: 2.4.0)

[jira] [Updated] (SPARK-26104) make pci devices visible to task scheduler

[ https://issues.apache.org/jira/browse/SPARK-26104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26104: -- Affects Version/s: (was: 2.4.0) 3.0.0 > make pci devices visible

  1   2   3   >