[jira] [Resolved] (SPARK-31016) [DEPLOY] Pack the user jars when submitting Spark Application

[ https://issues.apache.org/jira/browse/SPARK-31016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31016. -- Resolution: Won't Fix You can always add jars manually by many APIs such as {{sparkContext.add

[jira] [Assigned] (SPARK-31030) Backward Compatibility for Parsing and Formatting Datetime

[ https://issues.apache.org/jira/browse/SPARK-31030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31030: --- Assignee: Yuanjian Li > Backward Compatibility for Parsing and Formatting Datetime > --

[jira] [Resolved] (SPARK-31030) Backward Compatibility for Parsing and Formatting Datetime

[ https://issues.apache.org/jira/browse/SPARK-31030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31030. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27830 [https://gith

[jira] [Updated] (SPARK-31039) Unable to use vendor specific datatypes with JDBC (MSSQL)

[ https://issues.apache.org/jira/browse/SPARK-31039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31039: - Summary: Unable to use vendor specific datatypes with JDBC (MSSQL) (was: Unable to use vendor s

[jira] [Updated] (SPARK-31112) Use multiple extrenal catalog to speed up metastore access

[ https://issues.apache.org/jira/browse/SPARK-31112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] deshanxiao updated SPARK-31112: --- Description: Now, we use HiveClientImpl to access hive metastore. However, a long running rpc in h

[jira] [Resolved] (SPARK-31042) Error in writing a pyspark streaming dataframe created from Kafka source to a csv file

[ https://issues.apache.org/jira/browse/SPARK-31042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31042. -- Resolution: Invalid Sounds more like a question. Seems such cases are being tested in Spark si

[jira] [Updated] (SPARK-31112) Use multiple extrenal catalog to speed up metastore access

[ https://issues.apache.org/jira/browse/SPARK-31112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] deshanxiao updated SPARK-31112: --- Description: Now, we use HiveClientImpl to access hive metastore. However, a long running rpc in hi

[jira] [Updated] (SPARK-31112) Use multiple extrenal catalog to speed up metastore access

[ https://issues.apache.org/jira/browse/SPARK-31112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] deshanxiao updated SPARK-31112: --- Description: Now, we use HiveClientImpl to access hive metastore. However, a long running rpc in hi

[jira] [Resolved] (SPARK-31048) alter hive column datatype is not supported

[ https://issues.apache.org/jira/browse/SPARK-31048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31048. -- Resolution: Incomplete 2.2.2 is EOL release. Can you try this in Spark 2.4? > alter hive colu

[jira] [Resolved] (SPARK-31057) approxQuantile function of spark , not taking List as first parameter

[ https://issues.apache.org/jira/browse/SPARK-31057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31057. -- Resolution: Invalid > approxQuantile function of spark , not taking List as first parameter >

[jira] [Commented] (SPARK-31057) approxQuantile function of spark , not taking List as first parameter

[ https://issues.apache.org/jira/browse/SPARK-31057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056693#comment-17056693 ] Hyukjin Kwon commented on SPARK-31057: -- It sounds more like a question. Questions s

[jira] [Commented] (SPARK-30896) The behavior of JsonToStructs should not depend on SQLConf.get

[ https://issues.apache.org/jira/browse/SPARK-30896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056692#comment-17056692 ] Takeshi Yamamuro commented on SPARK-30896: -- Yea, +1 on the Wenchen suggestion;

[jira] [Resolved] (SPARK-31067) Spark 2.4.* SQL query with partition columns scans entire AVRO data

[ https://issues.apache.org/jira/browse/SPARK-31067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31067. -- Resolution: Cannot Reproduce > Spark 2.4.* SQL query with partition columns scans entire AVRO

[jira] [Updated] (SPARK-31057) approxQuantile function of spark , not taking List as first parameter

[ https://issues.apache.org/jira/browse/SPARK-31057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31057: - Component/s: (was: Spark Core) SQL > approxQuantile function of spark , no

[jira] [Commented] (SPARK-31057) approxQuantile function of spark , not taking List as first parameter

[ https://issues.apache.org/jira/browse/SPARK-31057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056689#comment-17056689 ] Hyukjin Kwon commented on SPARK-31057: -- Please avoid to set the target version whic

[jira] [Commented] (SPARK-31067) Spark 2.4.* SQL query with partition columns scans entire AVRO data

[ https://issues.apache.org/jira/browse/SPARK-31067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056688#comment-17056688 ] Hyukjin Kwon commented on SPARK-31067: -- [~dirrao], please clarify the steps to repr

[jira] [Updated] (SPARK-31057) approxQuantile function of spark , not taking List as first parameter

[ https://issues.apache.org/jira/browse/SPARK-31057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31057: - Target Version/s: (was: 2.4.3) > approxQuantile function of spark , not taking List as first

[jira] [Resolved] (SPARK-31083) .ClassNotFoundException CoarseGrainedClusterMessages$RetrieveDelegationTokens

[ https://issues.apache.org/jira/browse/SPARK-31083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31083. -- Resolution: Invalid > .ClassNotFoundException CoarseGrainedClusterMessages$RetrieveDelegationT

[jira] [Commented] (SPARK-31074) Avro serializer should not fail when a nullable Spark field is written to a non-null Avro column

[ https://issues.apache.org/jira/browse/SPARK-31074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056685#comment-17056685 ] Hyukjin Kwon commented on SPARK-31074: -- [~kyrill007] please show the steps to repro

[jira] [Resolved] (SPARK-31084) spark on k8s Exception "Database xxx not found" when hive MetaStoreClient lost connection

[ https://issues.apache.org/jira/browse/SPARK-31084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31084. -- Resolution: Invalid > spark on k8s Exception "Database xxx not found" when hive MetaStoreClien

[jira] [Commented] (SPARK-31084) spark on k8s Exception "Database xxx not found" when hive MetaStoreClient lost connection

[ https://issues.apache.org/jira/browse/SPARK-31084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056684#comment-17056684 ] Hyukjin Kwon commented on SPARK-31084: -- Please ask questions into mailing list, and

[jira] [Resolved] (SPARK-31108) Parameter cannot be passed to pandas udf of type map_iter

[ https://issues.apache.org/jira/browse/SPARK-31108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31108. -- Resolution: Invalid Please ask questions into the dev mailing list (https://spark.apache.org/

[jira] [Commented] (SPARK-31112) Use multiple extrenal catalog to speed up metastore access

[ https://issues.apache.org/jira/browse/SPARK-31112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056681#comment-17056681 ] Hyukjin Kwon commented on SPARK-31112: -- [~deshanxiao], can you fill the JIRA descri

[jira] [Created] (SPARK-31114) isNotNull should be inferred from equality constraint without cast

Yuming Wang created SPARK-31114: --- Summary: isNotNull should be inferred from equality constraint without cast Key: SPARK-31114 URL: https://issues.apache.org/jira/browse/SPARK-31114 Project: Spark

[jira] [Resolved] (SPARK-30895) The behavior of CsvToStructs should not depend on SQLConf.get

[ https://issues.apache.org/jira/browse/SPARK-30895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30895. -- Resolution: Later > The behavior of CsvToStructs should not depend on SQLConf.get > --

[jira] [Resolved] (SPARK-30896) The behavior of JsonToStructs should not depend on SQLConf.get

[ https://issues.apache.org/jira/browse/SPARK-30896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30896. -- Resolution: Later > The behavior of JsonToStructs should not depend on SQLConf.get > -

[jira] [Commented] (SPARK-30896) The behavior of JsonToStructs should not depend on SQLConf.get

[ https://issues.apache.org/jira/browse/SPARK-30896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056672#comment-17056672 ] Hyukjin Kwon commented on SPARK-30896: -- Yeah, let's don't fix it for now. > The be

[jira] [Commented] (SPARK-31113) Support DDL "SHOW VIEWS"

[ https://issues.apache.org/jira/browse/SPARK-31113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056669#comment-17056669 ] Xin Wu commented on SPARK-31113: Sure, I'm working on this! Thanks [~smilegator] > Supp

[jira] [Resolved] (SPARK-31070) make skew join split skewed partitions more evenly

[ https://issues.apache.org/jira/browse/SPARK-31070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-31070. - Fix Version/s: 3.0.0 Resolution: Fixed > make skew join split skewed partitions more evenly > ---

[jira] [Commented] (SPARK-30896) The behavior of JsonToStructs should not depend on SQLConf.get

[ https://issues.apache.org/jira/browse/SPARK-30896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056636#comment-17056636 ] Wenchen Fan commented on SPARK-30896: - More importantly, what should be the official

[jira] [Commented] (SPARK-30895) The behavior of CsvToStructs should not depend on SQLConf.get

[ https://issues.apache.org/jira/browse/SPARK-30895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056634#comment-17056634 ] Wenchen Fan commented on SPARK-30895: - See https://issues.apache.org/jira/browse/SPA

[jira] [Updated] (SPARK-30895) The behavior of CsvToStructs should not depend on SQLConf.get

[ https://issues.apache.org/jira/browse/SPARK-30895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-30895: Priority: Minor (was: Major) > The behavior of CsvToStructs should not depend on SQLConf.get > --

[jira] [Commented] (SPARK-31099) Create migration script for metastore_db

[ https://issues.apache.org/jira/browse/SPARK-31099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056633#comment-17056633 ] Wenchen Fan commented on SPARK-31099: - Is this only a problem for local hive metasto

[jira] [Reopened] (SPARK-30895) The behavior of CsvToStructs should not depend on SQLConf.get

[ https://issues.apache.org/jira/browse/SPARK-30895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reopened SPARK-30895: - > The behavior of CsvToStructs should not depend on SQLConf.get > --

[jira] [Commented] (SPARK-30896) The behavior of JsonToStructs should not depend on SQLConf.get

[ https://issues.apache.org/jira/browse/SPARK-30896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056632#comment-17056632 ] Wenchen Fan commented on SPARK-30896: - `JsonToStructs` already stores the config val

[jira] [Reopened] (SPARK-30896) The behavior of JsonToStructs should not depend on SQLConf.get

[ https://issues.apache.org/jira/browse/SPARK-30896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reopened SPARK-30896: - > The behavior of JsonToStructs should not depend on SQLConf.get > -

[jira] [Updated] (SPARK-30896) The behavior of JsonToStructs should not depend on SQLConf.get

[ https://issues.apache.org/jira/browse/SPARK-30896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-30896: Priority: Minor (was: Major) > The behavior of JsonToStructs should not depend on SQLConf.get > -

[jira] [Updated] (SPARK-30893) Expressions should not change its data type/nullability after it's created

[ https://issues.apache.org/jira/browse/SPARK-30893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-30893: Summary: Expressions should not change its data type/nullability after it's created (was: Express

[jira] [Updated] (SPARK-31113) Support DDL "SHOW VIEWS"

[ https://issues.apache.org/jira/browse/SPARK-31113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-31113: Issue Type: New Feature (was: Bug) > Support DDL "SHOW VIEWS" > > >

[jira] [Comment Edited] (SPARK-31113) Support DDL "SHOW VIEWS"

[ https://issues.apache.org/jira/browse/SPARK-31113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056628#comment-17056628 ] Xiao Li edited comment on SPARK-31113 at 3/11/20, 4:04 AM: --- cc

[jira] [Commented] (SPARK-31113) Support DDL "SHOW VIEWS"

[ https://issues.apache.org/jira/browse/SPARK-31113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056628#comment-17056628 ] Xiao Li commented on SPARK-31113: - cc [~EricWu]  > Support DDL "SHOW VIEWS" > -

[jira] [Created] (SPARK-31113) Support DDL "SHOW VIEWS"

Xiao Li created SPARK-31113: --- Summary: Support DDL "SHOW VIEWS" Key: SPARK-31113 URL: https://issues.apache.org/jira/browse/SPARK-31113 Project: Spark Issue Type: Bug Components: SQL

[jira] [Created] (SPARK-31112) Use multiple extrenal catalog to speed up metastore access

deshanxiao created SPARK-31112: -- Summary: Use multiple extrenal catalog to speed up metastore access Key: SPARK-31112 URL: https://issues.apache.org/jira/browse/SPARK-31112 Project: Spark Issue

[jira] [Resolved] (SPARK-30896) The behavior of JsonToStructs should not depend on SQLConf.get

[ https://issues.apache.org/jira/browse/SPARK-30896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-30896. - Resolution: Won't Fix > The behavior of JsonToStructs should not depend on SQLConf.get > ---

[jira] [Resolved] (SPARK-30895) The behavior of CsvToStructs should not depend on SQLConf.get

[ https://issues.apache.org/jira/browse/SPARK-30895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-30895. - Resolution: Won't Fix > The behavior of CsvToStructs should not depend on SQLConf.get >

[jira] [Resolved] (SPARK-30893) Expressions should not change its data type/behavior after it's created

[ https://issues.apache.org/jira/browse/SPARK-30893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-30893. - Fix Version/s: 3.0.0 Resolution: Fixed > Expressions should not change its data type/behavior aft

[jira] [Commented] (SPARK-31104) Add documentation for all the Json Functions

[ https://issues.apache.org/jira/browse/SPARK-31104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056584#comment-17056584 ] Rakesh Raushan commented on SPARK-31104: I am working on it. > Add documentatio

[jira] [Commented] (SPARK-31095) Upgrade netty-all to 4.1.47.Final

[ https://issues.apache.org/jira/browse/SPARK-31095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056561#comment-17056561 ] Dongjoon Hyun commented on SPARK-31095: --- For `branch-2.4`, https://github.com/apac

[jira] [Resolved] (SPARK-31095) Upgrade netty-all to 4.1.47.Final

[ https://issues.apache.org/jira/browse/SPARK-31095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31095. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27869 [https://

[jira] [Assigned] (SPARK-31095) Upgrade netty-all to 4.1.47.Final

[ https://issues.apache.org/jira/browse/SPARK-31095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-31095: - Assignee: Dongjoon Hyun > Upgrade netty-all to 4.1.47.Final > -

[jira] [Updated] (SPARK-31095) Upgrade netty-all to 4.1.47.Final

[ https://issues.apache.org/jira/browse/SPARK-31095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31095: -- Summary: Upgrade netty-all to 4.1.47.Final (was: Upgrade netty version to fix security vulner

[jira] [Updated] (SPARK-31095) Upgrade netty version to fix security vulnerabilities

[ https://issues.apache.org/jira/browse/SPARK-31095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31095: -- Affects Version/s: (was: 2.4.4) 3.1.0 3.0.0

[jira] [Commented] (SPARK-31099) Create migration script for metastore_db

[ https://issues.apache.org/jira/browse/SPARK-31099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056549#comment-17056549 ] Kris Mok commented on SPARK-31099: -- Just documenting the fact that users may encounter

[jira] [Resolved] (SPARK-30962) Document ALTER TABLE statement in SQL Reference [Phase 2]

[ https://issues.apache.org/jira/browse/SPARK-30962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-30962. -- Fix Version/s: 3.0.0 Assignee: kevin yu Resolution: Fixed Resolved by 

[jira] [Commented] (SPARK-31102) spark-sql fails to parse when contains comment

[ https://issues.apache.org/jira/browse/SPARK-31102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056466#comment-17056466 ] Javier Fuentes commented on SPARK-31102: Hey [~yumwang] I am checking this. Than

[jira] [Commented] (SPARK-31099) Create migration script for metastore_db

[ https://issues.apache.org/jira/browse/SPARK-31099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056465#comment-17056465 ] Jungtaek Lim commented on SPARK-31099: -- [~dongjoon] Could you elaborate your comme

[jira] [Commented] (SPARK-31099) Create migration script for metastore_db

[ https://issues.apache.org/jira/browse/SPARK-31099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056461#comment-17056461 ] Gengliang Wang commented on SPARK-31099: [~dongjoon]Make sense. Let me close thi

[jira] [Resolved] (SPARK-31099) Create migration script for metastore_db

[ https://issues.apache.org/jira/browse/SPARK-31099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-31099. Resolution: Won't Fix > Create migration script for metastore_db > ---

[jira] [Updated] (SPARK-31095) Upgrade netty version to fix security vulnerabilities

[ https://issues.apache.org/jira/browse/SPARK-31095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31095: -- Component/s: (was: Security) Build Fix Version/s: (was: 2.4.5)

[jira] [Updated] (SPARK-31095) Upgrade netty version to fix security vulnerabilities

[ https://issues.apache.org/jira/browse/SPARK-31095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31095: -- Priority: Major (was: Critical) > Upgrade netty version to fix security vulnerabilities > ---

[jira] [Commented] (SPARK-31095) Upgrade netty version to fix security vulnerabilities

[ https://issues.apache.org/jira/browse/SPARK-31095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056444#comment-17056444 ] Dongjoon Hyun commented on SPARK-31095: --- Hi, [~visvijay]. You should not set `Fix

[jira] [Updated] (SPARK-31095) Upgrade netty version to fix security vulnerabilities

[ https://issues.apache.org/jira/browse/SPARK-31095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31095: -- Issue Type: Bug (was: Improvement) > Upgrade netty version to fix security vulnerabilities >

[jira] [Commented] (SPARK-31098) Reading ORC files throws IndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-31098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056439#comment-17056439 ] Dongjoon Hyun commented on SPARK-31098: --- Thank you, [~Gengliang.Wang]. > Reading

[jira] [Commented] (SPARK-31099) Create migration script for metastore_db

[ https://issues.apache.org/jira/browse/SPARK-31099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056436#comment-17056436 ] Dongjoon Hyun commented on SPARK-31099: --- Hi, [~Gengliang.Wang] and [~smilegator] a

[jira] [Updated] (SPARK-31102) spark-sql fails to parse when contains comment

[ https://issues.apache.org/jira/browse/SPARK-31102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31102: -- Target Version/s: 3.0.0 > spark-sql fails to parse when contains comment > ---

[jira] [Resolved] (SPARK-30667) Support simple all gather in barrier task context

[ https://issues.apache.org/jira/browse/SPARK-30667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-30667. -- Resolution: Done > Support simple all gather in barrier task context > ---

[jira] [Updated] (SPARK-28594) Allow event logs for running streaming apps to be rolled over

[ https://issues.apache.org/jira/browse/SPARK-28594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28594: -- Summary: Allow event logs for running streaming apps to be rolled over (was: Allow event logs

[jira] [Updated] (SPARK-22783) event log directory(spark-history) filled by large .inprogress files for spark streaming applications

[ https://issues.apache.org/jira/browse/SPARK-22783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-22783: -- Parent: SPARK-28594 Issue Type: Sub-task (was: Bug) > event log directory(spark-histo

[jira] [Closed] (SPARK-22783) event log directory(spark-history) filled by large .inprogress files for spark streaming applications

[ https://issues.apache.org/jira/browse/SPARK-22783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-22783. - > event log directory(spark-history) filled by large .inprogress files for > spark streaming applic

[jira] [Updated] (SPARK-28594) Allow event logs for running streaming apps to be rolled over.

[ https://issues.apache.org/jira/browse/SPARK-28594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28594: -- Environment: (was: This has been reported on 2.0.2.22 but affects all currently available

[jira] [Commented] (SPARK-28594) Allow event logs for running streaming apps to be rolled over.

[ https://issues.apache.org/jira/browse/SPARK-28594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056417#comment-17056417 ] Dongjoon Hyun commented on SPARK-28594: --- I assigned this umbrella to [~kabhwan] si

[jira] [Assigned] (SPARK-28594) Allow event logs for running streaming apps to be rolled over.

[ https://issues.apache.org/jira/browse/SPARK-28594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-28594: - Assignee: Jungtaek Lim > Allow event logs for running streaming apps to be rolled over.

[jira] [Closed] (SPARK-29581) Enable cleanup old event log files

[ https://issues.apache.org/jira/browse/SPARK-29581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-29581. - > Enable cleanup old event log files > --- > > Key:

[jira] [Updated] (SPARK-30860) Different behavior between rolling and non-rolling event log

[ https://issues.apache.org/jira/browse/SPARK-30860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30860: -- Parent: SPARK-28594 Issue Type: Sub-task (was: Bug) > Different behavior between roll

[jira] [Commented] (SPARK-31098) Reading ORC files throws IndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-31098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056331#comment-17056331 ] Gengliang Wang commented on SPARK-31098: [~dongjoon]Thanks for the explanation!

[jira] [Resolved] (SPARK-31098) Reading ORC files throws IndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-31098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-31098. Resolution: Later > Reading ORC files throws IndexOutOfBoundsException > -

[jira] [Updated] (SPARK-31098) Reading ORC files throws IndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-31098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31098: -- Issue Type: Bug (was: Improvement) > Reading ORC files throws IndexOutOfBoundsException > ---

[jira] [Commented] (SPARK-31098) Reading ORC files throws IndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-31098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056326#comment-17056326 ] Dongjoon Hyun commented on SPARK-31098: --- Please note that the fixed behavior is no

[jira] [Commented] (SPARK-31098) Reading ORC files throws IndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-31098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056324#comment-17056324 ] Dongjoon Hyun commented on SPARK-31098: --- It will work only in the above cases (I a

[jira] [Updated] (SPARK-31110) refine sql doc for SELECT

[ https://issues.apache.org/jira/browse/SPARK-31110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31110: -- Issue Type: Documentation (was: Improvement) > refine sql doc for SELECT > --

[jira] [Updated] (SPARK-31110) refine sql doc for SELECT

[ https://issues.apache.org/jira/browse/SPARK-31110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31110: -- Component/s: Documentation > refine sql doc for SELECT > - > >

[jira] [Commented] (SPARK-31098) Reading ORC files throws IndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-31098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056286#comment-17056286 ] Gengliang Wang commented on SPARK-31098: [~dongjoon]Thank you so much for lookin

[jira] [Updated] (SPARK-30510) Publicly document options under spark.sql.*

[ https://issues.apache.org/jira/browse/SPARK-30510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30510: Labels: release-notes (was: ) > Publicly document options under spark.sql.* > ---

[jira] [Commented] (SPARK-31037) refine AQE config names

[ https://issues.apache.org/jira/browse/SPARK-31037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056227#comment-17056227 ] Dongjoon Hyun commented on SPARK-31037: --- This is resolved via https://github.com/a

[jira] [Resolved] (SPARK-31037) refine AQE config names

[ https://issues.apache.org/jira/browse/SPARK-31037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31037. --- Fix Version/s: 3.0.0 Assignee: Wenchen Fan Resolution: Fixed > refine AQE co

[jira] [Created] (SPARK-31111) Fix interval output issue in ExtractBenchmark

Kent Yao created SPARK-3: Summary: Fix interval output issue in ExtractBenchmark Key: SPARK-3 URL: https://issues.apache.org/jira/browse/SPARK-3 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-31110) refine sql doc for SELECT

Wenchen Fan created SPARK-31110: --- Summary: refine sql doc for SELECT Key: SPARK-31110 URL: https://issues.apache.org/jira/browse/SPARK-31110 Project: Spark Issue Type: Improvement Com

[jira] [Resolved] (SPARK-30189) Interval from year-month/date-time string handling whitespaces

[ https://issues.apache.org/jira/browse/SPARK-30189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30189. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26815 [https://gith

[jira] [Assigned] (SPARK-30189) Interval from year-month/date-time string handling whitespaces

[ https://issues.apache.org/jira/browse/SPARK-30189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-30189: --- Assignee: Kent Yao > Interval from year-month/date-time string handling whitespaces > -

[jira] [Resolved] (SPARK-31078) outputOrdering should handle aliases correctly

[ https://issues.apache.org/jira/browse/SPARK-31078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31078. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 27842 [https://gith

[jira] [Assigned] (SPARK-31078) outputOrdering should handle aliases correctly

[ https://issues.apache.org/jira/browse/SPARK-31078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31078: --- Assignee: Terry Kim > outputOrdering should handle aliases correctly >

[jira] [Assigned] (SPARK-31079) Add RuleExecutor metrics in Explain Formatted

[ https://issues.apache.org/jira/browse/SPARK-31079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31079: --- Assignee: Xin Wu > Add RuleExecutor metrics in Explain Formatted >

[jira] [Resolved] (SPARK-31079) Add RuleExecutor metrics in Explain Formatted

[ https://issues.apache.org/jira/browse/SPARK-31079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31079. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27846 [https://gith

[jira] [Created] (SPARK-31109) Add version information to the configuration of Mesos

jiaan.geng created SPARK-31109: -- Summary: Add version information to the configuration of Mesos Key: SPARK-31109 URL: https://issues.apache.org/jira/browse/SPARK-31109 Project: Spark Issue Type:

[jira] [Created] (SPARK-31108) Parameter cannot be passed to pandas udf of type map_iter

xge created SPARK-31108: --- Summary: Parameter cannot be passed to pandas udf of type map_iter Key: SPARK-31108 URL: https://issues.apache.org/jira/browse/SPARK-31108 Project: Spark Issue Type: Question

[jira] [Commented] (SPARK-31098) Reading ORC files throws IndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-31098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17055786#comment-17055786 ] Dongjoon Hyun commented on SPARK-31098: --- Hmm. [~Gengliang.Wang]. Unfortunately, th

[jira] [Created] (SPARK-31107) Extend FairScheduler to support pool level resource isolation

liupengcheng created SPARK-31107: Summary: Extend FairScheduler to support pool level resource isolation Key: SPARK-31107 URL: https://issues.apache.org/jira/browse/SPARK-31107 Project: Spark

[jira] [Commented] (SPARK-31106) Support IS_JSON

[ https://issues.apache.org/jira/browse/SPARK-31106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17055757#comment-17055757 ] Rakesh Raushan commented on SPARK-31106: I am working on it. > Support IS_JSON

[jira] [Created] (SPARK-31106) Support IS_JSON

Rakesh Raushan created SPARK-31106: -- Summary: Support IS_JSON Key: SPARK-31106 URL: https://issues.apache.org/jira/browse/SPARK-31106 Project: Spark Issue Type: New Feature Compone

[jira] [Created] (SPARK-31105) Respect sql execution id when scheduling taskSets

liupengcheng created SPARK-31105: Summary: Respect sql execution id when scheduling taskSets Key: SPARK-31105 URL: https://issues.apache.org/jira/browse/SPARK-31105 Project: Spark Issue Type:

  1   2   >