[jira] [Updated] (SPARK-33981) SparkUI: Storage page is empty even if cached

[ https://issues.apache.org/jira/browse/SPARK-33981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maple updated SPARK-33981: -- Description: scala> import org.apache.spark.storage.StorageLevel import org.apache.spark.storage.StorageLevel

[jira] [Updated] (SPARK-33981) SparkUI: Storage page is empty even if cached

[ https://issues.apache.org/jira/browse/SPARK-33981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maple updated SPARK-33981: -- Attachment: ba5a987152c6270f34b968bd89ca36a.png > SparkUI: Storage page is empty even if cached >

[jira] [Created] (SPARK-33981) SparkUI: Storage page is empty even if cached

maple created SPARK-33981: - Summary: SparkUI: Storage page is empty even if cached Key: SPARK-33981 URL: https://issues.apache.org/jira/browse/SPARK-33981 Project: Spark Issue Type: Question

[jira] [Commented] (SPARK-33980) invalidate char/varchar in spark.readStream.schema

[ https://issues.apache.org/jira/browse/SPARK-33980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17258040#comment-17258040 ] Apache Spark commented on SPARK-33980: -- User 'yaooqinn' has created a pull request

[jira] [Assigned] (SPARK-33980) invalidate char/varchar in spark.readStream.schema

[ https://issues.apache.org/jira/browse/SPARK-33980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33980: Assignee: Apache Spark > invalidate char/varchar in spark.readStream.schema > ---

[jira] [Assigned] (SPARK-33980) invalidate char/varchar in spark.readStream.schema

[ https://issues.apache.org/jira/browse/SPARK-33980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33980: Assignee: (was: Apache Spark) > invalidate char/varchar in spark.readStream.schema >

[jira] [Commented] (SPARK-33980) invalidate char/varchar in spark.readStream.schema

[ https://issues.apache.org/jira/browse/SPARK-33980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17258039#comment-17258039 ] Apache Spark commented on SPARK-33980: -- User 'yaooqinn' has created a pull request

[jira] [Created] (SPARK-33980) invalidate char/varchar in spark.readStream.schema

Kent Yao created SPARK-33980: Summary: invalidate char/varchar in spark.readStream.schema Key: SPARK-33980 URL: https://issues.apache.org/jira/browse/SPARK-33980 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-33832) Add an option in AQE to mitigate skew even if it causes an new shuffle

[ https://issues.apache.org/jira/browse/SPARK-33832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33832: -- Parent: SPARK-33828 Issue Type: Sub-task (was: Improvement) > Add an option in AQE to

[jira] [Resolved] (SPARK-33888) JDBC SQL TIME type represents incorrectly as TimestampType, it should be physical Int in millis

[ https://issues.apache.org/jira/browse/SPARK-33888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33888. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30902 [https://gith

[jira] [Resolved] (SPARK-33934) Support user-defined script command wrapper for more use case

[ https://issues.apache.org/jira/browse/SPARK-33934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33934. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30973 [https://gi

[jira] [Updated] (SPARK-33978) Support ZSTD compression in ORC data source

[ https://issues.apache.org/jira/browse/SPARK-33978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33978: -- Description: h3. What changes were proposed in this pull request? This PR aims to support ZST

[jira] [Assigned] (SPARK-33934) Support user-defined script command wrapper for more use case

[ https://issues.apache.org/jira/browse/SPARK-33934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33934: Assignee: angerszhu > Support user-defined script command wrapper for more use case > ---

[jira] [Commented] (SPARK-33638) Full support of V2 table creation in Structured Streaming writer path

[ https://issues.apache.org/jira/browse/SPARK-33638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17258012#comment-17258012 ] Jungtaek Lim commented on SPARK-33638: -- Let me change the version a bit. I wouldn't

[jira] [Updated] (SPARK-33638) Full support of V2 table creation in Structured Streaming writer path

[ https://issues.apache.org/jira/browse/SPARK-33638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-33638: - Affects Version/s: (was: 3.1.0) 3.2.0 > Full support of V2 table crea

[jira] [Created] (SPARK-33979) Filter predicate reorder

Yuming Wang created SPARK-33979: --- Summary: Filter predicate reorder Key: SPARK-33979 URL: https://issues.apache.org/jira/browse/SPARK-33979 Project: Spark Issue Type: New Feature Comp

[jira] [Commented] (SPARK-33979) Filter predicate reorder

[ https://issues.apache.org/jira/browse/SPARK-33979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17258011#comment-17258011 ] Yuming Wang commented on SPARK-33979: - I'm working on. > Filter predicate reorder >

[jira] [Assigned] (SPARK-33978) Support ZSTD compression in ORC data source

[ https://issues.apache.org/jira/browse/SPARK-33978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33978: Assignee: (was: Apache Spark) > Support ZSTD compression in ORC data source > ---

[jira] [Commented] (SPARK-33978) Support ZSTD compression in ORC data source

[ https://issues.apache.org/jira/browse/SPARK-33978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17258007#comment-17258007 ] Apache Spark commented on SPARK-33978: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-33978) Support ZSTD compression in ORC data source

[ https://issues.apache.org/jira/browse/SPARK-33978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33978: Assignee: Apache Spark > Support ZSTD compression in ORC data source > --

[jira] [Created] (SPARK-33978) Support ZSTD in ORC data source

Dongjoon Hyun created SPARK-33978: - Summary: Support ZSTD in ORC data source Key: SPARK-33978 URL: https://issues.apache.org/jira/browse/SPARK-33978 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-33978) Support ZSTD compression in ORC data source

[ https://issues.apache.org/jira/browse/SPARK-33978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33978: -- Summary: Support ZSTD compression in ORC data source (was: Support ZSTD in ORC data source)

[jira] [Updated] (SPARK-33896) Make Spark DAGScheduler datasource cache aware when scheduling tasks in a multi-replication HDFS

[ https://issues.apache.org/jira/browse/SPARK-33896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33896: - Priority: Major (was: Critical) > Make Spark DAGScheduler datasource cache aware when schedulin

[jira] [Commented] (SPARK-33977) Add doc for "'like any' and 'like all' operators"

[ https://issues.apache.org/jira/browse/SPARK-33977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257989#comment-17257989 ] Yuming Wang commented on SPARK-33977: - Example:  https://github.com/apache/spark/blo

[jira] [Commented] (SPARK-33977) Add doc for "'like any' and 'like all' operators"

[ https://issues.apache.org/jira/browse/SPARK-33977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257988#comment-17257988 ] jiaan.geng commented on SPARK-33977: I'm working on. > Add doc for "'like any' and

[jira] [Commented] (SPARK-33638) Full support of V2 table creation in Structured Streaming writer path

[ https://issues.apache.org/jira/browse/SPARK-33638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257987#comment-17257987 ] Hyukjin Kwon commented on SPARK-33638: -- [~XuanYuan] and [~kabhwan] which version do

[jira] [Updated] (SPARK-31851) Redesign PySpark documentation

[ https://issues.apache.org/jira/browse/SPARK-31851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31851: - Fix Version/s: 3.1.0 > Redesign PySpark documentation > -- > >

[jira] [Updated] (SPARK-32246) Have a way to optionally run streaming-kinesis-asl

[ https://issues.apache.org/jira/browse/SPARK-32246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32246: - Parent: (was: SPARK-32244) Issue Type: Bug (was: Sub-task) > Have a way to optional

[jira] [Resolved] (SPARK-32244) Build and run the Spark with test cases in Github Actions

[ https://issues.apache.org/jira/browse/SPARK-32244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32244. -- Fix Version/s: 3.1.0 Resolution: Done > Build and run the Spark with test cases in Gith

[jira] [Updated] (SPARK-31236) Spark error while consuming data from Kinesis direct end point

[ https://issues.apache.org/jira/browse/SPARK-31236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31236: - Priority: Major (was: Critical) > Spark error while consuming data from Kinesis direct end poin

[jira] [Resolved] (SPARK-28001) Dataframe throws 'socket.timeout: timed out' exception

[ https://issues.apache.org/jira/browse/SPARK-28001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28001. -- Resolution: Cannot Reproduce > Dataframe throws 'socket.timeout: timed out' exception > --

[jira] [Updated] (SPARK-28001) Dataframe throws 'socket.timeout: timed out' exception

[ https://issues.apache.org/jira/browse/SPARK-28001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-28001: - Priority: Major (was: Critical) > Dataframe throws 'socket.timeout: timed out' exception >

[jira] [Updated] (SPARK-26836) Columns get switched in Spark SQL using Avro backed Hive table if schema evolves

[ https://issues.apache.org/jira/browse/SPARK-26836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26836: - Target Version/s: (was: 3.0.0) > Columns get switched in Spark SQL using Avro backed Hive tabl

[jira] [Updated] (SPARK-24338) Spark SQL fails to create a Hive table when running in a Apache Sentry-secured Environment

[ https://issues.apache.org/jira/browse/SPARK-24338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24338: - Priority: Major (was: Critical) > Spark SQL fails to create a Hive table when running in a Apac

[jira] [Resolved] (SPARK-24338) Spark SQL fails to create a Hive table when running in a Apache Sentry-secured Environment

[ https://issues.apache.org/jira/browse/SPARK-24338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24338. -- Resolution: Incomplete Spark 2.3 is EOL. I am resolving this for now. Feel free to reopen if t

[jira] [Resolved] (SPARK-24098) ScriptTransformationExec should wait process exiting before output iterator finish

[ https://issues.apache.org/jira/browse/SPARK-24098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24098. -- Resolution: Incomplete > ScriptTransformationExec should wait process exiting before output it

[jira] [Resolved] (SPARK-33954) Some operator missing rowCount when enable CBO

[ https://issues.apache.org/jira/browse/SPARK-33954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33954. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30987 [https://gith

[jira] [Updated] (SPARK-24098) ScriptTransformationExec should wait process exiting before output iterator finish

[ https://issues.apache.org/jira/browse/SPARK-24098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24098: - Priority: Major (was: Critical) > ScriptTransformationExec should wait process exiting before o

[jira] [Assigned] (SPARK-33954) Some operator missing rowCount when enable CBO

[ https://issues.apache.org/jira/browse/SPARK-33954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33954: --- Assignee: Yuming Wang > Some operator missing rowCount when enable CBO > --

[jira] [Resolved] (SPARK-9182) filter and groupBy on DataFrames are not passed through to jdbc source

[ https://issues.apache.org/jira/browse/SPARK-9182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-9182. - Assignee: (was: Yijie Shen) Resolution: Duplicate I am going to resolve this as a dupli

[jira] [Commented] (SPARK-33005) Kubernetes GA Preparation

[ https://issues.apache.org/jira/browse/SPARK-33005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257979#comment-17257979 ] Hyukjin Kwon commented on SPARK-33005: -- [~dongjoon] would you mind taking an action

[jira] [Resolved] (SPARK-31851) Redesign PySpark documentation

[ https://issues.apache.org/jira/browse/SPARK-31851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31851. -- Resolution: Done > Redesign PySpark documentation > -- > >

[jira] [Commented] (SPARK-32185) User Guide - Monitoring

[ https://issues.apache.org/jira/browse/SPARK-32185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257977#comment-17257977 ] Hyukjin Kwon commented on SPARK-32185: -- Spark 3.1 RC will be cut out soon. I am goi

[jira] [Resolved] (SPARK-33951) Distinguish the error between filter and distinct

[ https://issues.apache.org/jira/browse/SPARK-33951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33951. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30982 [https://gith

[jira] [Assigned] (SPARK-33951) Distinguish the error between filter and distinct

[ https://issues.apache.org/jira/browse/SPARK-33951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33951: --- Assignee: jiaan.geng > Distinguish the error between filter and distinct >

[jira] [Updated] (SPARK-32185) User Guide - Monitoring

[ https://issues.apache.org/jira/browse/SPARK-32185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32185: - Parent: (was: SPARK-31851) Issue Type: Improvement (was: Sub-task) > User Guide - M

[jira] [Updated] (SPARK-32391) Install pydata_sphinx_theme in Jenkins machines

[ https://issues.apache.org/jira/browse/SPARK-32391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32391: - Issue Type: Test (was: Bug) > Install pydata_sphinx_theme in Jenkins machines > ---

[jira] [Commented] (SPARK-32391) Install pydata_sphinx_theme in Jenkins machines

[ https://issues.apache.org/jira/browse/SPARK-32391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257975#comment-17257975 ] Hyukjin Kwon commented on SPARK-32391: -- I will get this out of the parent JIRA beca

[jira] [Updated] (SPARK-32391) Install pydata_sphinx_theme in Jenkins machines

[ https://issues.apache.org/jira/browse/SPARK-32391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32391: - Parent: (was: SPARK-31851) Issue Type: Bug (was: Sub-task) > Install pydata_sphinx_

[jira] [Commented] (SPARK-33946) Cannot connect to spark hive after session timeout

[ https://issues.apache.org/jira/browse/SPARK-33946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257974#comment-17257974 ] Hyukjin Kwon commented on SPARK-33946: -- Spark 2.3 is EOL. Can you check if it works

[jira] [Commented] (SPARK-33952) Python-friendly dtypes for pyspark dataframes

[ https://issues.apache.org/jira/browse/SPARK-33952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257973#comment-17257973 ] Hyukjin Kwon commented on SPARK-33952: -- How/where the output string is used? > Pyt

[jira] [Resolved] (SPARK-33945) Handles a random seed consisting of an expr tree

[ https://issues.apache.org/jira/browse/SPARK-33945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33945. --- Fix Version/s: 3.1.0 Assignee: Takeshi Yamamuro Resolution: Fixed This is re

[jira] [Updated] (SPARK-33952) Python-friendly dtypes for pyspark dataframes

[ https://issues.apache.org/jira/browse/SPARK-33952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33952: - Fix Version/s: (was: 3.2.0) > Python-friendly dtypes for pyspark dataframes > --

[jira] [Resolved] (SPARK-33958) spark sql DoubleType(0 * (-1)) return "-0.0"

[ https://issues.apache.org/jira/browse/SPARK-33958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33958. -- Resolution: Not A Problem > spark sql DoubleType(0 * (-1)) return "-0.0" > --

[jira] [Commented] (SPARK-30210) Give more informative error for BinaryClassificationEvaluator when data with only one label is provided

[ https://issues.apache.org/jira/browse/SPARK-30210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257965#comment-17257965 ] Nicholas Brett Marcott commented on SPARK-30210: [~hyukjin.kwon] Thanks

[jira] [Commented] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257962#comment-17257962 ] Yukihito X commented on SPARK-33635: [~david.wyles], I tried your sample code in my

[jira] [Commented] (SPARK-33948) branch-3.1 jenkins test failed in Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-33948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257955#comment-17257955 ] Yang Jie commented on SPARK-33948: -- {quote} these tests passed before SPARK-33513, {quo

[jira] [Assigned] (SPARK-26399) Add new stage-level REST APIs and parameters

[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26399: Assignee: (was: Apache Spark) > Add new stage-level REST APIs and parameters > --

[jira] [Commented] (SPARK-26399) Add new stage-level REST APIs and parameters

[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257954#comment-17257954 ] Apache Spark commented on SPARK-26399: -- User 'AngersZh' has created a pull requ

[jira] [Assigned] (SPARK-26399) Add new stage-level REST APIs and parameters

[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26399: Assignee: Apache Spark > Add new stage-level REST APIs and parameters > -

[jira] [Comment Edited] (SPARK-33948) branch-3.1 jenkins test failed in Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-33948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257894#comment-17257894 ] Yang Jie edited comment on SPARK-33948 at 1/4/21, 4:12 AM: --- ju

[jira] [Resolved] (SPARK-33950) ALTER TABLE .. DROP PARTITION doesn't refresh cache

[ https://issues.apache.org/jira/browse/SPARK-33950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33950. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30983 [https://gith

[jira] [Assigned] (SPARK-33950) ALTER TABLE .. DROP PARTITION doesn't refresh cache

[ https://issues.apache.org/jira/browse/SPARK-33950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33950: --- Assignee: Maxim Gekk > ALTER TABLE .. DROP PARTITION doesn't refresh cache > --

[jira] [Resolved] (SPARK-30210) Give more informative error for BinaryClassificationEvaluator when data with only one label is provided

[ https://issues.apache.org/jira/browse/SPARK-30210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30210. -- Resolution: Cannot Reproduce > Give more informative error for BinaryClassificationEvaluator w

[jira] [Commented] (SPARK-33977) Add doc for "'like any' and 'like all' operators"

[ https://issues.apache.org/jira/browse/SPARK-33977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257939#comment-17257939 ] Xiao Li commented on SPARK-33977: - cc [~yumwang] > Add doc for "'like any' and 'like al

[jira] [Commented] (SPARK-33977) Add doc for "'like any' and 'like all' operators"

[ https://issues.apache.org/jira/browse/SPARK-33977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257938#comment-17257938 ] Xiao Li commented on SPARK-33977: - https://issues.apache.org/jira/browse/SPARK-30724 is

[jira] [Created] (SPARK-33977) Add doc for "'like any' and 'like all' operators"

Xiao Li created SPARK-33977: --- Summary: Add doc for "'like any' and 'like all' operators" Key: SPARK-33977 URL: https://issues.apache.org/jira/browse/SPARK-33977 Project: Spark Issue Type: Documenta

[jira] [Commented] (SPARK-28125) dataframes created by randomSplit have overlapping rows

[ https://issues.apache.org/jira/browse/SPARK-28125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257901#comment-17257901 ] Nicholas Brett Marcott commented on SPARK-28125: friendly ping: [~zachar

[jira] [Commented] (SPARK-30210) Give more informative error for BinaryClassificationEvaluator when data with only one label is provided

[ https://issues.apache.org/jira/browse/SPARK-30210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257898#comment-17257898 ] Nicholas Brett Marcott commented on SPARK-30210: + [~hyukjin.kwon], I ha

[jira] [Comment Edited] (SPARK-33948) branch-3.1 jenkins test failed in Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-33948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257894#comment-17257894 ] Yang Jie edited comment on SPARK-33948 at 1/4/21, 2:13 AM: --- ju

[jira] [Updated] (SPARK-33976) Add a dedicated SQL document page for the TRANSFORM-related functionality,

[ https://issues.apache.org/jira/browse/SPARK-33976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-33976: -- Parent: SPARK-31936 Issue Type: Sub-task (was: Improvement) > Add a dedicated SQL document pa

[jira] [Created] (SPARK-33976) Add a dedicated SQL document page for the TRANSFORM-related functionality,

angerszhu created SPARK-33976: - Summary: Add a dedicated SQL document page for the TRANSFORM-related functionality, Key: SPARK-33976 URL: https://issues.apache.org/jira/browse/SPARK-33976 Project: Spark

[jira] [Commented] (SPARK-33948) branch-3.1 jenkins test failed in Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-33948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257895#comment-17257895 ] Yang Jie commented on SPARK-33948: -- Wait for me to investigate why the master branch ca

[jira] [Commented] (SPARK-33948) branch-3.1 jenkins test failed in Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-33948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257894#comment-17257894 ] Yang Jie commented on SPARK-33948: -- just came back from my holiday, haha. It looks lik

[jira] [Commented] (SPARK-32048) PySpark: error in serializing ML pipelines with training strategy and pipeline as estimator

[ https://issues.apache.org/jira/browse/SPARK-32048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257888#comment-17257888 ] Nicholas Brett Marcott commented on SPARK-32048: friendly ping: [~weiche

[jira] [Assigned] (SPARK-33975) add_hours SQL function

[ https://issues.apache.org/jira/browse/SPARK-33975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33975: Assignee: Apache Spark > add_hours SQL function > -- > >

[jira] [Commented] (SPARK-33975) add_hours SQL function

[ https://issues.apache.org/jira/browse/SPARK-33975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257887#comment-17257887 ] Apache Spark commented on SPARK-33975: -- User 'MrPowers' has created a pull request

[jira] [Assigned] (SPARK-33975) add_hours SQL function

[ https://issues.apache.org/jira/browse/SPARK-33975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33975: Assignee: (was: Apache Spark) > add_hours SQL function > -- > >

[jira] [Created] (SPARK-33975) add_hours SQL function

Matthew Powers created SPARK-33975: -- Summary: add_hours SQL function Key: SPARK-33975 URL: https://issues.apache.org/jira/browse/SPARK-33975 Project: Spark Issue Type: New Feature

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257844#comment-17257844 ] Ron Hu edited comment on SPARK-26399 at 1/3/21, 9:00 PM: - [~ange

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: Add the peak values for the metrics to the stages REST API. Also add a new executorSummary RES

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257844#comment-17257844 ] Ron Hu edited comment on SPARK-26399 at 1/3/21, 7:46 PM: - [~ange

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257844#comment-17257844 ] Ron Hu edited comment on SPARK-26399 at 1/3/21, 7:45 PM: - [~ange

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257844#comment-17257844 ] Ron Hu edited comment on SPARK-26399 at 1/3/21, 7:44 PM: - [~ange

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Attachment: executorMetricsSummary.json > Add new stage-level REST APIs and parameters > ---

[jira] [Commented] (SPARK-26399) Add new stage-level REST APIs and parameters

[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257846#comment-17257846 ] Ron Hu commented on SPARK-26399: [^executorMetricsSummary.json] > Add new stage-level R

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Attachment: stage_executorSummary_image1.png > Add new stage-level REST APIs and parameters > --

[jira] [Commented] (SPARK-26399) Add new stage-level REST APIs and parameters

[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257845#comment-17257845 ] Ron Hu commented on SPARK-26399: !stage_executorSummary_image1.png! > Add new stage-lev

[jira] [Commented] (SPARK-26399) Add new stage-level REST APIs and parameters

[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257844#comment-17257844 ] Ron Hu commented on SPARK-26399: [~angerszhuuu] found that the "executorSummary" field a

[jira] [Resolved] (SPARK-33398) AnalysisException when loading a PipelineModel with Spark 3

[ https://issues.apache.org/jira/browse/SPARK-33398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-33398. -- Fix Version/s: 3.0.2 3.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-33398) AnalysisException when loading a PipelineModel with Spark 3

[ https://issues.apache.org/jira/browse/SPARK-33398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-33398: Assignee: zhengruifeng > AnalysisException when loading a PipelineModel with Spark 3 > --

[jira] [Created] (SPARK-33974) Hide values of sensitive partitioning columns

Gidon Gershinsky created SPARK-33974: Summary: Hide values of sensitive partitioning columns Key: SPARK-33974 URL: https://issues.apache.org/jira/browse/SPARK-33974 Project: Spark Issue T

[jira] [Created] (SPARK-33973) Prevent partitioning on sensitive columns

Gidon Gershinsky created SPARK-33973: Summary: Prevent partitioning on sensitive columns Key: SPARK-33973 URL: https://issues.apache.org/jira/browse/SPARK-33973 Project: Spark Issue Type:

[jira] [Created] (SPARK-33972) Partitioning on sensitive columns

Gidon Gershinsky created SPARK-33972: Summary: Partitioning on sensitive columns Key: SPARK-33972 URL: https://issues.apache.org/jira/browse/SPARK-33972 Project: Spark Issue Type: New Fea

[jira] [Commented] (SPARK-33971) Eliminate distinct from more aggregates

[ https://issues.apache.org/jira/browse/SPARK-33971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257793#comment-17257793 ] Apache Spark commented on SPARK-33971: -- User 'tanelk' has created a pull request fo

[jira] [Assigned] (SPARK-33971) Eliminate distinct from more aggregates

[ https://issues.apache.org/jira/browse/SPARK-33971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33971: Assignee: Apache Spark > Eliminate distinct from more aggregates > --

[jira] [Assigned] (SPARK-33971) Eliminate distinct from more aggregates

[ https://issues.apache.org/jira/browse/SPARK-33971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33971: Assignee: (was: Apache Spark) > Eliminate distinct from more aggregates > ---

[jira] [Commented] (SPARK-33971) Eliminate distinct from more aggregates

[ https://issues.apache.org/jira/browse/SPARK-33971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257792#comment-17257792 ] Apache Spark commented on SPARK-33971: -- User 'tanelk' has created a pull request fo

[jira] [Created] (SPARK-33971) Eliminate distinct from more aggregates

Tanel Kiis created SPARK-33971: -- Summary: Eliminate distinct from more aggregates Key: SPARK-33971 URL: https://issues.apache.org/jira/browse/SPARK-33971 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-33970) Check isNull and isNotNull in tests

Yuming Wang created SPARK-33970: --- Summary: Check isNull and isNotNull in tests Key: SPARK-33970 URL: https://issues.apache.org/jira/browse/SPARK-33970 Project: Spark Issue Type: Sub-task

  1   2   >