[jira] [Commented] (SPARK-28004) Update jquery to 3.4.1

[ https://issues.apache.org/jira/browse/SPARK-28004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166162#comment-17166162 ] Łukasz Żukowski commented on SPARK-28004: - Hi It it possible to backport this to 2.4.x ? This

[jira] [Commented] (SPARK-32361) Remove project if output is subset of child

[ https://issues.apache.org/jira/browse/SPARK-32361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166159#comment-17166159 ] ulysses you commented on SPARK-32361: - [~hyukjin.kwon] thanks for remind, added some description. >

[jira] [Updated] (SPARK-32361) Remove project if output is subset of child

[ https://issues.apache.org/jira/browse/SPARK-32361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-32361: Description: We can remove some redundant project after we completed pruning column. e.g.,

[jira] [Updated] (SPARK-32361) Remove project if output is subset of child

[ https://issues.apache.org/jira/browse/SPARK-32361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-32361: Description: We can remove some redundant project after we completed column pruning. e.g.,

[jira] [Assigned] (SPARK-32290) NotInSubquery SingleColumn Optimize

[ https://issues.apache.org/jira/browse/SPARK-32290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32290: --- Assignee: Leanken.Lin > NotInSubquery SingleColumn Optimize >

[jira] [Resolved] (SPARK-32290) NotInSubquery SingleColumn Optimize

[ https://issues.apache.org/jira/browse/SPARK-32290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32290. - Fix Version/s: (was: 3.0.1) Resolution: Fixed Issue resolved by pull request 29104

[jira] [Commented] (SPARK-32208) SparkSQL throw Illegal character exception when load certain abnormal path of HDFS

[ https://issues.apache.org/jira/browse/SPARK-32208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166147#comment-17166147 ] Hyukjin Kwon commented on SPARK-32208: -- Can you fill the PR description please? > SparkSQL throw

[jira] [Resolved] (SPARK-32213) saveAsTable deletes all files in path

[ https://issues.apache.org/jira/browse/SPARK-32213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32213. -- Resolution: Not A Problem > saveAsTable deletes all files in path >

[jira] [Commented] (SPARK-32213) saveAsTable deletes all files in path

[ https://issues.apache.org/jira/browse/SPARK-32213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166145#comment-17166145 ] Hyukjin Kwon commented on SPARK-32213: -- I think this is documented: {code} * In this method,

[jira] [Resolved] (SPARK-32261) PySpark regexp_replace not replacing JSON çlçlçl

[ https://issues.apache.org/jira/browse/SPARK-32261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32261. -- Resolution: Invalid > PySpark regexp_replace not replacing JSON çlçlçl >

[jira] [Resolved] (SPARK-32263) PySpark regexp_replace not replacing JSON çlçlçlçl

[ https://issues.apache.org/jira/browse/SPARK-32263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32263. -- Resolution: Invalid > PySpark regexp_replace not replacing JSON çlçlçlçl >

[jira] [Resolved] (SPARK-32260) PySpark regexp_replace not replacing JSON çlçl

[ https://issues.apache.org/jira/browse/SPARK-32260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32260. -- Resolution: Invalid > PySpark regexp_replace not replacing JSON çlçl >

[jira] [Resolved] (SPARK-32262) PySpark regexp_replace not replacing JSON çlçlçl

[ https://issues.apache.org/jira/browse/SPARK-32262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32262. -- Resolution: Invalid > PySpark regexp_replace not replacing JSON çlçlçl >

[jira] [Commented] (SPARK-32269) Failed to rename delta file on checkpoint path

[ https://issues.apache.org/jira/browse/SPARK-32269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166135#comment-17166135 ] Hyukjin Kwon commented on SPARK-32269: -- [~ElvisQaQ] can you check the image? Seems that's broken.

[jira] [Commented] (SPARK-32275) "None.org.apache.spark.api.java.JavaSparkContext" Issue With Spark-Mllib Algorithm and JDBC Connectors

[ https://issues.apache.org/jira/browse/SPARK-32275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166132#comment-17166132 ] Hyukjin Kwon commented on SPARK-32275: -- Looks like it tries to access to JVM instances within UDFs

[jira] [Issue Comment Deleted] (SPARK-32323) Javascript/HTML bug in spark application UI

[ https://issues.apache.org/jira/browse/SPARK-32323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32323: - Comment: was deleted (was: [~ibobak] can you double check if the image is uploaded or not?) >

[jira] [Commented] (SPARK-32323) Javascript/HTML bug in spark application UI

[ https://issues.apache.org/jira/browse/SPARK-32323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166130#comment-17166130 ] Hyukjin Kwon commented on SPARK-32323: -- [~ibobak] can you double check if the image is uploaded or

[jira] [Commented] (SPARK-32341) add mutiple filter in rdd function

[ https://issues.apache.org/jira/browse/SPARK-32341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166129#comment-17166129 ] Hyukjin Kwon commented on SPARK-32341: -- You can do it via DataFrame and SQL APIs. I think such

[jira] [Resolved] (SPARK-32341) add mutiple filter in rdd function

[ https://issues.apache.org/jira/browse/SPARK-32341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32341. -- Resolution: Won't Do > add mutiple filter in rdd function >

[jira] [Resolved] (SPARK-32327) Introduce UnresolvedTableOrPermanentView for commands that support a table/view but not a temporary view

[ https://issues.apache.org/jira/browse/SPARK-32327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Terry Kim resolved SPARK-32327. --- Resolution: Won't Fix > Introduce UnresolvedTableOrPermanentView for commands that support a >

[jira] [Commented] (SPARK-32268) Bloom Filter Join

[ https://issues.apache.org/jira/browse/SPARK-32268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166127#comment-17166127 ] Yuming Wang commented on SPARK-32268: - A simple benchmark for bloom filter. {code:scala} import

[jira] [Resolved] (SPARK-31525) Inconsistent result of df.head(1) and df.head()

[ https://issues.apache.org/jira/browse/SPARK-31525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31525. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29214

[jira] [Commented] (SPARK-32312) Upgrade Apache Arrow to 1.0.0

[ https://issues.apache.org/jira/browse/SPARK-32312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166117#comment-17166117 ] Hyukjin Kwon commented on SPARK-32312: -- Nice! > Upgrade Apache Arrow to 1.0.0 >

[jira] [Commented] (SPARK-32385) Publish a "bill of materials" (BOM) descriptor for Spark with correct versions of various dependencies

[ https://issues.apache.org/jira/browse/SPARK-32385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166115#comment-17166115 ] Hyukjin Kwon commented on SPARK-32385: -- Do you mean something like this?

[jira] [Commented] (SPARK-32423) class 'DataFrame' returns instance of type(self) instead of DataFrame

[ https://issues.apache.org/jira/browse/SPARK-32423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166114#comment-17166114 ] Hyukjin Kwon commented on SPARK-32423: -- Can you show some pseudo codes? It's a bit difficult to

[jira] [Resolved] (SPARK-32460) how spark collects non-match results after performing broadcast left outer join

[ https://issues.apache.org/jira/browse/SPARK-32460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32460. -- Resolution: Invalid > how spark collects non-match results after performing broadcast left

[jira] [Commented] (SPARK-32433) Spark Web UI shows Nan undefined in Shuffle Read Size / Records

[ https://issues.apache.org/jira/browse/SPARK-32433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166109#comment-17166109 ] Hyukjin Kwon commented on SPARK-32433: -- Seems the attached image is broken. Can you check

[jira] [Commented] (SPARK-32460) how spark collects non-match results after performing broadcast left outer join

[ https://issues.apache.org/jira/browse/SPARK-32460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166110#comment-17166110 ] Hyukjin Kwon commented on SPARK-32460: -- Let's ask questions into mailing list or stackoverflow

[jira] [Commented] (SPARK-32400) Test coverage of HiveScripTransformationExec

[ https://issues.apache.org/jira/browse/SPARK-32400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166108#comment-17166108 ] Hyukjin Kwon commented on SPARK-32400: -- Please fill JIRA description. > Test coverage of

[jira] [Commented] (SPARK-32388) TRANSFORM when schema less should keep same with hive

[ https://issues.apache.org/jira/browse/SPARK-32388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166106#comment-17166106 ] Hyukjin Kwon commented on SPARK-32388: -- Please fill JIRA description. > TRANSFORM when schema less

[jira] [Commented] (SPARK-32390) TRANSFORM with hive serde support CalendarIntervalType and UserDefinedType

[ https://issues.apache.org/jira/browse/SPARK-32390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166107#comment-17166107 ] Hyukjin Kwon commented on SPARK-32390: -- Please fill JIRA description. > TRANSFORM with hive serde

[jira] [Commented] (SPARK-32355) 使用Structured Streaming窗口统计不能实现topN

[ https://issues.apache.org/jira/browse/SPARK-32355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166104#comment-17166104 ] Hyukjin Kwon commented on SPARK-32355: -- Can you write it in English which most of dev people use to

[jira] [Resolved] (SPARK-32369) pyspark foreach/foreachPartition send http request failed

[ https://issues.apache.org/jira/browse/SPARK-32369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32369. -- Resolution: Invalid > pyspark foreach/foreachPartition send http request failed >

[jira] [Commented] (SPARK-29544) Optimize skewed join at runtime with new Adaptive Execution

[ https://issues.apache.org/jira/browse/SPARK-29544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166101#comment-17166101 ] Apache Spark commented on SPARK-29544: -- User 'JkSelf' has created a pull request for this issue:

[jira] [Commented] (SPARK-29544) Optimize skewed join at runtime with new Adaptive Execution

[ https://issues.apache.org/jira/browse/SPARK-29544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166102#comment-17166102 ] Apache Spark commented on SPARK-29544: -- User 'JkSelf' has created a pull request for this issue:

[jira] [Resolved] (SPARK-32370) pyspark foreach/foreachPartition send http request failed

[ https://issues.apache.org/jira/browse/SPARK-32370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32370. -- Resolution: Duplicate > pyspark foreach/foreachPartition send http request failed >

[jira] [Commented] (SPARK-32369) pyspark foreach/foreachPartition send http request failed

[ https://issues.apache.org/jira/browse/SPARK-32369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166096#comment-17166096 ] Hyukjin Kwon commented on SPARK-32369: -- Seems you're running it on Mac. You should probably set

[jira] [Updated] (SPARK-32369) pyspark foreach/foreachPartition send http request failed

[ https://issues.apache.org/jira/browse/SPARK-32369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32369: - Description: I use urllib.request to send http request in foreach/foreachPartition. pyspark

[jira] [Commented] (SPARK-32361) Remove project if output is subset of child

[ https://issues.apache.org/jira/browse/SPARK-32361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166094#comment-17166094 ] Hyukjin Kwon commented on SPARK-32361: -- Please fill JIRA description. > Remove project if output

[jira] [Commented] (SPARK-32359) Implement max_error metric evaluator for spark regression mllib

[ https://issues.apache.org/jira/browse/SPARK-32359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166095#comment-17166095 ] Hyukjin Kwon commented on SPARK-32359: -- Please fill JIRA description. > Implement max_error metric

[jira] [Commented] (SPARK-32424) Fix silent data change for timestamp parsing if overflow happens

[ https://issues.apache.org/jira/browse/SPARK-32424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166085#comment-17166085 ] Apache Spark commented on SPARK-32424: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Resolved] (SPARK-32439) Override datasource implementation during look up via configuration

[ https://issues.apache.org/jira/browse/SPARK-32439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32439. -- Resolution: Won't Fix > Override datasource implementation during look up via configuration >

[jira] [Assigned] (SPARK-32464) Support skew handling on join that has one side with no query stage

[ https://issues.apache.org/jira/browse/SPARK-32464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32464: Assignee: (was: Apache Spark) > Support skew handling on join that has one side with

[jira] [Assigned] (SPARK-32464) Support skew handling on join that has one side with no query stage

[ https://issues.apache.org/jira/browse/SPARK-32464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32464: Assignee: Apache Spark > Support skew handling on join that has one side with no query

[jira] [Commented] (SPARK-32464) Support skew handling on join that has one side with no query stage

[ https://issues.apache.org/jira/browse/SPARK-32464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166072#comment-17166072 ] Apache Spark commented on SPARK-32464: -- User 'wangshisan' has created a pull request for this

[jira] [Assigned] (SPARK-32464) Support skew handling on join that has one side with no query stage

[ https://issues.apache.org/jira/browse/SPARK-32464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32464: Assignee: Apache Spark > Support skew handling on join that has one side with no query

[jira] [Commented] (SPARK-32464) Support skew handling on join that has one side with no query stage

[ https://issues.apache.org/jira/browse/SPARK-32464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166071#comment-17166071 ] Wang, Gang commented on SPARK-32464: A PR [https://github.com/apache/spark/pull/29266] > Support

[jira] [Updated] (SPARK-32464) Support skew handling on join that has one side with no query stage

[ https://issues.apache.org/jira/browse/SPARK-32464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wang, Gang updated SPARK-32464: --- Summary: Support skew handling on join that has one side with no query stage (was: Support skew

[jira] [Created] (SPARK-32464) Support skew handling on join with one side that has no query stage

Wang, Gang created SPARK-32464: -- Summary: Support skew handling on join with one side that has no query stage Key: SPARK-32464 URL: https://issues.apache.org/jira/browse/SPARK-32464 Project: Spark

[jira] [Commented] (SPARK-32463) Document Data Type inference rule in SQL reference

[ https://issues.apache.org/jira/browse/SPARK-32463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166045#comment-17166045 ] Huaxin Gao commented on SPARK-32463: [~planga82] You are welcomed to work on this if you have free

[jira] [Created] (SPARK-32463) Document Data Type inference rule in SQL reference

Huaxin Gao created SPARK-32463: -- Summary: Document Data Type inference rule in SQL reference Key: SPARK-32463 URL: https://issues.apache.org/jira/browse/SPARK-32463 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-31753) Add missing keywords in the SQL documents

[ https://issues.apache.org/jira/browse/SPARK-31753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-31753. -- Fix Version/s: 3.0.1 Assignee: philipse Resolution: Fixed Resolved by 

[jira] [Updated] (SPARK-31753) Add missing keywords in the SQL documents

[ https://issues.apache.org/jira/browse/SPARK-31753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-31753: - Affects Version/s: (was: 3.0.0) 3.0.1 > Add missing keywords

[jira] [Updated] (SPARK-31753) Add missing keywords in the SQL documents

[ https://issues.apache.org/jira/browse/SPARK-31753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-31753: - Affects Version/s: 3.0.0 > Add missing keywords in the SQL documents >

[jira] [Comment Edited] (SPARK-28210) Shuffle Storage API: Reads

[ https://issues.apache.org/jira/browse/SPARK-28210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166027#comment-17166027 ] Tianchen Zhang edited comment on SPARK-28210 at 7/27/20, 11:40 PM: --- Hi

[jira] [Commented] (SPARK-28210) Shuffle Storage API: Reads

[ https://issues.apache.org/jira/browse/SPARK-28210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166027#comment-17166027 ] Tianchen Zhang commented on SPARK-28210: Hi [~devaraj], do you mind share some ideas about your

[jira] [Assigned] (SPARK-32462) Don't save the previous search text for datatable

[ https://issues.apache.org/jira/browse/SPARK-32462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32462: Assignee: Kousuke Saruta (was: Apache Spark) > Don't save the previous search text for

[jira] [Commented] (SPARK-32462) Don't save the previous search text for datatable

[ https://issues.apache.org/jira/browse/SPARK-32462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166025#comment-17166025 ] Apache Spark commented on SPARK-32462: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32462) Don't save the previous search text for datatable

[ https://issues.apache.org/jira/browse/SPARK-32462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32462: Assignee: Apache Spark (was: Kousuke Saruta) > Don't save the previous search text for

[jira] [Updated] (SPARK-32462) Don't save the previous search text for datatable

[ https://issues.apache.org/jira/browse/SPARK-32462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-32462: --- Description: DataTable is used in stage-page and executors-page for pagination and filter

[jira] [Created] (SPARK-32462) Don't save the previous search text for datatable

Kousuke Saruta created SPARK-32462: -- Summary: Don't save the previous search text for datatable Key: SPARK-32462 URL: https://issues.apache.org/jira/browse/SPARK-32462 Project: Spark Issue

[jira] [Commented] (SPARK-32417) Flaky test: BlockManagerDecommissionIntegrationSuite.verify that an already running task which is going to cache data succeeds on a decommissioned executor

[ https://issues.apache.org/jira/browse/SPARK-32417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17165979#comment-17165979 ] Apache Spark commented on SPARK-32417: -- User 'holdenk' has created a pull request for this issue:

[jira] [Commented] (SPARK-32461) Shuffled hash join improvement

[ https://issues.apache.org/jira/browse/SPARK-32461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17165951#comment-17165951 ] Cheng Su commented on SPARK-32461: -- Just FYI - I am working on each sub-tasks separately now. >

[jira] [Updated] (SPARK-21505) A dynamic join operator to improve the join reliability

[ https://issues.apache.org/jira/browse/SPARK-21505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-21505: - Parent: SPARK-32461 Issue Type: Sub-task (was: New Feature) > A dynamic join operator to

[jira] [Updated] (SPARK-32399) Support full outer join in shuffled hash join and broadcast hash join

[ https://issues.apache.org/jira/browse/SPARK-32399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-32399: - Parent: SPARK-32461 Issue Type: Sub-task (was: Improvement) > Support full outer join in

[jira] [Updated] (SPARK-32421) Add code-gen for shuffled hash join

[ https://issues.apache.org/jira/browse/SPARK-32421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-32421: - Parent: SPARK-32461 Issue Type: Sub-task (was: Improvement) > Add code-gen for shuffled hash

[jira] [Updated] (SPARK-32383) Preserve hash join (BHJ and SHJ) stream side ordering

[ https://issues.apache.org/jira/browse/SPARK-32383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-32383: - Parent: SPARK-32461 Issue Type: Sub-task (was: Improvement) > Preserve hash join (BHJ and SHJ)

[jira] [Updated] (SPARK-32420) Add handling for unique key in non-codegen hash join

[ https://issues.apache.org/jira/browse/SPARK-32420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-32420: - Parent: SPARK-32461 Issue Type: Sub-task (was: Improvement) > Add handling for unique key in

[jira] [Updated] (SPARK-32330) Preserve shuffled hash join build side partitioning

[ https://issues.apache.org/jira/browse/SPARK-32330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-32330: - Parent: SPARK-32461 Issue Type: Sub-task (was: Improvement) > Preserve shuffled hash join

[jira] [Updated] (SPARK-32286) Coalesce bucketed tables for shuffled hash join if applicable

[ https://issues.apache.org/jira/browse/SPARK-32286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-32286: - Parent: SPARK-32461 Issue Type: Sub-task (was: Improvement) > Coalesce bucketed tables for

[jira] [Created] (SPARK-32461) Shuffled hash join improvement

Cheng Su created SPARK-32461: Summary: Shuffled hash join improvement Key: SPARK-32461 URL: https://issues.apache.org/jira/browse/SPARK-32461 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-32429) Standalone Mode allow setting CUDA_VISIBLE_DEVICES on executor launch

[ https://issues.apache.org/jira/browse/SPARK-32429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17165934#comment-17165934 ] Thomas Graves commented on SPARK-32429: --- So this doesn't address the task side, it addresses the

[jira] [Assigned] (SPARK-32457) logParam thresholds in DT/GBT/FM/LR/MLP

[ https://issues.apache.org/jira/browse/SPARK-32457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao reassigned SPARK-32457: -- Assignee: zhengruifeng > logParam thresholds in DT/GBT/FM/LR/MLP >

[jira] [Resolved] (SPARK-32457) logParam thresholds in DT/GBT/FM/LR/MLP

[ https://issues.apache.org/jira/browse/SPARK-32457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao resolved SPARK-32457. Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29257

[jira] [Assigned] (SPARK-32443) Fix testCommandAvailable to use POSIX compatible `command -v`

[ https://issues.apache.org/jira/browse/SPARK-32443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32443: - Assignee: Hyukjin Kwon (was: Dongjoon Hyun) > Fix testCommandAvailable to use POSIX

[jira] [Resolved] (SPARK-32443) Fix testCommandAvailable to use POSIX compatible `command -v`

[ https://issues.apache.org/jira/browse/SPARK-32443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32443. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29241

[jira] [Created] (SPARK-32460) how spark collects non-match results after performing broadcast left outer join

farshad delavarpour created SPARK-32460: --- Summary: how spark collects non-match results after performing broadcast left outer join Key: SPARK-32460 URL: https://issues.apache.org/jira/browse/SPARK-32460

[jira] [Commented] (SPARK-32429) Standalone Mode allow setting CUDA_VISIBLE_DEVICES on executor launch

[ https://issues.apache.org/jira/browse/SPARK-32429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17165903#comment-17165903 ] Xiangrui Meng commented on SPARK-32429: --- Couple questions: 1. Which GPU resource name do we use?

[jira] [Updated] (SPARK-31993) Generated code in 'concat_ws' fails to compile when splitting method is in effect

[ https://issues.apache.org/jira/browse/SPARK-31993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-31993: Component/s: (was: Spark Core) SQL > Generated code in 'concat_ws' fails to compile

[jira] [Commented] (SPARK-32332) AQE doesn't adequately allow for Columnar Processing extension

[ https://issues.apache.org/jira/browse/SPARK-32332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17165898#comment-17165898 ] Apache Spark commented on SPARK-32332: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-32332) AQE doesn't adequately allow for Columnar Processing extension

[ https://issues.apache.org/jira/browse/SPARK-32332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17165897#comment-17165897 ] Apache Spark commented on SPARK-32332: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32424) Fix silent data change for timestamp parsing if overflow happens

[ https://issues.apache.org/jira/browse/SPARK-32424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32424: --- Assignee: Kent Yao > Fix silent data change for timestamp parsing if overflow happens >

[jira] [Resolved] (SPARK-32424) Fix silent data change for timestamp parsing if overflow happens

[ https://issues.apache.org/jira/browse/SPARK-32424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32424. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29220

[jira] [Resolved] (SPARK-32420) Add handling for unique key in non-codegen hash join

[ https://issues.apache.org/jira/browse/SPARK-32420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32420. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29216

[jira] [Assigned] (SPARK-32420) Add handling for unique key in non-codegen hash join

[ https://issues.apache.org/jira/browse/SPARK-32420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32420: --- Assignee: Cheng Su > Add handling for unique key in non-codegen hash join >

[jira] [Issue Comment Deleted] (SPARK-32431) The .schema() API behaves incorrectly for nested schemas that have column duplicates in case-insensitive mode

[ https://issues.apache.org/jira/browse/SPARK-32431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-32431: --- Comment: was deleted (was: I cannot reproduce the issue on master, branch-3.0 and branch-2.4. I

[jira] [Updated] (SPARK-32431) The .schema() API behaves incorrectly for nested schemas that have column duplicates in case-insensitive mode

[ https://issues.apache.org/jira/browse/SPARK-32431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-32431: --- Description: The code below throws org.apache.spark.sql.AnalysisException: Found duplicate

[jira] [Commented] (SPARK-32425) Spark sequence() fails if start and end of range are identical timestamps

[ https://issues.apache.org/jira/browse/SPARK-32425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17165810#comment-17165810 ] L. C. Hsieh commented on SPARK-32425: - Thanks [~JinxinTang]. > Spark sequence() fails if start and

[jira] [Resolved] (SPARK-32425) Spark sequence() fails if start and end of range are identical timestamps

[ https://issues.apache.org/jira/browse/SPARK-32425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-32425. - Resolution: Duplicate > Spark sequence() fails if start and end of range are identical

[jira] [Commented] (SPARK-32459) UDF regression of WrappedArray supporting caused by SPARK-31826

[ https://issues.apache.org/jira/browse/SPARK-32459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17165698#comment-17165698 ] Apache Spark commented on SPARK-32459: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32459) UDF regression of WrappedArray supporting caused by SPARK-31826

[ https://issues.apache.org/jira/browse/SPARK-32459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32459: Assignee: Apache Spark > UDF regression of WrappedArray supporting caused by SPARK-31826

[jira] [Assigned] (SPARK-32459) UDF regression of WrappedArray supporting caused by SPARK-31826

[ https://issues.apache.org/jira/browse/SPARK-32459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32459: Assignee: (was: Apache Spark) > UDF regression of WrappedArray supporting caused by

[jira] [Commented] (SPARK-32459) UDF regression of WrappedArray supporting caused by SPARK-31826

[ https://issues.apache.org/jira/browse/SPARK-32459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17165696#comment-17165696 ] Apache Spark commented on SPARK-32459: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Assigned] (SPARK-30794) Stage Level scheduling: Add ability to set off heap memory

[ https://issues.apache.org/jira/browse/SPARK-30794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-30794: - Assignee: Zhongwei Zhu > Stage Level scheduling: Add ability to set off heap memory >

[jira] [Resolved] (SPARK-30794) Stage Level scheduling: Add ability to set off heap memory

[ https://issues.apache.org/jira/browse/SPARK-30794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-30794. --- Fix Version/s: 3.1.0 Resolution: Fixed > Stage Level scheduling: Add ability to set

[jira] [Created] (SPARK-32459) UDF regression of WrappedArray supporting caused by SPARK-31826

wuyi created SPARK-32459: Summary: UDF regression of WrappedArray supporting caused by SPARK-31826 Key: SPARK-32459 URL: https://issues.apache.org/jira/browse/SPARK-32459 Project: Spark Issue Type:

[jira] [Commented] (SPARK-20680) Spark-sql do not support for void column datatype of view

[ https://issues.apache.org/jira/browse/SPARK-20680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17165669#comment-17165669 ] Apache Spark commented on SPARK-20680: -- User 'ulysses-you' has created a pull request for this

[jira] [Commented] (SPARK-19169) columns changed orc table encouter 'IndexOutOfBoundsException' when read the old schema files

[ https://issues.apache.org/jira/browse/SPARK-19169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17165657#comment-17165657 ] bianqi commented on SPARK-19169: [~hyukjin.kwon] hello We also encountered this problem in the

[jira] [Commented] (SPARK-29302) dynamic partition overwrite with speculation enabled

[ https://issues.apache.org/jira/browse/SPARK-29302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17165653#comment-17165653 ] Apache Spark commented on SPARK-29302: -- User 'WinkerDu' has created a pull request for this issue:

[jira] [Commented] (SPARK-29302) dynamic partition overwrite with speculation enabled

[ https://issues.apache.org/jira/browse/SPARK-29302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17165651#comment-17165651 ] Apache Spark commented on SPARK-29302: -- User 'WinkerDu' has created a pull request for this issue:

  1   2   >