[jira] [Resolved] (SPARK-34188) Char varchar length check blocks CBO statistics

[ https://issues.apache.org/jira/browse/SPARK-34188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-34188. -- Resolution: Not A Problem applied another solution, not a problem anymore > Char varchar length

[jira] [Resolved] (SPARK-34730) how use the idea and scala to develop the spark,(configuration environment)

[ https://issues.apache.org/jira/browse/SPARK-34730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-34730. - Resolution: Invalid Please see IDE Setup:

[jira] [Updated] (SPARK-34730) how use the idea and scala to develop the spark,(configuration environment)

[ https://issues.apache.org/jira/browse/SPARK-34730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-34730: Fix Version/s: (was: 3.0.1) > how use the idea and scala to develop the spark,(configuration

[jira] [Updated] (SPARK-34730) how use the idea and scala to develop the spark,(configuration environment)

[ https://issues.apache.org/jira/browse/SPARK-34730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-34730: Target Version/s: (was: 3.0.1) > how use the idea and scala to develop the spark,(configuration

[jira] [Comment Edited] (SPARK-7276) withColumn is very slow on dataframe with large number of columns

[ https://issues.apache.org/jira/browse/SPARK-7276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17300724#comment-17300724 ] Abdeali Kothari edited comment on SPARK-7276 at 3/13/21, 5:09 AM: -- I can

[jira] [Commented] (SPARK-7276) withColumn is very slow on dataframe with large number of columns

[ https://issues.apache.org/jira/browse/SPARK-7276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17300724#comment-17300724 ] Abdeali Kothari commented on SPARK-7276:  confirm that even with 2.4 there is a significant

[jira] [Comment Edited] (SPARK-7276) withColumn is very slow on dataframe with large number of columns

[ https://issues.apache.org/jira/browse/SPARK-7276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17300724#comment-17300724 ] Abdeali Kothari edited comment on SPARK-7276 at 3/13/21, 5:03 AM: -- I can

[jira] [Updated] (SPARK-7276) withColumn is very slow on dataframe with large number of columns

[ https://issues.apache.org/jira/browse/SPARK-7276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abdeali Kothari updated SPARK-7276: --- Attachment: test.py > withColumn is very slow on dataframe with large number of columns >

[jira] [Created] (SPARK-34731) ConcurrentModificationException in EventLoggingListener when redacting properties

Bruce Robbins created SPARK-34731: - Summary: ConcurrentModificationException in EventLoggingListener when redacting properties Key: SPARK-34731 URL: https://issues.apache.org/jira/browse/SPARK-34731

[jira] [Commented] (SPARK-34729) Faster execution for broadcast nested loop join (left semi/anti with no condition)

[ https://issues.apache.org/jira/browse/SPARK-34729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17300676#comment-17300676 ] Apache Spark commented on SPARK-34729: -- User 'c21' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34729) Faster execution for broadcast nested loop join (left semi/anti with no condition)

[ https://issues.apache.org/jira/browse/SPARK-34729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34729: Assignee: (was: Apache Spark) > Faster execution for broadcast nested loop join

[jira] [Commented] (SPARK-34729) Faster execution for broadcast nested loop join (left semi/anti with no condition)

[ https://issues.apache.org/jira/browse/SPARK-34729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17300675#comment-17300675 ] Apache Spark commented on SPARK-34729: -- User 'c21' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34729) Faster execution for broadcast nested loop join (left semi/anti with no condition)

[ https://issues.apache.org/jira/browse/SPARK-34729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34729: Assignee: Apache Spark > Faster execution for broadcast nested loop join (left semi/anti

[jira] [Created] (SPARK-34730) how use the idea and scala to develop the spark,(configuration environment)

funny liu created SPARK-34730: - Summary: how use the idea and scala to develop the spark,(configuration environment) Key: SPARK-34730 URL: https://issues.apache.org/jira/browse/SPARK-34730 Project: Spark

[jira] [Created] (SPARK-34729) Faster execution for broadcast nested loop join (left semi/anti with no condition)

Cheng Su created SPARK-34729: Summary: Faster execution for broadcast nested loop join (left semi/anti with no condition) Key: SPARK-34729 URL: https://issues.apache.org/jira/browse/SPARK-34729 Project:

[jira] [Commented] (SPARK-19256) Hive bucketing write support

[ https://issues.apache.org/jira/browse/SPARK-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17300611#comment-17300611 ] Cheng Su commented on SPARK-19256: -- [~zsxwing] - if users gets table path, and writes data directly to

[jira] [Commented] (SPARK-19256) Hive bucketing write support

[ https://issues.apache.org/jira/browse/SPARK-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17300594#comment-17300594 ] Shixiong Zhu commented on SPARK-19256: -- [~chengsu] Thanks for the answers. Sounds like Metastore is

[jira] [Commented] (SPARK-19256) Hive bucketing write support

[ https://issues.apache.org/jira/browse/SPARK-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17300591#comment-17300591 ] Cheng Su commented on SPARK-19256: -- [~zsxwing] - sorry the design doc was years old, and I can create a

[jira] [Commented] (SPARK-19256) Hive bucketing write support

[ https://issues.apache.org/jira/browse/SPARK-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17300578#comment-17300578 ] Shixiong Zhu commented on SPARK-19256: -- I read 

[jira] [Resolved] (SPARK-34279) Upgrade Junit from 4.x to 5.x

[ https://issues.apache.org/jira/browse/SPARK-34279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34279. -- Resolution: Won't Fix > Upgrade Junit from 4.x to 5.x > - > >

[jira] [Resolved] (SPARK-34403) Remove dependency to commons-httpclient, is not used and has vulnerabilities.

[ https://issues.apache.org/jira/browse/SPARK-34403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34403. -- Resolution: Won't Fix > Remove dependency to commons-httpclient, is not used and has

[jira] [Commented] (SPARK-34511) Current Security vulnerabilities in spark libraries

[ https://issues.apache.org/jira/browse/SPARK-34511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17300575#comment-17300575 ] Sean R. Owen commented on SPARK-34511: -- What on this list probably affects Spark? we generally

[jira] [Commented] (SPARK-34517) breeze.linalg.SparseVector.equals always return true may bring error.

[ https://issues.apache.org/jira/browse/SPARK-34517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17300573#comment-17300573 ] Sean R. Owen commented on SPARK-34517: -- That's wild. I think this is a Breeze issue really. Are you

[jira] [Assigned] (SPARK-33482) V2 Datasources that extend FileScan preclude exchange reuse

[ https://issues.apache.org/jira/browse/SPARK-33482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33482: Assignee: (was: Apache Spark) > V2 Datasources that extend FileScan preclude

[jira] [Commented] (SPARK-33482) V2 Datasources that extend FileScan preclude exchange reuse

[ https://issues.apache.org/jira/browse/SPARK-33482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17300548#comment-17300548 ] Apache Spark commented on SPARK-33482: -- User 'peter-toth' has created a pull request for this

[jira] [Assigned] (SPARK-33482) V2 Datasources that extend FileScan preclude exchange reuse

[ https://issues.apache.org/jira/browse/SPARK-33482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33482: Assignee: Apache Spark > V2 Datasources that extend FileScan preclude exchange reuse >

[jira] [Commented] (SPARK-34728) Remove all SQLConf.get if extends from Rule

[ https://issues.apache.org/jira/browse/SPARK-34728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17300533#comment-17300533 ] Yuming Wang commented on SPARK-34728: - I'm working on. > Remove all SQLConf.get if extends from

[jira] [Created] (SPARK-34728) Remove all SQLConf.get if extends from Rule

Yuming Wang created SPARK-34728: --- Summary: Remove all SQLConf.get if extends from Rule Key: SPARK-34728 URL: https://issues.apache.org/jira/browse/SPARK-34728 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-34522) Issue Tracker for JDK related Bugs

[ https://issues.apache.org/jira/browse/SPARK-34522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34522. -- Resolution: Done Yes, but I see no reason to have an open-ended umbrella. Report issues as

[jira] [Resolved] (SPARK-34564) DateTimeUtils.fromJavaDate fails for very late dates during casting to Int

[ https://issues.apache.org/jira/browse/SPARK-34564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34564. -- Resolution: Not A Problem > DateTimeUtils.fromJavaDate fails for very late dates during

[jira] [Resolved] (SPARK-34570) Remove dead code from constructors of [Hive]SessionStateBuilder

[ https://issues.apache.org/jira/browse/SPARK-34570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34570. -- Fix Version/s: 3.2.0 Resolution: Fixed Resolved by

[jira] [Resolved] (SPARK-34587) Replaces map and getOrElse(false/true) with exists/forall

[ https://issues.apache.org/jira/browse/SPARK-34587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34587. -- Resolution: Won't Fix > Replaces map and getOrElse(false/true) with exists/forall >

[jira] [Resolved] (SPARK-34617) CVEs in the library dependencies

[ https://issues.apache.org/jira/browse/SPARK-34617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34617. -- Resolution: Not A Problem You're welcome to propose dependency updates in a PR if they're

[jira] [Resolved] (SPARK-34618) CVEs in library dependencies

[ https://issues.apache.org/jira/browse/SPARK-34618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34618. -- Resolution: Duplicate > CVEs in library dependencies > > >

[jira] [Resolved] (SPARK-34641) ARM CI failed due to download hive-exec failed

[ https://issues.apache.org/jira/browse/SPARK-34641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34641. -- Resolution: Not A Problem I think this is a transient CI env issue. > ARM CI failed due to

[jira] [Commented] (SPARK-34680) Spark hangs when out of diskspace

[ https://issues.apache.org/jira/browse/SPARK-34680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17300501#comment-17300501 ] Sean R. Owen commented on SPARK-34680: -- Why do you think it's related to disk space, and, isn't

[jira] [Resolved] (SPARK-34717) run-tests fails on AArch64, Fedora 33

[ https://issues.apache.org/jira/browse/SPARK-34717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34717. -- Resolution: Not A Problem This isn't related to Spark's software. You have extraneous files

[jira] [Resolved] (SPARK-34444) Pushdown scalar-subquery filter to FileSourceScan

[ https://issues.apache.org/jira/browse/SPARK-3?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-3. - Resolution: Won't Fix > Pushdown scalar-subquery filter to FileSourceScan >

[jira] [Resolved] (SPARK-34721) Add an year-month interval to a date

[ https://issues.apache.org/jira/browse/SPARK-34721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34721. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31812

[jira] [Assigned] (SPARK-34721) Add an year-month interval to a date

[ https://issues.apache.org/jira/browse/SPARK-34721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-34721: --- Assignee: Maxim Gekk > Add an year-month interval to a date >

[jira] [Resolved] (SPARK-34429) KMeansSummary class is omitted from PySpark documentation

[ https://issues.apache.org/jira/browse/SPARK-34429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34429. -- Fix Version/s: 3.1.1 Assignee: Sean R. Owen Resolution: Fixed Resolved in

[jira] [Commented] (SPARK-34727) Difference in results of casting float to timestamp

[ https://issues.apache.org/jira/browse/SPARK-34727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17300309#comment-17300309 ] Apache Spark commented on SPARK-34727: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34727) Difference in results of casting float to timestamp

[ https://issues.apache.org/jira/browse/SPARK-34727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34727: Assignee: (was: Apache Spark) > Difference in results of casting float to timestamp

[jira] [Assigned] (SPARK-34727) Difference in results of casting float to timestamp

[ https://issues.apache.org/jira/browse/SPARK-34727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34727: Assignee: Apache Spark > Difference in results of casting float to timestamp >

[jira] [Commented] (SPARK-34727) Difference in results of casting float to timestamp

[ https://issues.apache.org/jira/browse/SPARK-34727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17300307#comment-17300307 ] Apache Spark commented on SPARK-34727: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Resolved] (SPARK-34724) Fix Interpreted evaluation by using getClass.getMethod instead of getDeclaredMethod

[ https://issues.apache.org/jira/browse/SPARK-34724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34724. -- Fix Version/s: 3.0.3 3.1.2 3.2.0

[jira] [Commented] (SPARK-34588) Support int64 buffer lengths in Java for pyspark Pandas UDF as buffer expanding

[ https://issues.apache.org/jira/browse/SPARK-34588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17300264#comment-17300264 ] Hyukjin Kwon commented on SPARK-34588: -- If it's a different issue from this, let's interact in

[jira] [Created] (SPARK-34727) Difference in results of casting float to timestamp

Maxim Gekk created SPARK-34727: -- Summary: Difference in results of casting float to timestamp Key: SPARK-34727 URL: https://issues.apache.org/jira/browse/SPARK-34727 Project: Spark Issue Type:

[jira] [Commented] (SPARK-34727) Difference in results of casting float to timestamp

[ https://issues.apache.org/jira/browse/SPARK-34727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17300263#comment-17300263 ] Maxim Gekk commented on SPARK-34727: I am working on a fix. > Difference in results of casting

[jira] [Commented] (SPARK-34726) Fix collectToPython timeouts

[ https://issues.apache.org/jira/browse/SPARK-34726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17300206#comment-17300206 ] Apache Spark commented on SPARK-34726: -- User 'peter-toth' has created a pull request for this

[jira] [Commented] (SPARK-34726) Fix collectToPython timeouts

[ https://issues.apache.org/jira/browse/SPARK-34726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17300203#comment-17300203 ] Apache Spark commented on SPARK-34726: -- User 'peter-toth' has created a pull request for this

[jira] [Assigned] (SPARK-34726) Fix collectToPython timeouts

[ https://issues.apache.org/jira/browse/SPARK-34726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34726: Assignee: (was: Apache Spark) > Fix collectToPython timeouts >

[jira] [Assigned] (SPARK-34726) Fix collectToPython timeouts

[ https://issues.apache.org/jira/browse/SPARK-34726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34726: Assignee: Apache Spark > Fix collectToPython timeouts > > >

[jira] [Updated] (SPARK-34726) Fix collectToPython timeouts

[ https://issues.apache.org/jira/browse/SPARK-34726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-34726: --- Affects Version/s: (was: 3.1.1) 2.4.7 > Fix collectToPython timeouts >

[jira] [Created] (SPARK-34726) Fix collectToPython timeouts

Peter Toth created SPARK-34726: -- Summary: Fix collectToPython timeouts Key: SPARK-34726 URL: https://issues.apache.org/jira/browse/SPARK-34726 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-34588) Support int64 buffer lengths in Java for pyspark Pandas UDF as buffer expanding

[ https://issues.apache.org/jira/browse/SPARK-34588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17300178#comment-17300178 ] Dmitry Kravchuk commented on SPARK-34588: - [~hyukjin.kwon] hey there. I have some issues with

[jira] [Assigned] (SPARK-34725) Do not show Statistics(sizeInBytes=8.0 EiB) if we don't have valid stats

[ https://issues.apache.org/jira/browse/SPARK-34725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34725: Assignee: Apache Spark > Do not show Statistics(sizeInBytes=8.0 EiB) if we don't have

[jira] [Assigned] (SPARK-34725) Do not show Statistics(sizeInBytes=8.0 EiB) if we don't have valid stats

[ https://issues.apache.org/jira/browse/SPARK-34725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34725: Assignee: (was: Apache Spark) > Do not show Statistics(sizeInBytes=8.0 EiB) if we

[jira] [Commented] (SPARK-34725) Do not show Statistics(sizeInBytes=8.0 EiB) if we don't have valid stats

[ https://issues.apache.org/jira/browse/SPARK-34725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17300168#comment-17300168 ] Apache Spark commented on SPARK-34725: -- User 'ulysses-you' has created a pull request for this

[jira] [Created] (SPARK-34725) Do not show Statistics(sizeInBytes=8.0 EiB) if we don't have valid stats

ulysses you created SPARK-34725: --- Summary: Do not show Statistics(sizeInBytes=8.0 EiB) if we don't have valid stats Key: SPARK-34725 URL: https://issues.apache.org/jira/browse/SPARK-34725 Project: