[jira] [Commented] (SPARK-30711) 64KB JVM bytecode limit - janino.InternalCompilerException

[ https://issues.apache.org/jira/browse/SPARK-30711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17033421#comment-17033421 ] Frederik Schreiber commented on SPARK-30711: Hey here is my setting:

[jira] [Created] (SPARK-30769) insertInto() with existing column as partition key cause weird partition result

Woong Seok Kang created SPARK-30769: --- Summary: insertInto() with existing column as partition key cause weird partition result Key: SPARK-30769 URL: https://issues.apache.org/jira/browse/SPARK-30769

[jira] [Updated] (SPARK-30768) Constraints should be inferred from inequality attributes

[ https://issues.apache.org/jira/browse/SPARK-30768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-30768: Description: How to reproduce: {code:sql} create table SPARK_30768_1(c1 int, c2 int); create

[jira] [Created] (SPARK-30768) Constraints should be inferred from inequality attributes

Yuming Wang created SPARK-30768: --- Summary: Constraints should be inferred from inequality attributes Key: SPARK-30768 URL: https://issues.apache.org/jira/browse/SPARK-30768 Project: Spark

[jira] [Commented] (SPARK-30762) Add dtype="float32" support to vector_to_array UDF

[ https://issues.apache.org/jira/browse/SPARK-30762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17033376#comment-17033376 ] Liang Zhang commented on SPARK-30762: - Hi Hyukjin, I updated the description. > Add dtype="float32"

[jira] [Updated] (SPARK-30762) Add dtype="float32" support to vector_to_array UDF

[ https://issues.apache.org/jira/browse/SPARK-30762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Zhang updated SPARK-30762: Description: Previous PR: 

[jira] [Commented] (SPARK-30619) org.slf4j.Logger and org.apache.commons.collections classes not built as part of hadoop-provided profile

[ https://issues.apache.org/jira/browse/SPARK-30619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17033374#comment-17033374 ] Abhishek Rao commented on SPARK-30619: -- Hi [~hyukjin.kwon] I just built container using 

[jira] [Assigned] (SPARK-30762) Add dtype="float32" support to vector_to_array UDF

[ https://issues.apache.org/jira/browse/SPARK-30762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-30762: - Assignee: Liang Zhang > Add dtype="float32" support to vector_to_array UDF >

[jira] [Updated] (SPARK-30762) Add dtype="float32" support to vector_to_array UDF

[ https://issues.apache.org/jira/browse/SPARK-30762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-30762: -- Component/s: PySpark > Add dtype="float32" support to vector_to_array UDF >

[jira] [Reopened] (SPARK-29721) Spark SQL reads unnecessary nested fields after using explode

[ https://issues.apache.org/jira/browse/SPARK-29721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reopened SPARK-29721: - > Spark SQL reads unnecessary nested fields after using explode >

[jira] [Resolved] (SPARK-30614) The native ALTER COLUMN syntax should change one thing at a time

[ https://issues.apache.org/jira/browse/SPARK-30614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30614. - Fix Version/s: 3.0.0 Assignee: Terry Kim Resolution: Fixed > The native ALTER

[jira] [Resolved] (SPARK-30732) BroadcastExchangeExec does not fully honor "spark.broadcast.compress"

[ https://issues.apache.org/jira/browse/SPARK-30732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30732. -- Resolution: Won't Fix > BroadcastExchangeExec does not fully honor "spark.broadcast.compress"

[jira] [Commented] (SPARK-30741) The data returned from SAS using JDBC reader contains column label

[ https://issues.apache.org/jira/browse/SPARK-30741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17033350#comment-17033350 ] Hyukjin Kwon commented on SPARK-30741: -- Spark 2.1.x is EOL. Can you try it in a higher version?

[jira] [Updated] (SPARK-30740) months_between wrong calculation

[ https://issues.apache.org/jira/browse/SPARK-30740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30740: - Priority: Major (was: Critical) > months_between wrong calculation >

[jira] [Resolved] (SPARK-30741) The data returned from SAS using JDBC reader contains column label

[ https://issues.apache.org/jira/browse/SPARK-30741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30741. -- Resolution: Incomplete > The data returned from SAS using JDBC reader contains column label >

[jira] [Resolved] (SPARK-30745) Spark streaming, kafka broker error, "Failed to get records for spark-executor- .... after polling for 512"

[ https://issues.apache.org/jira/browse/SPARK-30745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30745. -- Resolution: Incomplete > Spark streaming, kafka broker error, "Failed to get records for >

[jira] [Commented] (SPARK-30745) Spark streaming, kafka broker error, "Failed to get records for spark-executor- .... after polling for 512"

[ https://issues.apache.org/jira/browse/SPARK-30745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17033348#comment-17033348 ] Hyukjin Kwon commented on SPARK-30745: -- Spark 2.0.x is EOL. Can you try it in a higher version? >

[jira] [Commented] (SPARK-30762) Add dtype="float32" support to vector_to_array UDF

[ https://issues.apache.org/jira/browse/SPARK-30762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17033346#comment-17033346 ] Hyukjin Kwon commented on SPARK-30762: -- Sorry, can you clarify what you mean by dtype support? >

[jira] [Resolved] (SPARK-30767) from_json changes times of timestmaps by several minutes without error

[ https://issues.apache.org/jira/browse/SPARK-30767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30767. -- Resolution: Not A Problem > from_json changes times of timestmaps by several minutes without

[jira] [Updated] (SPARK-30767) from_json changes times of timestmaps by several minutes without error

[ https://issues.apache.org/jira/browse/SPARK-30767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30767: - Labels: (was: corruption) > from_json changes times of timestmaps by several minutes without

[jira] [Resolved] (SPARK-30711) 64KB JVM bytecode limit - janino.InternalCompilerException

[ https://issues.apache.org/jira/browse/SPARK-30711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30711. -- Resolution: Cannot Reproduce > 64KB JVM bytecode limit - janino.InternalCompilerException >

[jira] [Resolved] (SPARK-30687) When reading from a file with pre-defined schema and encountering a single value that is not the same type as that of its column , Spark nullifies the entire row

[ https://issues.apache.org/jira/browse/SPARK-30687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30687. -- Resolution: Cannot Reproduce > When reading from a file with pre-defined schema and

[jira] [Updated] (SPARK-30688) Spark SQL Unix Timestamp produces incorrect result with unix_timestamp UDF

[ https://issues.apache.org/jira/browse/SPARK-30688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30688: - Affects Version/s: 3.0.0 2.4.4 > Spark SQL Unix Timestamp produces

[jira] [Resolved] (SPARK-30619) org.slf4j.Logger and org.apache.commons.collections classes not built as part of hadoop-provided profile

[ https://issues.apache.org/jira/browse/SPARK-30619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30619. -- Resolution: Cannot Reproduce > org.slf4j.Logger and org.apache.commons.collections classes

[jira] [Commented] (SPARK-30619) org.slf4j.Logger and org.apache.commons.collections classes not built as part of hadoop-provided profile

[ https://issues.apache.org/jira/browse/SPARK-30619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17033345#comment-17033345 ] Hyukjin Kwon commented on SPARK-30619: -- [~abhisrao], can you show the exact step you did so I can

[jira] [Updated] (SPARK-28067) Incorrect results in decimal aggregation with whole-stage code gen enabled

[ https://issues.apache.org/jira/browse/SPARK-28067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-28067: - Priority: Critical (was: Blocker) > Incorrect results in decimal aggregation with whole-stage

[jira] [Commented] (SPARK-28067) Incorrect results in decimal aggregation with whole-stage code gen enabled

[ https://issues.apache.org/jira/browse/SPARK-28067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17033344#comment-17033344 ] Hyukjin Kwon commented on SPARK-28067: -- I am lowering the priority given that

[jira] [Commented] (SPARK-26449) Missing Dataframe.transform API in Python API

[ https://issues.apache.org/jira/browse/SPARK-26449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17033343#comment-17033343 ] Hyukjin Kwon commented on SPARK-26449: -- To match with Scala side. It should be easy to work around.

[jira] [Commented] (SPARK-30687) When reading from a file with pre-defined schema and encountering a single value that is not the same type as that of its column , Spark nullifies the entire row

[ https://issues.apache.org/jira/browse/SPARK-30687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17033298#comment-17033298 ] Maxim Gekk commented on SPARK-30687: This feature will come with Spark 3.0

[jira] [Comment Edited] (SPARK-30767) from_json changes times of timestmaps by several minutes without error

[ https://issues.apache.org/jira/browse/SPARK-30767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17033293#comment-17033293 ] Maxim Gekk edited comment on SPARK-30767 at 2/9/20 9:08 PM: The default

[jira] [Commented] (SPARK-30767) from_json changes times of timestmaps by several minutes without error

[ https://issues.apache.org/jira/browse/SPARK-30767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17033294#comment-17033294 ] Maxim Gekk commented on SPARK-30767: Also 2.4.4 supports only `SSS` for second fractions. That was

[jira] [Commented] (SPARK-30767) from_json changes times of timestmaps by several minutes without error

[ https://issues.apache.org/jira/browse/SPARK-30767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17033293#comment-17033293 ] Maxim Gekk commented on SPARK-30767: The default timestamp pattern in JSON datasource specifies only

[jira] [Created] (SPARK-30767) from_json changes times of timestmaps by several minutes without error

Benedikt Maria Beckermann created SPARK-30767: - Summary: from_json changes times of timestmaps by several minutes without error Key: SPARK-30767 URL: https://issues.apache.org/jira/browse/SPARK-30767

[jira] [Created] (SPARK-30766) Wrong truncation of old timestamps to hours and days

Maxim Gekk created SPARK-30766: -- Summary: Wrong truncation of old timestamps to hours and days Key: SPARK-30766 URL: https://issues.apache.org/jira/browse/SPARK-30766 Project: Spark Issue Type:

[jira] [Created] (SPARK-30765) Refine baes class abstraction code style

Xin Wu created SPARK-30765: -- Summary: Refine baes class abstraction code style Key: SPARK-30765 URL: https://issues.apache.org/jira/browse/SPARK-30765 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-30764) Improve the readability of EXPLAIN FORMATTED style

Xin Wu created SPARK-30764: -- Summary: Improve the readability of EXPLAIN FORMATTED style Key: SPARK-30764 URL: https://issues.apache.org/jira/browse/SPARK-30764 Project: Spark Issue Type:

[jira] [Updated] (SPARK-30763) Fix java.lang.IndexOutOfBoundsException No group 1 for regexp_extract

[ https://issues.apache.org/jira/browse/SPARK-30763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-30763: --- Description: The current implement of regexp_extract will throws a unprocessed exception show

[jira] [Created] (SPARK-30763) Fix java.lang.IndexOutOfBoundsException No group 1 for regexp_extract

jiaan.geng created SPARK-30763: -- Summary: Fix java.lang.IndexOutOfBoundsException No group 1 for regexp_extract Key: SPARK-30763 URL: https://issues.apache.org/jira/browse/SPARK-30763 Project: Spark

[jira] [Resolved] (SPARK-30510) Publicly document options under spark.sql.*

[ https://issues.apache.org/jira/browse/SPARK-30510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30510. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27459

[jira] [Assigned] (SPARK-30510) Publicly document options under spark.sql.*

[ https://issues.apache.org/jira/browse/SPARK-30510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-30510: Assignee: Hyukjin Kwon > Publicly document options under spark.sql.* >

[jira] [Commented] (SPARK-30762) Add dtype="float32" support to vector_to_array UDF

[ https://issues.apache.org/jira/browse/SPARK-30762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17033151#comment-17033151 ] Liang Zhang commented on SPARK-30762: - I'm now working on this issue. > Add dtype="float32" support