[jira] [Created] (SPARK-30762) Add dtype="float32" support to vector_to_array UDF

2020-02-08 Thread Liang Zhang (Jira)
Liang Zhang created SPARK-30762: --- Summary: Add dtype="float32" support to vector_to_array UDF Key: SPARK-30762 URL: https://issues.apache.org/jira/browse/SPARK-30762 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-30761) Nested pruning should not prune on required child outputs in Generate

2020-02-08 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-30761. - Resolution: Won't Fix > Nested pruning should not prune on required child outputs in Generate >

[jira] [Created] (SPARK-30761) Nested pruning should not prune on required child outputs in Generate

2020-02-08 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-30761: --- Summary: Nested pruning should not prune on required child outputs in Generate Key: SPARK-30761 URL: https://issues.apache.org/jira/browse/SPARK-30761 Project: Spark

[jira] [Commented] (SPARK-30711) 64KB JVM bytecode limit - janino.InternalCompilerException

2020-02-08 Thread Kazuaki Ishizaki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17033112#comment-17033112 ] Kazuaki Ishizaki commented on SPARK-30711: -- [~schreiber] Sorry, I made a mistak

[jira] [Updated] (SPARK-30274) Avoid BytesToBytesMap lookup hang forever when holding keys reaching max capacity

2020-02-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30274: -- Affects Version/s: 2.0.2 2.1.3 2.2.3

[jira] [Commented] (SPARK-24884) Implement regexp_extract_all

2020-02-08 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17033097#comment-17033097 ] jiaan.geng commented on SPARK-24884: I'm working on. > Implement regexp_extract_all

[jira] [Updated] (SPARK-30274) Avoid BytesToBytesMap lookup hang forever when holding keys reaching max capacity

2020-02-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30274: -- Affects Version/s: 2.4.4 > Avoid BytesToBytesMap lookup hang forever when holding keys reachin

[jira] [Updated] (SPARK-29918) RecordBinaryComparator should check endianness when compared by long

2020-02-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29918: -- Affects Version/s: 2.4.4 > RecordBinaryComparator should check endianness when compared by lon

[jira] [Commented] (SPARK-29042) Sampling-based RDD with unordered input should be INDETERMINATE

2020-02-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17033096#comment-17033096 ] Dongjoon Hyun commented on SPARK-29042: --- Hi, [~viirya]. Could you update the `Affe

[jira] [Updated] (SPARK-29042) Sampling-based RDD with unordered input should be INDETERMINATE

2020-02-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29042: -- Affects Version/s: 2.4.4 > Sampling-based RDD with unordered input should be INDETERMINATE > -

[jira] [Updated] (SPARK-30274) Avoid BytesToBytesMap lookup hang forever when holding keys reaching max capacity

2020-02-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30274: -- Labels: release-notes (was: ) > Avoid BytesToBytesMap lookup hang forever when holding keys r

[jira] [Updated] (SPARK-30312) Preserve path permission when truncate table

2020-02-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30312: -- Labels: release-notes (was: ) > Preserve path permission when truncate table > --

[jira] [Updated] (SPARK-29890) Unable to fill na with 0 with duplicate columns

2020-02-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29890: -- Labels: release-notes (was: ) > Unable to fill na with 0 with duplicate columns > ---

[jira] [Updated] (SPARK-30065) Unable to drop na with duplicate columns

2020-02-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30065: -- Labels: release-notes (was: ) > Unable to drop na with duplicate columns > --

[jira] [Updated] (SPARK-28939) SQL configuration are not always propagated

2020-02-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28939: -- Labels: release-notes (was: ) > SQL configuration are not always propagated > ---

[jira] [Updated] (SPARK-28152) Mapped ShortType to SMALLINT and FloatType to REAL for MsSqlServerDialect

2020-02-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28152: -- Labels: release-notes (was: ) > Mapped ShortType to SMALLINT and FloatType to REAL for MsSqlS

[jira] [Updated] (SPARK-27812) kubernetes client import non-daemon thread which block jvm exit.

2020-02-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27812: -- Labels: release-notes (was: ) > kubernetes client import non-daemon thread which block jvm ex

[jira] [Updated] (SPARK-21492) Memory leak in SortMergeJoin

2020-02-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-21492: -- Labels: release-notes (was: ) > Memory leak in SortMergeJoin > >

[jira] [Updated] (SPARK-30755) Support Hive 1.2.1's Serde after making built-in Hive to 2.3

2020-02-08 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30755: Priority: Blocker (was: Major) > Support Hive 1.2.1's Serde after making built-in Hive to 2.3 > -

[jira] [Updated] (SPARK-30755) Support Hive 1.2.1's Serde after making built-in Hive to 2.3

2020-02-08 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30755: Target Version/s: 3.0.0 Description: {noformat} 2020-01-27 05:11:20.446 - stderr> 20/01/27 05:11:

[jira] [Created] (SPARK-30760) Port `millisToDays` and `daysToMillis` on Java 8 time API

2020-02-08 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-30760: -- Summary: Port `millisToDays` and `daysToMillis` on Java 8 time API Key: SPARK-30760 URL: https://issues.apache.org/jira/browse/SPARK-30760 Project: Spark Issue T

[jira] [Updated] (SPARK-30759) The cache in StringRegexExpression is not initialized for foldable patterns

2020-02-08 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-30759: --- Priority: Minor (was: Major) > The cache in StringRegexExpression is not initialized for foldable p

[jira] [Created] (SPARK-30759) The cache in StringRegexExpression is not initialized for foldable patterns

2020-02-08 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-30759: -- Summary: The cache in StringRegexExpression is not initialized for foldable patterns Key: SPARK-30759 URL: https://issues.apache.org/jira/browse/SPARK-30759 Project: Spar

[jira] [Updated] (SPARK-30759) The cache in StringRegexExpression is not initialized for foldable patterns

2020-02-08 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-30759: --- Attachment: Screen Shot 2020-02-08 at 22.45.50.png > The cache in StringRegexExpression is not initi

[jira] [Commented] (SPARK-29292) Fix internal usages of mutable collection for Seq in 2.13

2020-02-08 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17032923#comment-17032923 ] Sean R. Owen commented on SPARK-29292: -- This is a pretty good example of most of th

[jira] [Commented] (SPARK-30740) months_between wrong calculation

2020-02-08 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17032922#comment-17032922 ] Maxim Gekk commented on SPARK-30740: This is because of the special *if*  [https://g

[jira] [Commented] (SPARK-30758) Spark SQL can't display bracketed comments well in generated golden files

2020-02-08 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17032895#comment-17032895 ] jiaan.geng commented on SPARK-30758: I'm working on. > Spark SQL can't display brac

[jira] [Updated] (SPARK-30758) Spark SQL can't display bracketed comments well in generated golden files

2020-02-08 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-30758: --- Summary: Spark SQL can't display bracketed comments well in generated golden files (was: Spark SQL

[jira] [Created] (SPARK-30758) Spark SQL can't treat bracketed comments well and lead to generated golden files can't display bracketed comments well.

2020-02-08 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-30758: -- Summary: Spark SQL can't treat bracketed comments well and lead to generated golden files can't display bracketed comments well. Key: SPARK-30758 URL: https://issues.apache.org/jira/b

[jira] [Commented] (SPARK-30740) months_between wrong calculation

2020-02-08 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17032894#comment-17032894 ] Yuming Wang commented on SPARK-30740: - cc [~maxgekk] > months_between wrong calcula

[jira] [Updated] (SPARK-28880) ANSI SQL: Nested bracketed comments

2020-02-08 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-28880: --- Description: Spark SQL support these bracketed comments: *Case 1*: {code:sql} /* This is an example

[jira] [Updated] (SPARK-28880) ANSI SQL: Nested bracketed comments

2020-02-08 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-28880: --- Summary: ANSI SQL: Nested bracketed comments (was: ANSI SQL: Bracketed comments) > ANSI SQL: Neste

[jira] [Issue Comment Deleted] (SPARK-30724) Support 'like any' and 'like all' operators

2020-02-08 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-30724: --- Comment: was deleted (was: I will investigate this feature.) > Support 'like any' and 'like all' op