[jira] [Assigned] (SPARK-31869) BroadcastHashJoinExe's outputPartitioning can utilize the build side

[ https://issues.apache.org/jira/browse/SPARK-31869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31869: Assignee: Apache Spark > BroadcastHashJoinExe's outputPartitioning can utilize the build

[jira] [Commented] (SPARK-31869) BroadcastHashJoinExe's outputPartitioning can utilize the build side

[ https://issues.apache.org/jira/browse/SPARK-31869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17120107#comment-17120107 ] Apache Spark commented on SPARK-31869: -- User 'imback82' has created a pull request for this issue:

[jira] [Assigned] (SPARK-31869) BroadcastHashJoinExe's outputPartitioning can utilize the build side

[ https://issues.apache.org/jira/browse/SPARK-31869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31869: Assignee: (was: Apache Spark) > BroadcastHashJoinExe's outputPartitioning can

[jira] [Updated] (SPARK-31869) BroadcastHashJoinExe's outputPartitioning can utilize the build side

[ https://issues.apache.org/jira/browse/SPARK-31869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Terry Kim updated SPARK-31869: -- Description: Currently, the BroadcastHashJoinExec's outputPartitioning only uses the streamed side's

[jira] [Created] (SPARK-31869) BroadcastHashJoinExe's outputPartitioning can utilize the build side

Terry Kim created SPARK-31869: - Summary: BroadcastHashJoinExe's outputPartitioning can utilize the build side Key: SPARK-31869 URL: https://issues.apache.org/jira/browse/SPARK-31869 Project: Spark

[jira] [Comment Edited] (SPARK-31779) Redefining struct inside array incorrectly wraps child fields in array

[ https://issues.apache.org/jira/browse/SPARK-31779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119931#comment-17119931 ] Jeff Evans edited comment on SPARK-31779 at 5/29/20, 9:01 PM: -- Thanks,

[jira] [Resolved] (SPARK-31541) Backport SPARK-26095 Disable parallelization in make-distibution.sh. (Avoid build hanging)

[ https://issues.apache.org/jira/browse/SPARK-31541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau resolved SPARK-31541. -- Fix Version/s: 2.4.6 Assignee: Marcelo Masiero Vanzin Resolution: Fixed >

[jira] [Commented] (SPARK-31779) Redefining struct inside array incorrectly wraps child fields in array

[ https://issues.apache.org/jira/browse/SPARK-31779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119931#comment-17119931 ] Jeff Evans commented on SPARK-31779: Thanks, using {{arrays_zip}} (along with an extra {{cast}} to

[jira] [Updated] (SPARK-26095) make-distribution.sh is hanging in jenkins

[ https://issues.apache.org/jira/browse/SPARK-26095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-26095: - Fix Version/s: 2.4.6 Affects Version/s: 2.4.6 > make-distribution.sh is hanging in

[jira] [Reopened] (SPARK-31541) Backport SPARK-26095 Disable parallelization in make-distibution.sh. (Avoid build hanging)

[ https://issues.apache.org/jira/browse/SPARK-31541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau reopened SPARK-31541: -- > Backport SPARK-26095 Disable parallelization in make-distibution.sh. > (Avoid build

[jira] [Commented] (SPARK-31541) Backport SPARK-26095 Disable parallelization in make-distibution.sh. (Avoid build hanging)

[ https://issues.apache.org/jira/browse/SPARK-31541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119930#comment-17119930 ] Holden Karau commented on SPARK-31541: -- Ran into issues with this during the RC7 build so I'm

[jira] [Commented] (SPARK-31756) Add real headless browser support for UI test

[ https://issues.apache.org/jira/browse/SPARK-31756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119792#comment-17119792 ] Dongjoon Hyun commented on SPARK-31756: --- This is resolved via

[jira] [Assigned] (SPARK-31868) Wrong results for week-based-year

[ https://issues.apache.org/jira/browse/SPARK-31868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31868: Assignee: (was: Apache Spark) > Wrong results for week-based-year >

[jira] [Commented] (SPARK-31868) Wrong results for week-based-year

[ https://issues.apache.org/jira/browse/SPARK-31868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119776#comment-17119776 ] Apache Spark commented on SPARK-31868: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-31868) Wrong results for week-based-year

[ https://issues.apache.org/jira/browse/SPARK-31868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31868: Assignee: Apache Spark > Wrong results for week-based-year >

[jira] [Commented] (SPARK-31868) Wrong results for week-based-year

[ https://issues.apache.org/jira/browse/SPARK-31868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119777#comment-17119777 ] Apache Spark commented on SPARK-31868: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Resolved] (SPARK-31858) Upgrade commons-io to 2.5 in Hadoop 3.2 profile

[ https://issues.apache.org/jira/browse/SPARK-31858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31858. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28665

[jira] [Resolved] (SPARK-31214) Upgrade Janino to 3.1.2

[ https://issues.apache.org/jira/browse/SPARK-31214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31214. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 27860

[jira] [Assigned] (SPARK-31214) Upgrade Janino to 3.1.2

[ https://issues.apache.org/jira/browse/SPARK-31214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-31214: - Assignee: Dongjoon Hyun > Upgrade Janino to 3.1.2 > --- > >

[jira] [Assigned] (SPARK-31214) Upgrade Janino to 3.1.2

[ https://issues.apache.org/jira/browse/SPARK-31214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-31214: - Assignee: Jungtaek Lim (was: Dongjoon Hyun) > Upgrade Janino to 3.1.2 >

[jira] [Comment Edited] (SPARK-31815) Support Hive Kerberos login in JDBC connector

[ https://issues.apache.org/jira/browse/SPARK-31815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119607#comment-17119607 ] Agrim Bansal edited comment on SPARK-31815 at 5/29/20, 1:35 PM: The best

[jira] [Commented] (SPARK-31817) Pass-through of Kerberos credentials from Spark SQL to a jdbc source

[ https://issues.apache.org/jira/browse/SPARK-31817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119609#comment-17119609 ] Gabor Somogyi commented on SPARK-31817: --- Thanks [~hyukjin.kwon] pinging me! Interesting topic and

[jira] [Commented] (SPARK-31815) Support Hive Kerberos login in JDBC connector

[ https://issues.apache.org/jira/browse/SPARK-31815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119607#comment-17119607 ] Agrim Bansal commented on SPARK-31815: -- The best thing possible is if we can pass jdbc url as 

[jira] [Comment Edited] (SPARK-31815) Support Hive Kerberos login in JDBC connector

[ https://issues.apache.org/jira/browse/SPARK-31815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119597#comment-17119597 ] Agrim Bansal edited comment on SPARK-31815 at 5/29/20, 1:24 PM: Hi

[jira] [Commented] (SPARK-31815) Support Hive Kerberos login in JDBC connector

[ https://issues.apache.org/jira/browse/SPARK-31815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119597#comment-17119597 ] Agrim Bansal commented on SPARK-31815: -- Hi [~gsomogyi], I am trying to read two dataframes from two

[jira] [Commented] (SPARK-31813) Cannot write snappy-compressed text files

[ https://issues.apache.org/jira/browse/SPARK-31813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119584#comment-17119584 ] Hyukjin Kwon commented on SPARK-31813: -- Well, ORC and Parquet has its own snappy logic IIRC. They

[jira] [Commented] (SPARK-31813) Cannot write snappy-compressed text files

[ https://issues.apache.org/jira/browse/SPARK-31813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119574#comment-17119574 ] Ondrej Kokes commented on SPARK-31813: -- [~hyukjin.kwon] I'm not disputing the lack of

[jira] [Commented] (SPARK-31813) Cannot write snappy-compressed text files

[ https://issues.apache.org/jira/browse/SPARK-31813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119568#comment-17119568 ] Hyukjin Kwon commented on SPARK-31813: -- Seems like you should have native libraries as the error

[jira] [Resolved] (SPARK-31813) Cannot write snappy-compressed text files

[ https://issues.apache.org/jira/browse/SPARK-31813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31813. -- Resolution: Invalid > Cannot write snappy-compressed text files >

[jira] [Commented] (SPARK-31814) Null in Date conversion from yyMMddHHmmss for specific date and time

[ https://issues.apache.org/jira/browse/SPARK-31814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119566#comment-17119566 ] Hyukjin Kwon commented on SPARK-31814: -- Yes, it seems fixed in the master. > Null in Date

[jira] [Resolved] (SPARK-31814) Null in Date conversion from yyMMddHHmmss for specific date and time

[ https://issues.apache.org/jira/browse/SPARK-31814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31814. -- Resolution: Cannot Reproduce > Null in Date conversion from yyMMddHHmmss for specific date

[jira] [Updated] (SPARK-31814) Null in Date conversion from yyMMddHHmmss for specific date and time

[ https://issues.apache.org/jira/browse/SPARK-31814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31814: - Description: Hi,   We are trying to convert a column with string datatype to date type

[jira] [Updated] (SPARK-31814) Null in Date conversion from yyMMddHHmmss for specific date and time

[ https://issues.apache.org/jira/browse/SPARK-31814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31814: - Description: Hi,   We are trying to convert a column with string datatype to date type using

[jira] [Commented] (SPARK-31817) Pass-through of Kerberos credentials from Spark SQL to a jdbc source

[ https://issues.apache.org/jira/browse/SPARK-31817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119548#comment-17119548 ] Hyukjin Kwon commented on SPARK-31817: -- cc [~gsomogyi] FYI > Pass-through of Kerberos credentials

[jira] [Commented] (SPARK-31836) input_file_name() gives wrong value following Python UDF usage

[ https://issues.apache.org/jira/browse/SPARK-31836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119546#comment-17119546 ] Hyukjin Kwon commented on SPARK-31836: -- I believe this is a long standing bug. SPARK-28153 didn't

[jira] [Updated] (SPARK-31836) input_file_name() gives wrong value following Python UDF usage

[ https://issues.apache.org/jira/browse/SPARK-31836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31836: - Component/s: (was: Spark Core) Structured Streaming SQL >

[jira] [Resolved] (SPARK-31841) Dataset.repartition leverage adaptive execution

[ https://issues.apache.org/jira/browse/SPARK-31841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31841. -- Resolution: Duplicate > Dataset.repartition leverage adaptive execution >

[jira] [Updated] (SPARK-26352) Join reordering should not change the order of output attributes

[ https://issues.apache.org/jira/browse/SPARK-26352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-26352: --- Summary: Join reordering should not change the order of output attributes (was: join reordering

[jira] [Commented] (SPARK-31857) Support Azure SQLDB Kerberos login in JDBC connector

[ https://issues.apache.org/jira/browse/SPARK-31857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119488#comment-17119488 ] Gabor Somogyi commented on SPARK-31857: --- I think it's better to add an API where external database

[jira] [Updated] (SPARK-31868) Wrong results for week-based-year

[ https://issues.apache.org/jira/browse/SPARK-31868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-31868: - Description: {code:sql} spark-sql> set spark.sql.legacy.timeParserPolicy=exception;

[jira] [Updated] (SPARK-31868) Wrong results for week-based-year

[ https://issues.apache.org/jira/browse/SPARK-31868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-31868: - Description: {code:sql} spark-sql> set spark.sql.legacy.timeParserPolicy=exception;

[jira] [Updated] (SPARK-31868) Wrong results for week-based-year

[ https://issues.apache.org/jira/browse/SPARK-31868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-31868: - Description: {code:sql} spark-sql> set spark.sql.legacy.timeParserPolicy=exception;

[jira] [Created] (SPARK-31868) Wrong results for week-based-year

Kent Yao created SPARK-31868: Summary: Wrong results for week-based-year Key: SPARK-31868 URL: https://issues.apache.org/jira/browse/SPARK-31868 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-31867) Fix silent data change for datetime formatting

[ https://issues.apache.org/jira/browse/SPARK-31867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-31867: - Description: {code:java} spark-sql> select from_unixtime(1, 'yyy-MM-dd'); NULL spark-sql> set

[jira] [Updated] (SPARK-31867) Fix silent data change for datetime formatting

[ https://issues.apache.org/jira/browse/SPARK-31867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-31867: - Description: {code:java} spark-sql> select from_unixtime(1, 'yyy-MM-dd'); NULL spark-sql> set

[jira] [Assigned] (SPARK-31867) Fix silent data change for datetime formatting

[ https://issues.apache.org/jira/browse/SPARK-31867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31867: Assignee: Apache Spark > Fix silent data change for datetime formatting >

[jira] [Assigned] (SPARK-31867) Fix silent data change for datetime formatting

[ https://issues.apache.org/jira/browse/SPARK-31867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31867: Assignee: (was: Apache Spark) > Fix silent data change for datetime formatting >

[jira] [Commented] (SPARK-31867) Fix silent data change for datetime formatting

[ https://issues.apache.org/jira/browse/SPARK-31867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119389#comment-17119389 ] Apache Spark commented on SPARK-31867: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Updated] (SPARK-31867) Fix silent data change for datetime formatting

[ https://issues.apache.org/jira/browse/SPARK-31867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-31867: - Priority: Blocker (was: Major) > Fix silent data change for datetime formatting >

[jira] [Created] (SPARK-31867) Fix silent data change for datetime formatting

Kent Yao created SPARK-31867: Summary: Fix silent data change for datetime formatting Key: SPARK-31867 URL: https://issues.apache.org/jira/browse/SPARK-31867 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-31866) Add partitioning hints in SQL reference

[ https://issues.apache.org/jira/browse/SPARK-31866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119355#comment-17119355 ] Apache Spark commented on SPARK-31866: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-31866) Add partitioning hints in SQL reference

[ https://issues.apache.org/jira/browse/SPARK-31866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31866: Assignee: Apache Spark > Add partitioning hints in SQL reference >

[jira] [Assigned] (SPARK-31866) Add partitioning hints in SQL reference

[ https://issues.apache.org/jira/browse/SPARK-31866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31866: Assignee: Apache Spark > Add partitioning hints in SQL reference >

[jira] [Assigned] (SPARK-31866) Add partitioning hints in SQL reference

[ https://issues.apache.org/jira/browse/SPARK-31866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31866: Assignee: (was: Apache Spark) > Add partitioning hints in SQL reference >

[jira] [Assigned] (SPARK-28481) More expressions should extend NullIntolerant

[ https://issues.apache.org/jira/browse/SPARK-28481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-28481: --- Assignee: Yuming Wang > More expressions should extend NullIntolerant >

[jira] [Resolved] (SPARK-28481) More expressions should extend NullIntolerant

[ https://issues.apache.org/jira/browse/SPARK-28481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-28481. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28626

[jira] [Commented] (SPARK-31854) Different results of query execution with wholestage codegen on and off

[ https://issues.apache.org/jira/browse/SPARK-31854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119352#comment-17119352 ] Paul Finkelshteyn commented on SPARK-31854: --- What do you mean by query? It is written in

[jira] [Created] (SPARK-31866) Add partitioning hints in SQL reference

Huaxin Gao created SPARK-31866: -- Summary: Add partitioning hints in SQL reference Key: SPARK-31866 URL: https://issues.apache.org/jira/browse/SPARK-31866 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-31854) Different results of query execution with wholestage codegen on and off

[ https://issues.apache.org/jira/browse/SPARK-31854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119334#comment-17119334 ] Takeshi Yamamuro commented on SPARK-31854: -- Could you show us a query to reproduce the issue in

[jira] [Commented] (SPARK-31779) Redefining struct inside array incorrectly wraps child fields in array

[ https://issues.apache.org/jira/browse/SPARK-31779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119319#comment-17119319 ] Pablo Langa Blanco commented on SPARK-31779: How about using array_zip? {code:java} // code