[jira] [Comment Edited] (SPARK-27742) Security Support in Sources and Sinks for SS and Batch

2019-05-30 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851924#comment-16851924 ] Stavros Kontopoulos edited comment on SPARK-27742 at 5/30/19 2:28 PM:

[jira] [Comment Edited] (SPARK-27742) Security Support in Sources and Sinks for SS and Batch

2019-05-30 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851924#comment-16851924 ] Stavros Kontopoulos edited comment on SPARK-27742 at 5/30/19 2:27 PM:

[jira] [Comment Edited] (SPARK-27742) Security Support in Sources and Sinks for SS and Batch

2019-05-30 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851924#comment-16851924 ] Stavros Kontopoulos edited comment on SPARK-27742 at 5/30/19 2:24 PM:

[jira] [Comment Edited] (SPARK-27742) Security Support in Sources and Sinks for SS and Batch

2019-05-30 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851924#comment-16851924 ] Stavros Kontopoulos edited comment on SPARK-27742 at 5/30/19 2:23 PM:

[jira] [Commented] (SPARK-27742) Security Support in Sources and Sinks for SS and Batch

2019-05-30 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851924#comment-16851924 ] Stavros Kontopoulos commented on SPARK-27742: - {quote}From client side the m

[jira] [Commented] (SPARK-25557) ORC predicate pushdown for nested fields

2019-05-30 Thread Ivan Vergiliev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851920#comment-16851920 ] Ivan Vergiliev commented on SPARK-25557: It seems like this was partially addres

[jira] [Created] (SPARK-27883) Add aggregates.sql - Part2

2019-05-30 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27883: --- Summary: Add aggregates.sql - Part2 Key: SPARK-27883 URL: https://issues.apache.org/jira/browse/SPARK-27883 Project: Spark Issue Type: Sub-task Compo

[jira] [Updated] (SPARK-27882) Support SQL:2016 compatible datetime patterns

2019-05-30 Thread Peter Toth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-27882: --- Description: Date and time related functions in Spark use [DateTimeFormatter|https://docs.oracle.co

[jira] [Updated] (SPARK-27881) Support CAST (... FORMAT ) expression

2019-05-30 Thread Peter Toth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-27881: --- Description: SQL:2016 standard supports the following Cast functions {noformat} CAST( AS [FORMAT ])

[jira] [Created] (SPARK-27882) Support SQL:2016 compatible datetime patterns

2019-05-30 Thread Peter Toth (JIRA)
Peter Toth created SPARK-27882: -- Summary: Support SQL:2016 compatible datetime patterns Key: SPARK-27882 URL: https://issues.apache.org/jira/browse/SPARK-27882 Project: Spark Issue Type: Improve

[jira] [Commented] (SPARK-24149) Automatic namespaces discovery in HDFS federation

2019-05-30 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851896#comment-16851896 ] Marco Gaido commented on SPARK-24149: - That's true, the point is: if you want to acc

[jira] [Commented] (SPARK-27742) Security Support in Sources and Sinks for SS and Batch

2019-05-30 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851883#comment-16851883 ] Gabor Somogyi commented on SPARK-27742: --- {quote}what happens with Kafka delegation

[jira] [Commented] (SPARK-27798) from_avro can modify variables in other rows in local mode

2019-05-30 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851877#comment-16851877 ] Gengliang Wang commented on SPARK-27798: [~ykmori15]Thanks for reporting. I am l

[jira] [Assigned] (SPARK-27881) Support CAST (... FORMAT ) expression

2019-05-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27881: Assignee: Apache Spark > Support CAST (... FORMAT ) expression >

[jira] [Assigned] (SPARK-27881) Support CAST (... FORMAT ) expression

2019-05-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27881: Assignee: (was: Apache Spark) > Support CAST (... FORMAT ) expression > -

[jira] [Created] (SPARK-27881) Support CAST (... FORMAT ) expression

2019-05-30 Thread Peter Toth (JIRA)
Peter Toth created SPARK-27881: -- Summary: Support CAST (... FORMAT ) expression Key: SPARK-27881 URL: https://issues.apache.org/jira/browse/SPARK-27881 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-27742) Security Support in Sources and Sinks for SS and Batch

2019-05-30 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851828#comment-16851828 ] Stavros Kontopoulos edited comment on SPARK-27742 at 5/30/19 12:57 PM: ---

[jira] [Commented] (SPARK-27742) Security Support in Sources and Sinks for SS and Batch

2019-05-30 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851828#comment-16851828 ] Stavros Kontopoulos commented on SPARK-27742: - [~gsomogyi] what happens with

[jira] [Resolved] (SPARK-27835) Resource Scheduling: change driver config from addresses to resourcesFile

2019-05-30 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-27835. --- Resolution: Fixed Fix Version/s: 3.0.0 > Resource Scheduling: change driver config fr

[jira] [Resolved] (SPARK-27875) Wrap all PrintWriter with Utils.tryWithResource

2019-05-30 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27875. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24739 [https://gi

[jira] [Assigned] (SPARK-27875) Wrap all PrintWriter with Utils.tryWithResource

2019-05-30 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-27875: Assignee: Yuming Wang > Wrap all PrintWriter with Utils.tryWithResource > ---

[jira] [Resolved] (SPARK-27706) Add SQL metrics of numOutputRows for BroadcastExchangeExec

2019-05-30 Thread dzcxzl (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dzcxzl resolved SPARK-27706. Resolution: Not A Problem > Add SQL metrics of numOutputRows for BroadcastExchangeExec > -

[jira] [Updated] (SPARK-27875) Wrap all PrintWriter with Utils.tryWithResource

2019-05-30 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27875: Summary: Wrap all PrintWriter with Utils.tryWithResource (was: Wrap all PrintWrite with Utils.try

[jira] [Updated] (SPARK-27630) Stage retry causes totalRunningTasks calculation to be negative

2019-05-30 Thread dzcxzl (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dzcxzl updated SPARK-27630: --- Description: In the case of stage retry, the {{taskEnd}} event from the zombie stage sometimes makes the nu

[jira] [Updated] (SPARK-27788) Session aware event log for Spark thrift server

2019-05-30 Thread Lantao Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin updated SPARK-27788: --- Affects Version/s: (was: 2.4.3) 3.0.0 > Session aware event log for Spark

[jira] [Updated] (SPARK-27880) Implement boolean aggregates(BOOL_AND, BOOL_OR and EVERY)

2019-05-30 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27880: Description: {code:sql} bool_and(expression) -- true if all input values are true, otherwise false

[jira] [Created] (SPARK-27880) Implement boolean aggregates(BOOL_AND, BOOL_OR and EVERY)

2019-05-30 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27880: --- Summary: Implement boolean aggregates(BOOL_AND, BOOL_OR and EVERY) Key: SPARK-27880 URL: https://issues.apache.org/jira/browse/SPARK-27880 Project: Spark Issue

[jira] [Created] (SPARK-27879) Implement bitwise integer aggregates(BIT_AND and BIT_OR)

2019-05-30 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27879: --- Summary: Implement bitwise integer aggregates(BIT_AND and BIT_OR) Key: SPARK-27879 URL: https://issues.apache.org/jira/browse/SPARK-27879 Project: Spark Issue

[jira] [Created] (SPARK-27878) Support ARRAY(sub-SELECT) expressions

2019-05-30 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27878: --- Summary: Support ARRAY(sub-SELECT) expressions Key: SPARK-27878 URL: https://issues.apache.org/jira/browse/SPARK-27878 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-27877) Implement SQL-standard LATERAL subqueries

2019-05-30 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27877: --- Summary: Implement SQL-standard LATERAL subqueries Key: SPARK-27877 URL: https://issues.apache.org/jira/browse/SPARK-27877 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-27784) Alias ID reuse can break correctness when substituting foldable expressions

2019-05-30 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851588#comment-16851588 ] Dongjoon Hyun commented on SPARK-27784: --- Is this still in `branch-2.4`, too? > Al

[jira] [Updated] (SPARK-27798) from_avro can modify variables in other rows in local mode

2019-05-30 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27798: -- Priority: Blocker (was: Major) > from_avro can modify variables in other rows in local mode >

[jira] [Commented] (SPARK-27798) from_avro can modify variables in other rows in local mode

2019-05-30 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851586#comment-16851586 ] Dongjoon Hyun commented on SPARK-27798: --- BTW, thank you for reporting, [~ykmori15]

[jira] [Commented] (SPARK-27798) from_avro can modify variables in other rows in local mode

2019-05-30 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851583#comment-16851583 ] Dongjoon Hyun commented on SPARK-27798: --- Ur, this seems to exist in `branch-2.4` u

<    1   2