[jira] [Commented] (SPARK-32084) Replace dictionary-based function definitions to proper functions in functions.py

2020-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217263#comment-17217263 ] Hyukjin Kwon commented on SPARK-32084: -- [~zero323] are you interested in this? > Replace

[jira] [Resolved] (SPARK-33182) Using Delegation Tokens to access HBase through Spark (Java)

2020-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33182. -- Resolution: Incomplete > Using Delegation Tokens to access HBase through Spark (Java) >

[jira] [Updated] (SPARK-33184) spark doesn't read data source column if it is used as an index to an array under a struct

2020-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33184: - Component/s: (was: Spark Core) SQL > spark doesn't read data source column

[jira] [Commented] (SPARK-33182) Using Delegation Tokens to access HBase through Spark (Java)

2020-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217262#comment-17217262 ] Hyukjin Kwon commented on SPARK-33182: -- Spark 2.3.x is EOL. Mind testing in higher Spark versions?

[jira] [Updated] (SPARK-33188) PipelineModel load resulting in error

2020-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33188: - Component/s: ML > PipelineModel load resulting in error > -

[jira] [Assigned] (SPARK-33140) make Analyzer rules using SQLConf.get

2020-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33140: Assignee: Apache Spark > make Analyzer rules using SQLConf.get >

[jira] [Commented] (SPARK-33140) make Analyzer rules using SQLConf.get

2020-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217257#comment-17217257 ] Apache Spark commented on SPARK-33140: -- User 'leanken' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33140) make Analyzer rules using SQLConf.get

2020-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33140: Assignee: (was: Apache Spark) > make Analyzer rules using SQLConf.get >

[jira] [Created] (SPARK-33188) PipelineModel load resulting in error

2020-10-19 Thread rohit agarwal (Jira)
rohit agarwal created SPARK-33188: - Summary: PipelineModel load resulting in error Key: SPARK-33188 URL: https://issues.apache.org/jira/browse/SPARK-33188 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-33187) Add a check on the number of returned partitions in the HiveShim#getPartitionsByFilter method

2020-10-19 Thread jinhai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jinhai updated SPARK-33187: --- Description: In the method HiveShim#getPartitionsByFilter, when filter is empty or when the hive table has

[jira] [Assigned] (SPARK-32351) Partially pushed partition filters are not explained

2020-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32351: Assignee: Nan Zhu (was: pavithra ramachandran) > Partially pushed partition filters are

[jira] [Commented] (SPARK-32351) Partially pushed partition filters are not explained

2020-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217246#comment-17217246 ] Hyukjin Kwon commented on SPARK-32351: -- Oops, sure. > Partially pushed partition filters are not

[jira] [Created] (SPARK-33187) Add a check on the number of returned partitions in the HiveShim#getPartitionsByFilter method

2020-10-19 Thread jinhai (Jira)
jinhai created SPARK-33187: -- Summary: Add a check on the number of returned partitions in the HiveShim#getPartitionsByFilter method Key: SPARK-33187 URL: https://issues.apache.org/jira/browse/SPARK-33187

[jira] [Updated] (SPARK-33186) Respect ANSI standard and change FIRST_VALUE as pure window function.

2020-10-19 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-33186: --- Description: FIRST_VALUE is an ANSI window function. But the current implement of FIRST_VALUE in

[jira] [Commented] (SPARK-33186) Respect ANSI standard and change FIRST_VALUE as pure window function.

2020-10-19 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217240#comment-17217240 ] jiaan.geng commented on SPARK-33186: I'm working on. > Respect ANSI standard and change FIRST_VALUE

[jira] [Created] (SPARK-33186) Respect ANSI standard and change FIRST_VALUE as pure window function.

2020-10-19 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-33186: -- Summary: Respect ANSI standard and change FIRST_VALUE as pure window function. Key: SPARK-33186 URL: https://issues.apache.org/jira/browse/SPARK-33186 Project: Spark

[jira] [Commented] (SPARK-32351) Partially pushed partition filters are not explained

2020-10-19 Thread Nan Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217238#comment-17217238 ] Nan Zhu commented on SPARK-32351: - [~hyukjin.kwon] nit: could you reassign this to me? [~codingcat] ;)

[jira] [Updated] (SPARK-32863) Full outer stream-stream join

2020-10-19 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-32863: - Priority: Major (was: Trivial) > Full outer stream-stream join > -

[jira] [Updated] (SPARK-32883) Stream-stream join improvement

2020-10-19 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-32883: - Priority: Major (was: Minor) > Stream-stream join improvement > --

[jira] [Updated] (SPARK-32862) Left semi stream-stream join

2020-10-19 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-32862: - Priority: Major (was: Minor) > Left semi stream-stream join > > >

[jira] [Assigned] (SPARK-32351) Partially pushed partition filters are not explained

2020-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32351: Assignee: pavithra ramachandran > Partially pushed partition filters are not explained >

[jira] [Resolved] (SPARK-32351) Partially pushed partition filters are not explained

2020-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32351. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29831

[jira] [Resolved] (SPARK-33181) SQL Reference: Run SQL on files directly

2020-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33181. -- Fix Version/s: 3.0.2 3.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-33181) SQL Reference: Run SQL on files directly

2020-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33181: Assignee: Aoyuan Liao > SQL Reference: Run SQL on files directly >

[jira] [Commented] (SPARK-31760) Simplification Based on Containment

2020-10-19 Thread Aoyuan Liao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217180#comment-17217180 ] Aoyuan Liao commented on SPARK-31760: - [~yumwang] To implement the containment rule:  If A contains

[jira] [Updated] (SPARK-31760) Simplification Based on Containment

2020-10-19 Thread Aoyuan Liao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aoyuan Liao updated SPARK-31760: Attachment: image-2020-10-19-17-18-31-849.png > Simplification Based on Containment >

[jira] [Commented] (SPARK-33185) YARN: Print direct links to driver logs alongside application report in cluster mode

2020-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217169#comment-17217169 ] Apache Spark commented on SPARK-33185: -- User 'xkrogen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33185) YARN: Print direct links to driver logs alongside application report in cluster mode

2020-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33185: Assignee: (was: Apache Spark) > YARN: Print direct links to driver logs alongside

[jira] [Commented] (SPARK-33185) YARN: Print direct links to driver logs alongside application report in cluster mode

2020-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217167#comment-17217167 ] Apache Spark commented on SPARK-33185: -- User 'xkrogen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33185) YARN: Print direct links to driver logs alongside application report in cluster mode

2020-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33185: Assignee: Apache Spark > YARN: Print direct links to driver logs alongside application

[jira] [Updated] (SPARK-33185) YARN: Print direct links to driver logs alongside application report in cluster mode

2020-10-19 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated SPARK-33185: Description: Currently when run in {{cluster}} mode on YARN, the Spark {{yarn.Client}} will

[jira] [Created] (SPARK-33185) YARN: Print direct links to driver logs alongside application report in cluster mode

2020-10-19 Thread Erik Krogen (Jira)
Erik Krogen created SPARK-33185: --- Summary: YARN: Print direct links to driver logs alongside application report in cluster mode Key: SPARK-33185 URL: https://issues.apache.org/jira/browse/SPARK-33185

[jira] [Resolved] (SPARK-33069) Skip test result report if no JUnit XML files are found

2020-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33069. -- Resolution: Not A Problem > Skip test result report if no JUnit XML files are found >

[jira] [Assigned] (SPARK-33181) SQL Reference: Run SQL on files directly

2020-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33181: Assignee: Apache Spark > SQL Reference: Run SQL on files directly >

[jira] [Assigned] (SPARK-33181) SQL Reference: Run SQL on files directly

2020-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33181: Assignee: (was: Apache Spark) > SQL Reference: Run SQL on files directly >

[jira] [Commented] (SPARK-33003) Add type hints guideliness to the documentation

2020-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217137#comment-17217137 ] Apache Spark commented on SPARK-33003: -- User 'zero323' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33003) Add type hints guideliness to the documentation

2020-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33003: Assignee: (was: Apache Spark) > Add type hints guideliness to the documentation >

[jira] [Commented] (SPARK-33181) SQL Reference: Run SQL on files directly

2020-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217139#comment-17217139 ] Apache Spark commented on SPARK-33181: -- User 'liaoaoyuan97' has created a pull request for this

[jira] [Assigned] (SPARK-33003) Add type hints guideliness to the documentation

2020-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33003: Assignee: Apache Spark > Add type hints guideliness to the documentation >

[jira] [Commented] (SPARK-33044) Add a Jenkins build and test job for Scala 2.13

2020-10-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217075#comment-17217075 ] Dongjoon Hyun commented on SPARK-33044: --- Gentle ping, [~shaneknapp]. Could you setup for Scala

[jira] [Commented] (SPARK-33146) Encountering an invalid rolling event log folder prevents loading other applications in SHS

2020-10-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217064#comment-17217064 ] Dongjoon Hyun commented on SPARK-33146: --- Hi, [~kabhwan] I don't think the revert of revert is a

[jira] [Assigned] (SPARK-17333) Make pyspark interface friendly with mypy static analysis

2020-10-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-17333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-17333: - Assignee: Fokko Driesprong > Make pyspark interface friendly with mypy static analysis

[jira] [Resolved] (SPARK-17333) Make pyspark interface friendly with mypy static analysis

2020-10-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-17333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-17333. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30088

[jira] [Commented] (SPARK-33181) SQL Reference: Run SQL on files directly

2020-10-19 Thread Aoyuan Liao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217023#comment-17217023 ] Aoyuan Liao commented on SPARK-33181: - [~smilegator] If this one hasn't been done, can I work on it?

[jira] [Commented] (SPARK-33183) Bug in optimizer rule EliminateSorts

2020-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217008#comment-17217008 ] Apache Spark commented on SPARK-33183: -- User 'allisonwang-db' has created a pull request for this

[jira] [Assigned] (SPARK-33183) Bug in optimizer rule EliminateSorts

2020-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33183: Assignee: Apache Spark > Bug in optimizer rule EliminateSorts >

[jira] [Assigned] (SPARK-33183) Bug in optimizer rule EliminateSorts

2020-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33183: Assignee: (was: Apache Spark) > Bug in optimizer rule EliminateSorts >

[jira] [Commented] (SPARK-33184) spark doesn't read data source column if it is used as an index to an array under a struct

2020-10-19 Thread colin fang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217001#comment-17217001 ] colin fang commented on SPARK-33184: I notice there is a quotation mark before `Project`. What does

[jira] [Updated] (SPARK-33184) spark doesn't read data source column if it is used as an index to an array under a struct

2020-10-19 Thread colin fang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] colin fang updated SPARK-33184: --- Issue Type: Bug (was: Improvement) > spark doesn't read data source column if it is used as an

[jira] [Updated] (SPARK-33184) spark doesn't read data source column if it is used as an index to an array under a struct

2020-10-19 Thread colin fang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] colin fang updated SPARK-33184: --- Summary: spark doesn't read data source column if it is used as an index to an array under a struct

[jira] [Updated] (SPARK-33184) spark doesn't read data source column if it is needed as an index to an array in a nested struct

2020-10-19 Thread colin fang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] colin fang updated SPARK-33184: --- Description: {code:python} df = spark.createDataFrame([[1, [[1, 2, schema='x:int,y:struct>')

[jira] [Updated] (SPARK-33184) spark doesn't read data source column if it is needed as an index to an array in a nested struct

2020-10-19 Thread colin fang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] colin fang updated SPARK-33184: --- Description: {code:python} df = spark.createDataFrame([[1, [[1, 2, schema='x:int,y:struct>')

[jira] [Created] (SPARK-33184) spark doesn't read data source column if it is needed as an index to an array in a nested struct

2020-10-19 Thread colin fang (Jira)
colin fang created SPARK-33184: -- Summary: spark doesn't read data source column if it is needed as an index to an array in a nested struct Key: SPARK-33184 URL: https://issues.apache.org/jira/browse/SPARK-33184

[jira] [Created] (SPARK-33183) Bug in optimizer rule EliminateSorts

2020-10-19 Thread Allison Wang (Jira)
Allison Wang created SPARK-33183: Summary: Bug in optimizer rule EliminateSorts Key: SPARK-33183 URL: https://issues.apache.org/jira/browse/SPARK-33183 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-33069) Skip test result report if no JUnit XML files are found

2020-10-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33069: -- Fix Version/s: (was: 3.0.2) (was: 2.4.8)

[jira] [Reopened] (SPARK-33069) Skip test result report if no JUnit XML files are found

2020-10-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-33069: --- Assignee: (was: Hyukjin Kwon) > Skip test result report if no JUnit XML files are

[jira] [Resolved] (SPARK-32941) Optimize WithFields expression chain

2020-10-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32941. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29812

[jira] [Updated] (SPARK-33182) Using Delegation Tokens to access HBase through Spark (Java)

2020-10-19 Thread Rishi S Balajis (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rishi S Balajis updated SPARK-33182: Description: I have a requirement to access a kerberized HBase through delegation tokens

[jira] [Updated] (SPARK-33182) Using Delegation Tokens to access HBase through Spark (Java)

2020-10-19 Thread Rishi S Balajis (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rishi S Balajis updated SPARK-33182: Description: I have a requirement to access a kerberized HBase through delegation tokens

[jira] [Updated] (SPARK-33182) Using Delegation Tokens to access HBase through Spark (Java)

2020-10-19 Thread Rishi S Balajis (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rishi S Balajis updated SPARK-33182: Affects Version/s: (was: 2.4.0) 2.3.0 > Using Delegation

[jira] [Created] (SPARK-33182) Using Delegation Tokens to access HBase through Spark (Java)

2020-10-19 Thread Rishi S Balajis (Jira)
Rishi S Balajis created SPARK-33182: --- Summary: Using Delegation Tokens to access HBase through Spark (Java) Key: SPARK-33182 URL: https://issues.apache.org/jira/browse/SPARK-33182 Project: Spark

[jira] [Commented] (SPARK-24266) Spark client terminates while driver is still running

2020-10-19 Thread Martin Tapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216759#comment-17216759 ] Martin Tapp commented on SPARK-24266: - Can we update Fix version to 3.0.2 then? Thanks > Spark

[jira] [Commented] (SPARK-24266) Spark client terminates while driver is still running

2020-10-19 Thread Nicola Bova (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216749#comment-17216749 ] Nicola Bova commented on SPARK-24266: - It would be great if the fix could be target to 3.0.2. >

[jira] [Commented] (SPARK-33168) spark REST API Unable to get JobDescription

2020-10-19 Thread zhaoyachao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216712#comment-17216712 ] zhaoyachao commented on SPARK-33168: I don't know what I did wrong, this result is not what I

[jira] [Updated] (SPARK-33168) spark REST API Unable to get JobDescription

2020-10-19 Thread zhaoyachao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhaoyachao updated SPARK-33168: --- Attachment: (was: image-2020-10-19-21-08-45-123.png) > spark REST API Unable to get

[jira] [Updated] (SPARK-33168) spark REST API Unable to get JobDescription

2020-10-19 Thread zhaoyachao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhaoyachao updated SPARK-33168: --- Attachment: (was: image-2020-10-19-21-07-01-514.png) > spark REST API Unable to get

[jira] [Updated] (SPARK-33168) spark REST API Unable to get JobDescription

2020-10-19 Thread zhaoyachao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhaoyachao updated SPARK-33168: --- Attachment: image-2020-10-19-21-13-09-960.png > spark REST API Unable to get JobDescription >

[jira] [Updated] (SPARK-33168) spark REST API Unable to get JobDescription

2020-10-19 Thread zhaoyachao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhaoyachao updated SPARK-33168: --- Attachment: image-2020-10-19-21-12-41-145.png > spark REST API Unable to get JobDescription >

[jira] [Updated] (SPARK-33168) spark REST API Unable to get JobDescription

2020-10-19 Thread zhaoyachao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhaoyachao updated SPARK-33168: --- Attachment: image-2020-10-19-21-08-45-123.png > spark REST API Unable to get JobDescription >

[jira] [Updated] (SPARK-33168) spark REST API Unable to get JobDescription

2020-10-19 Thread zhaoyachao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhaoyachao updated SPARK-33168: --- Attachment: image-2020-10-19-21-07-01-514.png > spark REST API Unable to get JobDescription >

[jira] [Resolved] (SPARK-33169) Check propagation of datasource options to underlying file system for built-in file-based datasources

2020-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33169. -- Fix Version/s: 3.1.0 Resolution: Fixed Fixed in

[jira] [Commented] (SPARK-33128) mismatched input since Spark 3.0

2020-10-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216539#comment-17216539 ] Yuming Wang commented on SPARK-33128: - Thank you [~LuciferYang]. > mismatched input since Spark 3.0

[jira] [Resolved] (SPARK-33069) Skip test result report if no JUnit XML files are found

2020-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33069. -- Resolution: Invalid We don't need this fix anymore. Now it fails to download when there are

[jira] [Reopened] (SPARK-33069) Skip test result report if no JUnit XML files are found

2020-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-33069: -- Reverted at https://github.com/apache/spark/commit/a7a8dae4836f455a26ba6cb3c7d733775b6af0f6 >

[jira] [Resolved] (SPARK-33180) Enables 'fail_if_no_tests' when reporting test results

2020-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33180. -- Resolution: Invalid Now it aborts at downloading when there are no artifacts to download, see

[jira] [Created] (SPARK-33181) SQL Reference: Run SQL on files directly

2020-10-19 Thread Xiao Li (Jira)
Xiao Li created SPARK-33181: --- Summary: SQL Reference: Run SQL on files directly Key: SPARK-33181 URL: https://issues.apache.org/jira/browse/SPARK-33181 Project: Spark Issue Type: Documentation

[jira] [Updated] (SPARK-33181) SQL Reference: Run SQL on files directly

2020-10-19 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-33181: Labels: starter (was: ) > SQL Reference: Run SQL on files directly >

[jira] [Resolved] (SPARK-33179) Switch default Hadoop profile in run-tests.py

2020-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33179. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30090

[jira] [Assigned] (SPARK-33179) Switch default Hadoop profile in run-tests.py

2020-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33179: Assignee: William Hyun > Switch default Hadoop profile in run-tests.py >

[jira] [Comment Edited] (SPARK-33128) mismatched input since Spark 3.0

2020-10-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216485#comment-17216485 ] Yang Jie edited comment on SPARK-33128 at 10/19/20, 6:43 AM: - [~yumwang] I

[jira] [Commented] (SPARK-33128) mismatched input since Spark 3.0

2020-10-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216485#comment-17216485 ] Yang Jie commented on SPARK-33128: -- [~yumwang] I found that without SPARK-21136, case can be passed and

[jira] [Updated] (SPARK-32557) Logging and Swallowing the Exception Per Entry in History Server

2020-10-19 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-32557: - Fix Version/s: 3.0.2 > Logging and Swallowing the Exception Per Entry in History Server >

[jira] [Updated] (SPARK-33146) Encountering an invalid rolling event log folder prevents loading other applications in SHS

2020-10-19 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-33146: - Fix Version/s: 3.0.2 > Encountering an invalid rolling event log folder prevents loading other

[jira] [Commented] (SPARK-33139) protect setActiveSession and clearActiveSession

2020-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216468#comment-17216468 ] Apache Spark commented on SPARK-33139: -- User 'leanken' has created a pull request for this issue:

[jira] [Commented] (SPARK-33139) protect setActiveSession and clearActiveSession

2020-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216467#comment-17216467 ] Apache Spark commented on SPARK-33139: -- User 'leanken' has created a pull request for this issue: