[jira] [Reopened] (SPARK-29345) Add an API that allows a user to define and observe arbitrary metrics on batch and streaming queries

2020-06-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-29345: -- > Add an API that allows a user to define and observe arbitrary metrics on > batch and streaming

[jira] [Resolved] (SPARK-29345) Add an API that allows a user to define and observe arbitrary metrics on batch and streaming queries

2020-06-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29345. -- Resolution: Done > Add an API that allows a user to define and observe arbitrary metrics on >

[jira] [Resolved] (SPARK-32006) Create date/timestamp formatters once before collect in `hiveResultString()`

2020-06-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32006. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28842 [https://gith

[jira] [Assigned] (SPARK-32006) Create date/timestamp formatters once before collect in `hiveResultString()`

2020-06-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32006: --- Assignee: Maxim Gekk > Create date/timestamp formatters once before collect in `hiveResultS

[jira] [Commented] (SPARK-29345) Add an API that allows a user to define and observe arbitrary metrics on batch and streaming queries

2020-06-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138120#comment-17138120 ] Wenchen Fan commented on SPARK-29345: - I'm resolving it as the only remaining thing

[jira] [Resolved] (SPARK-29345) Add an API that allows a user to define and observe arbitrary metrics on batch and streaming queries

2020-06-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29345. - Fix Version/s: 3.0.0 Resolution: Fixed > Add an API that allows a user to define and obse

[jira] [Commented] (SPARK-30108) Add robust accumulator for observable metrics

2020-06-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138118#comment-17138118 ] Wenchen Fan commented on SPARK-30108: - is there any progress on it? > Add robust ac

[jira] [Commented] (SPARK-32003) Shuffle files for lost executor are not unregistered if fetch failure occurs after executor is lost

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138100#comment-17138100 ] Apache Spark commented on SPARK-32003: -- User 'wypoon' has created a pull request fo

[jira] [Assigned] (SPARK-32003) Shuffle files for lost executor are not unregistered if fetch failure occurs after executor is lost

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32003: Assignee: Apache Spark > Shuffle files for lost executor are not unregistered if fetch fa

[jira] [Commented] (SPARK-32003) Shuffle files for lost executor are not unregistered if fetch failure occurs after executor is lost

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138099#comment-17138099 ] Apache Spark commented on SPARK-32003: -- User 'wypoon' has created a pull request fo

[jira] [Assigned] (SPARK-32003) Shuffle files for lost executor are not unregistered if fetch failure occurs after executor is lost

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32003: Assignee: (was: Apache Spark) > Shuffle files for lost executor are not unregistered

[jira] [Commented] (SPARK-30876) Optimizer cannot infer from inferred constraints with join

2020-06-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138072#comment-17138072 ] Yuming Wang commented on SPARK-30876: - Thank you [~navinvishy]. > Optimizer cannot

[jira] [Assigned] (SPARK-32011) Remove warnings about pin-thread modes and guide collectWithJobGroups for now

2020-06-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32011: Assignee: Hyukjin Kwon > Remove warnings about pin-thread modes and guide collectWithJobG

[jira] [Resolved] (SPARK-32011) Remove warnings about pin-thread modes and guide collectWithJobGroups for now

2020-06-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32011. -- Fix Version/s: 3.1.0 3.0.1 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-31989) Generate JSON files with rebasing switch points using smaller steps

2020-06-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-31989: Assignee: Maxim Gekk > Generate JSON files with rebasing switch points using smaller step

[jira] [Resolved] (SPARK-31989) Generate JSON files with rebasing switch points using smaller steps

2020-06-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31989. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28827 [https://gi

[jira] [Commented] (SPARK-29148) Modify dynamic allocation manager for stage level scheduling

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138046#comment-17138046 ] Apache Spark commented on SPARK-29148: -- User 'dongjoon-hyun' has created a pull req

[jira] [Commented] (SPARK-29148) Modify dynamic allocation manager for stage level scheduling

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138048#comment-17138048 ] Apache Spark commented on SPARK-29148: -- User 'dongjoon-hyun' has created a pull req

[jira] [Updated] (SPARK-32002) spark error while select nest data

2020-06-16 Thread Yiqun Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Zhang updated SPARK-32002: Priority: Major (was: Minor) > spark error while select nest data > -

[jira] [Assigned] (SPARK-32012) Incrementally create and materialize query stage to avoid unnecessary local shuffle

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32012: Assignee: L. C. Hsieh (was: Apache Spark) > Incrementally create and materialize query s

[jira] [Assigned] (SPARK-32012) Incrementally create and materialize query stage to avoid unnecessary local shuffle

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32012: Assignee: Apache Spark (was: L. C. Hsieh) > Incrementally create and materialize query s

[jira] [Commented] (SPARK-32012) Incrementally create and materialize query stage to avoid unnecessary local shuffle

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138023#comment-17138023 ] Apache Spark commented on SPARK-32012: -- User 'viirya' has created a pull request fo

[jira] [Commented] (SPARK-32012) Incrementally create and materialize query stage to avoid unnecessary local shuffle

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138024#comment-17138024 ] Apache Spark commented on SPARK-32012: -- User 'viirya' has created a pull request fo

[jira] [Created] (SPARK-32012) Incrementally create and materialize query stage to avoid unnecessary local shuffle

2020-06-16 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-32012: --- Summary: Incrementally create and materialize query stage to avoid unnecessary local shuffle Key: SPARK-32012 URL: https://issues.apache.org/jira/browse/SPARK-32012 Pro

[jira] [Commented] (SPARK-32011) Remove warnings about pin-thread modes and guide collectWithJobGroups for now

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138014#comment-17138014 ] Apache Spark commented on SPARK-32011: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-32011) Remove warnings about pin-thread modes and guide collectWithJobGroups for now

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32011: Assignee: Apache Spark > Remove warnings about pin-thread modes and guide collectWithJobG

[jira] [Commented] (SPARK-32010) Thread leaks in pinned thread mode

2020-06-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138012#comment-17138012 ] Hyukjin Kwon commented on SPARK-32010: -- I have a kind of a POC fix already. Will op

[jira] [Commented] (SPARK-32011) Remove warnings about pin-thread modes and guide collectWithJobGroups for now

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138013#comment-17138013 ] Apache Spark commented on SPARK-32011: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-32011) Remove warnings about pin-thread modes and guide collectWithJobGroups for now

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32011: Assignee: (was: Apache Spark) > Remove warnings about pin-thread modes and guide coll

[jira] [Created] (SPARK-32010) Thread leaks in pinned thread mode

2020-06-16 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-32010: Summary: Thread leaks in pinned thread mode Key: SPARK-32010 URL: https://issues.apache.org/jira/browse/SPARK-32010 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-32010) Thread leaks in pinned thread mode

2020-06-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32010: Assignee: Hyukjin Kwon > Thread leaks in pinned thread mode > ---

[jira] [Resolved] (SPARK-31337) Support MS Sql Kerberos login in JDBC connector

2020-06-16 Thread Marcelo Masiero Vanzin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-31337. Fix Version/s: 3.1.0 Assignee: Gabor Somogyi Resolution: Fi

[jira] [Created] (SPARK-32011) Remove warnings about pin-thread modes and guide collectWithJobGroups for now

2020-06-16 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-32011: Summary: Remove warnings about pin-thread modes and guide collectWithJobGroups for now Key: SPARK-32011 URL: https://issues.apache.org/jira/browse/SPARK-32011 Project

[jira] [Commented] (SPARK-32009) remove deprecated method BisectingKMeansModel.computeCost

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17137996#comment-17137996 ] Apache Spark commented on SPARK-32009: -- User 'huaxingao' has created a pull request

[jira] [Assigned] (SPARK-32009) remove deprecated method BisectingKMeansModel.computeCost

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32009: Assignee: Apache Spark > remove deprecated method BisectingKMeansModel.computeCost >

[jira] [Commented] (SPARK-32009) remove deprecated method BisectingKMeansModel.computeCost

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17137995#comment-17137995 ] Apache Spark commented on SPARK-32009: -- User 'huaxingao' has created a pull request

[jira] [Assigned] (SPARK-32009) remove deprecated method BisectingKMeansModel.computeCost

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32009: Assignee: (was: Apache Spark) > remove deprecated method BisectingKMeansModel.compute

[jira] [Updated] (SPARK-32009) remove deprecated method BisectingKMeansModel.computeCost

2020-06-16 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-32009: --- Summary: remove deprecated method BisectingKMeansModel.computeCost (was: remove deprecated method B

[jira] [Resolved] (SPARK-31777) CrossValidator supports user-supplied folds

2020-06-16 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-31777. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28704 [https://gith

[jira] [Assigned] (SPARK-31777) CrossValidator supports user-supplied folds

2020-06-16 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-31777: --- Assignee: L. C. Hsieh > CrossValidator supports user-supplied folds >

[jira] [Created] (SPARK-32009) remove deprecated method BisectingKMeans.computeCost

2020-06-16 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-32009: -- Summary: remove deprecated method BisectingKMeans.computeCost Key: SPARK-32009 URL: https://issues.apache.org/jira/browse/SPARK-32009 Project: Spark Issue Type:

[jira] [Commented] (SPARK-32008) 3.0.0 release build fails

2020-06-16 Thread Shivaram Venkataraman (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17137937#comment-17137937 ] Shivaram Venkataraman commented on SPARK-32008: --- It looks like the R vigne

[jira] [Commented] (SPARK-31998) Change package references for ArrowBuf

2020-06-16 Thread Kouhei Sutou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17137931#comment-17137931 ] Kouhei Sutou commented on SPARK-31998: -- FYI: The corresponding change in Apache Arr

[jira] [Updated] (SPARK-31962) Provide option to load files after a specified date when reading from a folder path

2020-06-16 Thread Christopher Highman (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Highman updated SPARK-31962: Description: When using structured streaming or just loading from a file data sour

[jira] [Updated] (SPARK-31962) Provide option to load files after a specified date when reading from a folder path

2020-06-16 Thread Christopher Highman (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Highman updated SPARK-31962: Description: When using structured streaming or just loading from a file data sour

[jira] [Updated] (SPARK-31962) Provide option to load files after a specified date when reading from a folder path

2020-06-16 Thread Christopher Highman (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Highman updated SPARK-31962: Description: When using structured streaming or just loading from a file data sour

[jira] [Created] (SPARK-32008) 3.0.0 release build fails

2020-06-16 Thread Philipp Dallig (Jira)
Philipp Dallig created SPARK-32008: -- Summary: 3.0.0 release build fails Key: SPARK-32008 URL: https://issues.apache.org/jira/browse/SPARK-32008 Project: Spark Issue Type: Bug Compo

[jira] [Updated] (SPARK-32007) Spark Driver Supervise does not work reliably

2020-06-16 Thread Suraj Sharma (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suraj Sharma updated SPARK-32007: - Environment: |Java Version|1.8.0_121 (Oracle Corporation)| |Java Home|/usr/java/jdk1.8.0_121/jre

[jira] [Created] (SPARK-32007) Spark Driver Supervise does not work reliably

2020-06-16 Thread Suraj Sharma (Jira)
Suraj Sharma created SPARK-32007: Summary: Spark Driver Supervise does not work reliably Key: SPARK-32007 URL: https://issues.apache.org/jira/browse/SPARK-32007 Project: Spark Issue Type: Que

[jira] [Assigned] (SPARK-32006) Create date/timestamp formatters once before collect in `hiveResultString()`

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32006: Assignee: Apache Spark > Create date/timestamp formatters once before collect in `hiveRes

[jira] [Commented] (SPARK-32006) Create date/timestamp formatters once before collect in `hiveResultString()`

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17137852#comment-17137852 ] Apache Spark commented on SPARK-32006: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-32006) Create date/timestamp formatters once before collect in `hiveResultString()`

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32006: Assignee: (was: Apache Spark) > Create date/timestamp formatters once before collect

[jira] [Created] (SPARK-32005) Add aggregate functions for computing percentiles on weighted data

2020-06-16 Thread Devesh Parekh (Jira)
Devesh Parekh created SPARK-32005: - Summary: Add aggregate functions for computing percentiles on weighted data Key: SPARK-32005 URL: https://issues.apache.org/jira/browse/SPARK-32005 Project: Spark

[jira] [Created] (SPARK-32006) Create date/timestamp formatters once before collect in `hiveResultString()`

2020-06-16 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-32006: -- Summary: Create date/timestamp formatters once before collect in `hiveResultString()` Key: SPARK-32006 URL: https://issues.apache.org/jira/browse/SPARK-32006 Project: Spa

[jira] [Created] (SPARK-32004) Drop references to slave

2020-06-16 Thread Holden Karau (Jira)
Holden Karau created SPARK-32004: Summary: Drop references to slave Key: SPARK-32004 URL: https://issues.apache.org/jira/browse/SPARK-32004 Project: Spark Issue Type: Improvement Co

[jira] [Closed] (SPARK-31099) Create migration script for metastore_db

2020-06-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-31099. - > Create migration script for metastore_db > > >

[jira] [Resolved] (SPARK-31929) local cache size exceeding "spark.history.store.maxDiskUsage" triggered "java.io.IOException" in history server on Windows

2020-06-16 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-31929. -- Fix Version/s: 3.1.0 Assignee: zhli Resolution: Fixed Resolved by https://gith

[jira] [Updated] (SPARK-32003) Shuffle files for lost executor are not unregistered if fetch failure occurs after executor is lost

2020-06-16 Thread Wing Yew Poon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wing Yew Poon updated SPARK-32003: -- Affects Version/s: 3.0.0 > Shuffle files for lost executor are not unregistered if fetch failu

[jira] [Commented] (SPARK-32003) Shuffle files for lost executor are not unregistered if fetch failure occurs after executor is lost

2020-06-16 Thread Wing Yew Poon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136792#comment-17136792 ] Wing Yew Poon commented on SPARK-32003: --- I will open a PR soon with a solution. >

[jira] [Assigned] (SPARK-31996) Specify the version of ChromeDriver and RemoteWebDriver which can work with guava 14.0.1

2020-06-16 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta reassigned SPARK-31996: -- Assignee: (was: Kousuke Saruta) > Specify the version of ChromeDriver and RemoteW

[jira] [Comment Edited] (SPARK-26833) Kubernetes RBAC documentation is unclear on exact RBAC requirements

2020-06-16 Thread Sonia Dalwani (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136773#comment-17136773 ] Sonia Dalwani edited comment on SPARK-26833 at 6/16/20, 4:08 PM: -

[jira] [Commented] (SPARK-26833) Kubernetes RBAC documentation is unclear on exact RBAC requirements

2020-06-16 Thread Sonia Dalwani (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136773#comment-17136773 ] Sonia Dalwani commented on SPARK-26833: --- Documentation cab be changed like the fol

[jira] [Created] (SPARK-32003) Shuffle files for lost executor are not unregistered if fetch failure occurs after executor is lost

2020-06-16 Thread Wing Yew Poon (Jira)
Wing Yew Poon created SPARK-32003: - Summary: Shuffle files for lost executor are not unregistered if fetch failure occurs after executor is lost Key: SPARK-32003 URL: https://issues.apache.org/jira/browse/SPARK-32

[jira] [Commented] (SPARK-30876) Optimizer cannot infer from inferred constraints with join

2020-06-16 Thread Navin Viswanath (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136753#comment-17136753 ] Navin Viswanath commented on SPARK-30876: - Hi [~yumwang] I'd be happy to take a

[jira] [Commented] (SPARK-31710) Fail casting numeric to timestamp by default

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136674#comment-17136674 ] Apache Spark commented on SPARK-31710: -- User 'MaxGekk' has created a pull request f

[jira] [Commented] (SPARK-31710) Fail casting numeric to timestamp by default

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136672#comment-17136672 ] Apache Spark commented on SPARK-31710: -- User 'MaxGekk' has created a pull request f

[jira] [Updated] (SPARK-32002) spark error while select nest data

2020-06-16 Thread Yiqun Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Zhang updated SPARK-32002: Description: nest-data.json {code:java} {"a": [{"b": [{"c": [1,2]}]}]} {"a": [{"b": [{"c": [1]}, {

[jira] [Updated] (SPARK-32002) spark error while select nest data

2020-06-16 Thread Yiqun Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Zhang updated SPARK-32002: Description: nest-data.json {code:java} {"a": [{"b": [{"c": [1,2]}]}]} {"a": [{"b": [{"c": [1]}, {

[jira] [Updated] (SPARK-32002) spark error while select nest data

2020-06-16 Thread Yiqun Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Zhang updated SPARK-32002: Description: nest-data.json {code:java} {"a": [{"b": [{"c": [1,2]}]}]} {"a": [{"b": [{"c": [1]}, {

[jira] [Created] (SPARK-32002) spark error while select nest data

2020-06-16 Thread Yiqun Zhang (Jira)
Yiqun Zhang created SPARK-32002: --- Summary: spark error while select nest data Key: SPARK-32002 URL: https://issues.apache.org/jira/browse/SPARK-32002 Project: Spark Issue Type: Bug Co

[jira] [Assigned] (SPARK-31984) Make micros rebasing functions via local timestamps pure

2020-06-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31984: --- Assignee: Maxim Gekk > Make micros rebasing functions via local timestamps pure > -

[jira] [Resolved] (SPARK-31984) Make micros rebasing functions via local timestamps pure

2020-06-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31984. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28824 [https://gith

[jira] [Updated] (SPARK-31981) Keep TimestampType when taking an average of a Timestamp

2020-06-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-31981: Fix Version/s: (was: 3.1.0) > Keep TimestampType when taking an average of a Timestamp > -

[jira] [Updated] (SPARK-32001) Create Kerberos authentication provider API in JDBC connector

2020-06-16 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi updated SPARK-32001: -- Description: JDBC connection providers must be loaded independently just like delegation token

[jira] [Commented] (SPARK-31857) Support Azure SQLDB Kerberos login in JDBC connector

2020-06-16 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136579#comment-17136579 ] Gabor Somogyi commented on SPARK-31857: --- My plan is to support external providers

[jira] [Commented] (SPARK-31815) Support Hive Kerberos login in JDBC connector

2020-06-16 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136580#comment-17136580 ] Gabor Somogyi commented on SPARK-31815: --- My plan is to support external providers

[jira] [Commented] (SPARK-31884) Support MongoDB Kerberos login in JDBC connector

2020-06-16 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136578#comment-17136578 ] Gabor Somogyi commented on SPARK-31884: --- My plan is to support external providers

[jira] [Created] (SPARK-32001) Create Kerberos authentication provider API in JDBC connector

2020-06-16 Thread Gabor Somogyi (Jira)
Gabor Somogyi created SPARK-32001: - Summary: Create Kerberos authentication provider API in JDBC connector Key: SPARK-32001 URL: https://issues.apache.org/jira/browse/SPARK-32001 Project: Spark

[jira] [Commented] (SPARK-31955) Beeline discard the last line of the sql file when submited to thriftserver via beeline

2020-06-16 Thread Lin Gang Deng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136550#comment-17136550 ] Lin Gang Deng commented on SPARK-31955: --- [~dengzh], the version of beeline in spar

[jira] [Commented] (SPARK-31955) Beeline discard the last line of the sql file when submited to thriftserver via beeline

2020-06-16 Thread Zhihua Deng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136535#comment-17136535 ] Zhihua Deng commented on SPARK-31955: - The issue seems have been fixed in   [HIVE-10

[jira] [Commented] (SPARK-31955) Beeline discard the last line of the sql file when submited to thriftserver via beeline

2020-06-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136527#comment-17136527 ] Yuming Wang commented on SPARK-31955: - The issue fixed by upgrading the beeline to 2

[jira] [Reopened] (SPARK-31955) Beeline discard the last line of the sql file when submited to thriftserver via beeline

2020-06-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reopened SPARK-31955: - > Beeline discard the last line of the sql file when submited to thriftserver > via beeline >

[jira] [Resolved] (SPARK-31955) Beeline discard the last line of the sql file when submited to thriftserver via beeline

2020-06-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-31955. - Resolution: Fixed > Beeline discard the last line of the sql file when submited to thriftserver

[jira] [Updated] (SPARK-31955) Beeline discard the last line of the sql file when submited to thriftserver via beeline

2020-06-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-31955: Fix Version/s: 3.0.0 > Beeline discard the last line of the sql file when submited to thriftserve

[jira] [Assigned] (SPARK-31997) Should drop test_udtf table when SingleSessionSuite completed

2020-06-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-31997: Assignee: Yang Jie > Should drop test_udtf table when SingleSessionSuite completed >

[jira] [Resolved] (SPARK-31997) Should drop test_udtf table when SingleSessionSuite completed

2020-06-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31997. -- Fix Version/s: 3.1.0 2.4.7 3.0.1 Resolution: Fixed

[jira] [Commented] (SPARK-31962) Provide option to load files after a specified date when reading from a folder path

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136495#comment-17136495 ] Apache Spark commented on SPARK-31962: -- User 'cchighman' has created a pull request

[jira] [Commented] (SPARK-31962) Provide option to load files after a specified date when reading from a folder path

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136494#comment-17136494 ] Apache Spark commented on SPARK-31962: -- User 'cchighman' has created a pull request

[jira] [Assigned] (SPARK-31962) Provide option to load files after a specified date when reading from a folder path

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31962: Assignee: (was: Apache Spark) > Provide option to load files after a specified date w

[jira] [Assigned] (SPARK-31962) Provide option to load files after a specified date when reading from a folder path

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31962: Assignee: Apache Spark > Provide option to load files after a specified date when reading

[jira] [Updated] (SPARK-31962) Provide option to load files after a specified date when reading from a folder path

2020-06-16 Thread Christopher Highman (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Highman updated SPARK-31962: Description: When using structured streaming or just loading from a file data sour

[jira] [Assigned] (SPARK-31999) Add refresh function command

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31999: Assignee: Apache Spark > Add refresh function command > > >

[jira] [Assigned] (SPARK-31999) Add refresh function command

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31999: Assignee: (was: Apache Spark) > Add refresh function command > --

[jira] [Updated] (SPARK-31962) Provide option to load files after a specified date when reading from a folder path

2020-06-16 Thread Christopher Highman (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Highman updated SPARK-31962: Description: When using structured streaming with a FileDataSource, I've encounter

[jira] [Updated] (SPARK-31962) Provide option to load files after a specified date when reading from a folder path

2020-06-16 Thread Christopher Highman (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Highman updated SPARK-31962: Component/s: (was: Structured Streaming) > Provide option to load files after

[jira] [Commented] (SPARK-31999) Add refresh function command

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136458#comment-17136458 ] Apache Spark commented on SPARK-31999: -- User 'ulysses-you' has created a pull reque

[jira] [Commented] (SPARK-31999) Add refresh function command

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136460#comment-17136460 ] Apache Spark commented on SPARK-31999: -- User 'ulysses-you' has created a pull reque

[jira] [Assigned] (SPARK-32000) Fix the flaky testcase for partially launched task in barrier-mode.

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32000: Assignee: Apache Spark (was: Kousuke Saruta) > Fix the flaky testcase for partially laun

[jira] [Commented] (SPARK-32000) Fix the flaky testcase for partially launched task in barrier-mode.

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136452#comment-17136452 ] Apache Spark commented on SPARK-32000: -- User 'sarutak' has created a pull request f

[jira] [Assigned] (SPARK-32000) Fix the flaky testcase for partially launched task in barrier-mode.

2020-06-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32000: Assignee: Kousuke Saruta (was: Apache Spark) > Fix the flaky testcase for partially laun

  1   2   >