[jira] [Created] (SPARK-33429) Support drop column in spark also like in postgresql

2020-11-11 Thread jobit mathew (Jira)
jobit mathew created SPARK-33429: Summary: Support drop column in spark also like in postgresql Key: SPARK-33429 URL: https://issues.apache.org/jira/browse/SPARK-33429 Project: Spark Issue Ty

[jira] [Assigned] (SPARK-33261) Allow people to extend the pod feature steps

2020-11-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33261: - Assignee: Holden Karau > Allow people to extend the pod feature steps > ---

[jira] [Assigned] (SPARK-33209) Clean up unit test file UnsupportedOperationsSuite.scala

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33209: Assignee: (was: Apache Spark) > Clean up unit test file UnsupportedOperationsSuite.sc

[jira] [Assigned] (SPARK-33209) Clean up unit test file UnsupportedOperationsSuite.scala

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33209: Assignee: Apache Spark > Clean up unit test file UnsupportedOperationsSuite.scala > -

[jira] [Commented] (SPARK-33209) Clean up unit test file UnsupportedOperationsSuite.scala

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17230388#comment-17230388 ] Apache Spark commented on SPARK-33209: -- User 'c21' has created a pull request for t

[jira] [Updated] (SPARK-33418) TaskSchedulerImpl: Check pending tasks in advance when resource offers

2020-11-11 Thread dingbei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dingbei updated SPARK-33418: Description: It begins with the needs to  start a lot of spark streaming receivers(custom receivers) .  *

[jira] [Updated] (SPARK-33418) TaskSchedulerImpl: Check pending tasks in advance when resource offers

2020-11-11 Thread dingbei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dingbei updated SPARK-33418: Description: It begins with the needs to  start a lot of spark streaming receivers(custom receivers) .  *

[jira] [Updated] (SPARK-33418) TaskSchedulerImpl: Check pending tasks in advance when resource offers

2020-11-11 Thread dingbei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dingbei updated SPARK-33418: Priority: Minor (was: Major) > TaskSchedulerImpl: Check pending tasks in advance when resource offers > -

[jira] [Updated] (SPARK-33418) TaskSchedulerImpl: Check pending tasks in advance when resource offers

2020-11-11 Thread dingbei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dingbei updated SPARK-33418: Attachment: (was: 6_2.png) > TaskSchedulerImpl: Check pending tasks in advance when resource offers >

[jira] [Updated] (SPARK-33418) TaskSchedulerImpl: Check pending tasks in advance when resource offers

2020-11-11 Thread dingbei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dingbei updated SPARK-33418: Attachment: (was: 6_1.png) > TaskSchedulerImpl: Check pending tasks in advance when resource offers >

[jira] [Updated] (SPARK-33408) Use R 3.6.3 in K8s R image

2020-11-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33408: - Summary: Use R 3.6.3 in K8s R image (was: Use R 3.6 in K8s R image) > Use R 3.6.3 in K8s R imag

[jira] [Resolved] (SPARK-32354) Fix & re-enable failing R K8s tests

2020-11-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32354. -- Resolution: Fixed Fixed in https://github.com/apache/spark/pull/30130 > Fix & re-enable faili

[jira] [Updated] (SPARK-33418) TaskSchedulerImpl: Check pending tasks in advance when resource offers

2020-11-11 Thread dingbei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dingbei updated SPARK-33418: Attachment: (was: 4_1.png) > TaskSchedulerImpl: Check pending tasks in advance when resource offers >

[jira] [Updated] (SPARK-33418) TaskSchedulerImpl: Check pending tasks in advance when resource offers

2020-11-11 Thread dingbei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dingbei updated SPARK-33418: Attachment: (was: 4_2.png) > TaskSchedulerImpl: Check pending tasks in advance when resource offers >

[jira] [Updated] (SPARK-33418) TaskSchedulerImpl: Check pending tasks in advance when resource offers

2020-11-11 Thread dingbei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dingbei updated SPARK-33418: Attachment: (was: 5.png) > TaskSchedulerImpl: Check pending tasks in advance when resource offers > --

[jira] [Updated] (SPARK-33408) Use R 3.6 in K8s R image

2020-11-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33408: - Summary: Use R 3.6 in K8s R image (was: Use R 4.0 in K8s R image) > Use R 3.6 in K8s R image >

[jira] [Updated] (SPARK-33418) TaskSchedulerImpl: Check pending tasks in advance when resource offers

2020-11-11 Thread dingbei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dingbei updated SPARK-33418: Description: It begins with the needs to  start a lot of spark streaming receivers(custom receivers) .  *

[jira] [Resolved] (SPARK-33408) Use R 4.0 in K8s R image

2020-11-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33408. -- Fix Version/s: 3.0.2 3.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Updated] (SPARK-33418) TaskSchedulerImpl: Check pending tasks in advance when resource offers

2020-11-11 Thread dingbei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dingbei updated SPARK-33418: Description: It begins with the needs to  start a lot of spark streaming receivers(custom receivers) .  *

[jira] [Commented] (SPARK-33253) Migration to NumPy documentation style in Streaming (pyspark.streaming.*)

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17230370#comment-17230370 ] Apache Spark commented on SPARK-33253: -- User 'itholic' has created a pull request f

[jira] [Commented] (SPARK-33253) Migration to NumPy documentation style in Streaming (pyspark.streaming.*)

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17230368#comment-17230368 ] Apache Spark commented on SPARK-33253: -- User 'itholic' has created a pull request f

[jira] [Assigned] (SPARK-33253) Migration to NumPy documentation style in Streaming (pyspark.streaming.*)

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33253: Assignee: (was: Apache Spark) > Migration to NumPy documentation style in Streaming (

[jira] [Assigned] (SPARK-33253) Migration to NumPy documentation style in Streaming (pyspark.streaming.*)

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33253: Assignee: Apache Spark > Migration to NumPy documentation style in Streaming (pyspark.str

[jira] [Assigned] (SPARK-32085) Migrate to NumPy documentation style

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32085: Assignee: Apache Spark > Migrate to NumPy documentation style > -

[jira] [Commented] (SPARK-32085) Migrate to NumPy documentation style

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17230367#comment-17230367 ] Apache Spark commented on SPARK-32085: -- User 'itholic' has created a pull request f

[jira] [Assigned] (SPARK-32085) Migrate to NumPy documentation style

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32085: Assignee: (was: Apache Spark) > Migrate to NumPy documentation style > --

[jira] [Updated] (SPARK-33428) conv UDF returns incorrect value

2020-11-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33428: Summary: conv UDF returns incorrect value (was: conv returns incorrect value) > conv UDF returns

[jira] [Created] (SPARK-33428) conv returns incorrect value

2020-11-11 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33428: --- Summary: conv returns incorrect value Key: SPARK-33428 URL: https://issues.apache.org/jira/browse/SPARK-33428 Project: Spark Issue Type: Bug Componen

[jira] [Commented] (SPARK-32691) Update commons-crypto to v1.1.0

2020-11-11 Thread huangtianhua (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17230325#comment-17230325 ] huangtianhua commented on SPARK-32691: -- The ARM Jenkins CI still fail after this me

[jira] [Commented] (SPARK-30294) Read-only state store unnecessarily creates and deletes the temp file for delta file every batch

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17230321#comment-17230321 ] Apache Spark commented on SPARK-30294: -- User 'xuanyuanking' has created a pull requ

[jira] [Updated] (SPARK-33152) Constraint Propagation code causes OOM issues or increasing compilation time to hours

2020-11-11 Thread Asif (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif updated SPARK-33152: - Affects Version/s: 3.0.1 > Constraint Propagation code causes OOM issues or increasing compilation time > to ho

[jira] [Updated] (SPARK-33404) "date_trunc" expression returns incorrect results

2020-11-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33404: -- Fix Version/s: 3.0.2 > "date_trunc" expression returns incorrect results > ---

[jira] [Assigned] (SPARK-33402) Jobs launched in same second have duplicate MapReduce JobIDs

2020-11-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33402: - Assignee: Steve Loughran > Jobs launched in same second have duplicate MapReduce JobIDs

[jira] [Resolved] (SPARK-33402) Jobs launched in same second have duplicate MapReduce JobIDs

2020-11-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33402. --- Fix Version/s: 3.0.2 3.1.0 Resolution: Fixed Issue resolved by pul

[jira] [Assigned] (SPARK-33427) Interpreted subexpression elimination

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33427: Assignee: Apache Spark (was: L. C. Hsieh) > Interpreted subexpression elimination >

[jira] [Commented] (SPARK-33427) Interpreted subexpression elimination

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17230257#comment-17230257 ] Apache Spark commented on SPARK-33427: -- User 'viirya' has created a pull request fo

[jira] [Assigned] (SPARK-33427) Interpreted subexpression elimination

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33427: Assignee: L. C. Hsieh (was: Apache Spark) > Interpreted subexpression elimination >

[jira] [Created] (SPARK-33427) Interpreted subexpression elimination

2020-11-11 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-33427: --- Summary: Interpreted subexpression elimination Key: SPARK-33427 URL: https://issues.apache.org/jira/browse/SPARK-33427 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-33426) Unify Hive SHOW TABLES tests

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17230186#comment-17230186 ] Apache Spark commented on SPARK-33426: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-33426) Unify Hive SHOW TABLES tests

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33426: Assignee: (was: Apache Spark) > Unify Hive SHOW TABLES tests > --

[jira] [Assigned] (SPARK-33426) Unify Hive SHOW TABLES tests

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33426: Assignee: Apache Spark > Unify Hive SHOW TABLES tests > > >

[jira] [Commented] (SPARK-33426) Unify Hive SHOW TABLES tests

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17230184#comment-17230184 ] Apache Spark commented on SPARK-33426: -- User 'MaxGekk' has created a pull request f

[jira] [Created] (SPARK-33426) Unify Hive SHOW TABLES tests

2020-11-11 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33426: -- Summary: Unify Hive SHOW TABLES tests Key: SPARK-33426 URL: https://issues.apache.org/jira/browse/SPARK-33426 Project: Spark Issue Type: Sub-task Compo

[jira] [Commented] (SPARK-33404) "date_trunc" expression returns incorrect results

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17230152#comment-17230152 ] Apache Spark commented on SPARK-33404: -- User 'utkarsh39' has created a pull request

[jira] [Commented] (SPARK-33404) "date_trunc" expression returns incorrect results

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17230150#comment-17230150 ] Apache Spark commented on SPARK-33404: -- User 'utkarsh39' has created a pull request

[jira] [Commented] (SPARK-33404) "date_trunc" expression returns incorrect results

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17230077#comment-17230077 ] Apache Spark commented on SPARK-33404: -- User 'MaxGekk' has created a pull request f

[jira] [Commented] (SPARK-33425) Credentials are not passed in the `doFetchFile` when running spark-submit with https url

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17230036#comment-17230036 ] Apache Spark commented on SPARK-33425: -- User 'pprzetacznik' has created a pull requ

[jira] [Assigned] (SPARK-33425) Credentials are not passed in the `doFetchFile` when running spark-submit with https url

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33425: Assignee: Apache Spark > Credentials are not passed in the `doFetchFile` when running spa

[jira] [Assigned] (SPARK-33425) Credentials are not passed in the `doFetchFile` when running spark-submit with https url

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33425: Assignee: (was: Apache Spark) > Credentials are not passed in the `doFetchFile` when

[jira] [Created] (SPARK-33425) Credentials are not passed in the `doFetchFile` when running spark-submit with https url

2020-11-11 Thread Piotr Przetacznik (Jira)
Piotr Przetacznik created SPARK-33425: - Summary: Credentials are not passed in the `doFetchFile` when running spark-submit with https url Key: SPARK-33425 URL: https://issues.apache.org/jira/browse/SPARK-33425

[jira] [Updated] (SPARK-33402) Jobs launched in same second have duplicate JobIDs

2020-11-11 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-33402: --- Description: Spark uses the current timestamp to generate a MapReduce JobID. If > 1 job atte

[jira] [Updated] (SPARK-33402) Jobs launched in same second have duplicate MapReduce JobIDs

2020-11-11 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-33402: --- Summary: Jobs launched in same second have duplicate MapReduce JobIDs (was: Jobs launched i

[jira] [Resolved] (SPARK-33415) Column.__repr__ shouldn't encode JVM response

2020-11-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33415. -- Target Version/s: 3.1.0 Assignee: Maciej Szymkiewicz Resolution: Fixed F

[jira] [Updated] (SPARK-33424) Doubts about the use of the "DiskBlockObjectWriter#revertPartialWritesAndClose" method in Spark Code

2020-11-11 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-33424: - Description: Although there are some similar discussions in SPARK-17562, but I still have some question

[jira] [Updated] (SPARK-33424) Doubts about the use of the "DiskBlockObjectWriter#revertPartialWritesAndClose" method in Spark Code

2020-11-11 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-33424: - Description: Although there are some similar discussions in SPARK-17562, but I still have some question

[jira] [Updated] (SPARK-33424) Doubts about the use of the "DiskBlockObjectWriter#revertPartialWritesAndClose" method in Spark Code

2020-11-11 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-33424: - Description: Although there are some similar discussions in SPARK-17562, but I still have some question

[jira] [Updated] (SPARK-33402) Jobs launched in same second have duplicate JobIDs

2020-11-11 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-33402: --- Summary: Jobs launched in same second have duplicate JobIDs (was: SparkHadoopWriter to pass

[jira] [Updated] (SPARK-33424) Doubts about the use of the "DiskBlockObjectWriter#revertPartialWritesAndClose" method in Spark Code

2020-11-11 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-33424: - Description: Although there are some similar discussions in SPARK-17562, but I still have some question

[jira] [Updated] (SPARK-33424) Doubts about the use of the "DiskBlockObjectWriter#revertPartialWritesAndClose" method in Spark Code

2020-11-11 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-33424: - Description: Although there are some similar discussions in SPARK-17562, but I still have some question

[jira] [Created] (SPARK-33424) Doubts about the use of the "DiskBlockObjectWriter#revertPartialWritesAndClose" method in Spark Code

2020-11-11 Thread Yang Jie (Jira)
Yang Jie created SPARK-33424: Summary: Doubts about the use of the "DiskBlockObjectWriter#revertPartialWritesAndClose" method in Spark Code Key: SPARK-33424 URL: https://issues.apache.org/jira/browse/SPARK-33424

[jira] [Updated] (SPARK-33424) Doubts about the use of the "DiskBlockObjectWriter#revertPartialWritesAndClose" method in Spark Code

2020-11-11 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-33424: - Description: Although there are some similar discussions in SPARK-17562, but I still have some question

[jira] [Updated] (SPARK-33424) Doubts about the use of the "DiskBlockObjectWriter#revertPartialWritesAndClose" method in Spark Code

2020-11-11 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-33424: - Description: Although there are some similar discussions in SPARK-17562, but I still have some question

[jira] [Updated] (SPARK-33402) SparkHadoopWriter to pass job UUID down in spark.sql.sources.writeJobUUID

2020-11-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33402: -- Issue Type: Bug (was: Improvement) > SparkHadoopWriter to pass job UUID down in spark.sql.sou

[jira] [Updated] (SPARK-33402) SparkHadoopWriter to pass job UUID down in spark.sql.sources.writeJobUUID

2020-11-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33402: -- Affects Version/s: 3.0.1 > SparkHadoopWriter to pass job UUID down in spark.sql.sources.writeJ

[jira] [Resolved] (SPARK-33412) OverwriteByExpression should resolve its delete condition based on the table relation not the input query

2020-11-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33412. - Fix Version/s: 3.1.0 3.0.2 Resolution: Fixed > OverwriteByExpression s

[jira] [Assigned] (SPARK-33287) Expose state custom metrics information on SS UI

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33287: Assignee: Apache Spark > Expose state custom metrics information on SS UI > -

[jira] [Assigned] (SPARK-33287) Expose state custom metrics information on SS UI

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33287: Assignee: (was: Apache Spark) > Expose state custom metrics information on SS UI > --

[jira] [Commented] (SPARK-33287) Expose state custom metrics information on SS UI

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17229966#comment-17229966 ] Apache Spark commented on SPARK-33287: -- User 'gaborgsomogyi' has created a pull req

[jira] [Commented] (SPARK-33287) Expose state custom metrics information on SS UI

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17229967#comment-17229967 ] Apache Spark commented on SPARK-33287: -- User 'gaborgsomogyi' has created a pull req

[jira] [Created] (SPARK-33423) DoubleParam can't parse a JSON number without decimal places

2020-11-11 Thread Alexander Bouriakov (Jira)
Alexander Bouriakov created SPARK-33423: --- Summary: DoubleParam can't parse a JSON number without decimal places Key: SPARK-33423 URL: https://issues.apache.org/jira/browse/SPARK-33423 Project: S

[jira] [Issue Comment Deleted] (SPARK-33411) Cardinality estimation of union, range and sort logical operators

2020-11-11 Thread Ayushi Agarwal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayushi Agarwal updated SPARK-33411: --- Comment: was deleted (was: I am working on this) > Cardinality estimation of union, range a

[jira] [Commented] (SPARK-33411) Cardinality estimation of union, range and sort logical operators

2020-11-11 Thread Ayushi Agarwal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17229915#comment-17229915 ] Ayushi Agarwal commented on SPARK-33411: I am working on this > Cardinality est

[jira] [Commented] (SPARK-33422) Incomplete menu item display in documention

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17229914#comment-17229914 ] Apache Spark commented on SPARK-33422: -- User 'liucht-inspur' has created a pull req

[jira] [Commented] (SPARK-33411) Cardinality estimation of union, range and sort logical operators

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17229913#comment-17229913 ] Apache Spark commented on SPARK-33411: -- User 'ayushi-agarwal' has created a pull re

[jira] [Assigned] (SPARK-33422) Incomplete menu item display in documention

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33422: Assignee: Apache Spark > Incomplete menu item display in documention > --

[jira] [Assigned] (SPARK-33411) Cardinality estimation of union, range and sort logical operators

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33411: Assignee: Apache Spark > Cardinality estimation of union, range and sort logical operator

[jira] [Commented] (SPARK-33422) Incomplete menu item display in documention

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17229911#comment-17229911 ] Apache Spark commented on SPARK-33422: -- User 'liucht-inspur' has created a pull req

[jira] [Commented] (SPARK-33411) Cardinality estimation of union, range and sort logical operators

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17229912#comment-17229912 ] Apache Spark commented on SPARK-33411: -- User 'ayushi-agarwal' has created a pull re

[jira] [Assigned] (SPARK-33411) Cardinality estimation of union, range and sort logical operators

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33411: Assignee: (was: Apache Spark) > Cardinality estimation of union, range and sort logic

[jira] [Assigned] (SPARK-33422) Incomplete menu item display in documention

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33422: Assignee: (was: Apache Spark) > Incomplete menu item display in documention > ---

[jira] [Assigned] (SPARK-32240) UDF parse_url with a URL that contains pipe(|) will give incorrect result

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32240: Assignee: Apache Spark > UDF parse_url with a URL that contains pipe(|) will give incorre

[jira] [Commented] (SPARK-32240) UDF parse_url with a URL that contains pipe(|) will give incorrect result

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17229907#comment-17229907 ] Apache Spark commented on SPARK-32240: -- User 'ulysses-you' has created a pull reque

[jira] [Updated] (SPARK-33422) Incomplete menu item display in documention

2020-11-11 Thread liucht-inspur (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liucht-inspur updated SPARK-33422: -- Description: The bottom menu item cannot be displayed when the left menu tree is long   wa

[jira] [Updated] (SPARK-33422) Incomplete menu item display in documention

2020-11-11 Thread liucht-inspur (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liucht-inspur updated SPARK-33422: -- Attachment: left-menu.jpg > Incomplete menu item display in documention >

[jira] [Commented] (SPARK-32240) UDF parse_url with a URL that contains pipe(|) will give incorrect result

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17229908#comment-17229908 ] Apache Spark commented on SPARK-32240: -- User 'ulysses-you' has created a pull reque

[jira] [Assigned] (SPARK-32240) UDF parse_url with a URL that contains pipe(|) will give incorrect result

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32240: Assignee: (was: Apache Spark) > UDF parse_url with a URL that contains pipe(|) will g

[jira] [Created] (SPARK-33422) Incomplete menu item display in documention

2020-11-11 Thread liucht-inspur (Jira)
liucht-inspur created SPARK-33422: - Summary: Incomplete menu item display in documention Key: SPARK-33422 URL: https://issues.apache.org/jira/browse/SPARK-33422 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-33419) Unexpected behavior when using SET commands before a query in SparkSession.sql

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17229892#comment-17229892 ] Apache Spark commented on SPARK-33419: -- User 'yaooqinn' has created a pull request

[jira] [Assigned] (SPARK-33419) Unexpected behavior when using SET commands before a query in SparkSession.sql

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33419: Assignee: (was: Apache Spark) > Unexpected behavior when using SET commands before a

[jira] [Assigned] (SPARK-33419) Unexpected behavior when using SET commands before a query in SparkSession.sql

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33419: Assignee: Apache Spark > Unexpected behavior when using SET commands before a query in Sp

[jira] [Commented] (SPARK-33383) Improve performance of Column.isin Expression

2020-11-11 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-33383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17229870#comment-17229870 ] Felix Wollschläger commented on SPARK-33383: This makes perfectly sense. Tha

[jira] [Commented] (SPARK-33128) mismatched input since Spark 3.0

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17229869#comment-17229869 ] Apache Spark commented on SPARK-33128: -- User 'wangyum' has created a pull request f

[jira] [Assigned] (SPARK-33128) mismatched input since Spark 3.0

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33128: Assignee: Apache Spark > mismatched input since Spark 3.0 > -

[jira] [Assigned] (SPARK-33128) mismatched input since Spark 3.0

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33128: Assignee: (was: Apache Spark) > mismatched input since Spark 3.0 > --

[jira] [Commented] (SPARK-33128) mismatched input since Spark 3.0

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17229868#comment-17229868 ] Apache Spark commented on SPARK-33128: -- User 'wangyum' has created a pull request f

[jira] [Resolved] (SPARK-32512) Add basic partition command for datasourcev2

2020-11-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32512. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29339 [https://gith

[jira] [Assigned] (SPARK-32512) Add basic partition command for datasourcev2

2020-11-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32512: --- Assignee: Jackey Lee > Add basic partition command for datasourcev2 > -

[jira] [Commented] (SPARK-33147) Avoid distribute user jar from driver in yarn client mode

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17229863#comment-17229863 ] Apache Spark commented on SPARK-33147: -- User 'hzyangkai' has created a pull request

[jira] [Assigned] (SPARK-33420) BroadCastJoin failure when keys on join side has cast from DateTyte to String

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33420: Assignee: Apache Spark > BroadCastJoin failure when keys on join side has cast from DateT

[jira] [Assigned] (SPARK-33420) BroadCastJoin failure when keys on join side has cast from DateTyte to String

2020-11-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33420: Assignee: (was: Apache Spark) > BroadCastJoin failure when keys on join side has cast

  1   2   >