[jira] [Created] (SPARK-34134) LDAP authentication of spark thrift server support user id mapping

2021-01-15 Thread Timothy Zhang (Jira)
Timothy Zhang created SPARK-34134: - Summary: LDAP authentication of spark thrift server support user id mapping Key: SPARK-34134 URL: https://issues.apache.org/jira/browse/SPARK-34134 Project: Spark

[jira] [Commented] (SPARK-34119) Keep necessary stats after partition pruning

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266522#comment-17266522 ] Apache Spark commented on SPARK-34119: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34119) Keep necessary stats after partition pruning

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34119: Assignee: Apache Spark > Keep necessary stats after partition pruning >

[jira] [Assigned] (SPARK-34119) Keep necessary stats after partition pruning

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34119: Assignee: (was: Apache Spark) > Keep necessary stats after partition pruning >

[jira] [Commented] (SPARK-34119) Keep necessary stats after partition pruning

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266521#comment-17266521 ] Apache Spark commented on SPARK-34119: -- User 'wangyum' has created a pull request for this issue:

[jira] [Issue Comment Deleted] (SPARK-34119) Keep necessary stats after partition pruning

2021-01-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-34119: Comment: was deleted (was: [~leoluan] Would you like work on this?) > Keep necessary stats after

[jira] [Resolved] (SPARK-34110) Upgrade ZooKeeper to 3.6.2

2021-01-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34110. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31177

[jira] [Assigned] (SPARK-34110) Upgrade ZooKeeper to 3.6.2

2021-01-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34110: - Assignee: Yuming Wang > Upgrade ZooKeeper to 3.6.2 > -- > >

[jira] [Commented] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266500#comment-17266500 ] Apache Spark commented on SPARK-26399: -- User 'AngersZh' has created a pull request for this

[jira] [Resolved] (SPARK-34080) Add UnivariateFeatureSelector to deprecate existing selectors

2021-01-15 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu resolved SPARK-34080. Fix Version/s: 3.2.0 3.1.1 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-34080) Add UnivariateFeatureSelector to deprecate existing selectors

2021-01-15 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu reassigned SPARK-34080: -- Assignee: Huaxin Gao > Add UnivariateFeatureSelector to deprecate existing selectors >

[jira] [Commented] (SPARK-33507) Improve and fix cache behavior in v1 and v2

2021-01-15 Thread Anton Okolnychyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266479#comment-17266479 ] Anton Okolnychyi commented on SPARK-33507: -- [~csun], shall we also handle streaming writes? >

[jira] [Assigned] (SPARK-34132) Update Roxygen version references to 7.1.1

2021-01-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34132: - Assignee: Maciej Szymkiewicz > Update Roxygen version references to 7.1.1 >

[jira] [Resolved] (SPARK-34132) Update Roxygen version references to 7.1.1

2021-01-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34132. --- Fix Version/s: 3.1.1 Resolution: Fixed Issue resolved by pull request 31200

[jira] [Commented] (SPARK-33212) Upgrade to Hadoop 3.2.2 and move to shaded clients for Hadoop 3.x profile

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266422#comment-17266422 ] Apache Spark commented on SPARK-33212: -- User 'sunchao' has created a pull request for this issue:

[jira] [Commented] (SPARK-33212) Upgrade to Hadoop 3.2.2 and move to shaded clients for Hadoop 3.x profile

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266420#comment-17266420 ] Apache Spark commented on SPARK-33212: -- User 'sunchao' has created a pull request for this issue:

[jira] [Commented] (SPARK-34133) [AVRO] Respect case sensitivity when performing Catalyst-to-Avro field matching and enhance error messages

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266394#comment-17266394 ] Apache Spark commented on SPARK-34133: -- User 'xkrogen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34133) [AVRO] Respect case sensitivity when performing Catalyst-to-Avro field matching and enhance error messages

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34133: Assignee: (was: Apache Spark) > [AVRO] Respect case sensitivity when performing

[jira] [Commented] (SPARK-34133) [AVRO] Respect case sensitivity when performing Catalyst-to-Avro field matching and enhance error messages

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266392#comment-17266392 ] Apache Spark commented on SPARK-34133: -- User 'xkrogen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34133) [AVRO] Respect case sensitivity when performing Catalyst-to-Avro field matching and enhance error messages

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34133: Assignee: Apache Spark > [AVRO] Respect case sensitivity when performing

[jira] [Assigned] (SPARK-34064) Broadcast job is not aborted even the SQL statement canceled

2021-01-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34064: - Assignee: (was: Lantao Jin) > Broadcast job is not aborted even the SQL statement

[jira] [Created] (SPARK-34133) [AVRO] Respect case sensitivity when performing Catalyst-to-Avro field matching and enhance error messages

2021-01-15 Thread Erik Krogen (Jira)
Erik Krogen created SPARK-34133: --- Summary: [AVRO] Respect case sensitivity when performing Catalyst-to-Avro field matching and enhance error messages Key: SPARK-34133 URL:

[jira] [Resolved] (SPARK-33212) Move to shaded clients for Hadoop 3.x profile

2021-01-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33212. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30701

[jira] [Updated] (SPARK-33212) Upgrade to Hadoop 3.2.2 and move to shaded clients for Hadoop 3.x profile

2021-01-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33212: -- Summary: Upgrade to Hadoop 3.2.2 and move to shaded clients for Hadoop 3.x profile (was:

[jira] [Assigned] (SPARK-33212) Move to shaded clients for Hadoop 3.x profile

2021-01-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33212: - Assignee: Chao Sun > Move to shaded clients for Hadoop 3.x profile >

[jira] [Updated] (SPARK-33262) Keep pending pods in account while scheduling new pods

2021-01-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33262: -- Parent: SPARK-33005 Issue Type: Sub-task (was: Improvement) > Keep pending pods in

[jira] [Comment Edited] (SPARK-33288) Support k8s cluster manager with stage level scheduling

2021-01-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266350#comment-17266350 ] Dongjoon Hyun edited comment on SPARK-33288 at 1/15/21, 9:17 PM: - Hi,

[jira] [Commented] (SPARK-33288) Support k8s cluster manager with stage level scheduling

2021-01-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266350#comment-17266350 ] Dongjoon Hyun commented on SPARK-33288: --- Hi, [~tgraves]. To give more visibility to this big

[jira] [Updated] (SPARK-33288) Support k8s cluster manager with stage level scheduling

2021-01-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33288: -- Parent: SPARK-33005 Issue Type: Sub-task (was: New Feature) > Support k8s cluster

[jira] [Updated] (SPARK-33668) Fix flaky test "Verify logging configuration is picked from the provided SPARK_CONF_DIR/log4j.properties."

2021-01-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33668: -- Parent: SPARK-33005 Issue Type: Sub-task (was: Bug) > Fix flaky test "Verify logging

[jira] [Updated] (SPARK-33727) `gpg: keyserver receive failed: No name` during K8s IT

2021-01-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33727: -- Parent: SPARK-33005 Issue Type: Sub-task (was: Task) > `gpg: keyserver receive

[jira] [Updated] (SPARK-33732) Kubernetes integration tests doesn't work with Minikube 1.9+

2021-01-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33732: -- Parent: SPARK-33005 Issue Type: Sub-task (was: Bug) > Kubernetes integration tests

[jira] [Updated] (SPARK-33754) Update kubernetes/integration-tests/README.md to follow the default Hadoop profile updated

2021-01-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33754: -- Parent: SPARK-33005 Issue Type: Sub-task (was: Improvement) > Update

[jira] [Updated] (SPARK-33874) Spark may report PodRunning if there is a sidecar that has not exited

2021-01-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33874: -- Parent: SPARK-33005 Issue Type: Sub-task (was: Bug) > Spark may report PodRunning if

[jira] [Updated] (SPARK-33711) Race condition in Spark k8s Pod lifecycle manager that leads to shutdowns

2021-01-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33711: -- Fix Version/s: 3.0.2 > Race condition in Spark k8s Pod lifecycle manager that leads to

[jira] [Updated] (SPARK-33711) Race condition in Spark k8s Pod lifecycle manager that leads to shutdowns

2021-01-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33711: -- Parent: SPARK-33005 Issue Type: Sub-task (was: Bug) > Race condition in Spark k8s

[jira] [Assigned] (SPARK-34132) Update Roxygen version references to 7.1.1

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34132: Assignee: (was: Apache Spark) > Update Roxygen version references to 7.1.1 >

[jira] [Assigned] (SPARK-34132) Update Roxygen version references to 7.1.1

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34132: Assignee: Apache Spark > Update Roxygen version references to 7.1.1 >

[jira] [Commented] (SPARK-34132) Update Roxygen version references to 7.1.1

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266303#comment-17266303 ] Apache Spark commented on SPARK-34132: -- User 'zero323' has created a pull request for this issue:

[jira] [Created] (SPARK-34132) Update Roxygen version references to 7.1.1

2021-01-15 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-34132: -- Summary: Update Roxygen version references to 7.1.1 Key: SPARK-34132 URL: https://issues.apache.org/jira/browse/SPARK-34132 Project: Spark Issue

[jira] [Commented] (SPARK-33212) Move to shaded clients for Hadoop 3.x profile

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266282#comment-17266282 ] Apache Spark commented on SPARK-33212: -- User 'sunchao' has created a pull request for this issue:

[jira] [Commented] (SPARK-33212) Move to shaded clients for Hadoop 3.x profile

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266281#comment-17266281 ] Apache Spark commented on SPARK-33212: -- User 'sunchao' has created a pull request for this issue:

[jira] [Resolved] (SPARK-34037) Remove unnecessary upcasting for Avg & Sum which handle by themself internally

2021-01-15 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-34037. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31079

[jira] [Assigned] (SPARK-34037) Remove unnecessary upcasting for Avg & Sum which handle by themself internally

2021-01-15 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-34037: --- Assignee: Kent Yao > Remove unnecessary upcasting for Avg & Sum which handle by themself

[jira] [Created] (SPARK-34131) NPE when driver.podTemplateFile defines no containers

2021-01-15 Thread Jacek Laskowski (Jira)
Jacek Laskowski created SPARK-34131: --- Summary: NPE when driver.podTemplateFile defines no containers Key: SPARK-34131 URL: https://issues.apache.org/jira/browse/SPARK-34131 Project: Spark

[jira] [Updated] (SPARK-34037) Remove unnecessary upcasting for Avg & Sum which handle by themself internally

2021-01-15 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-34037: Affects Version/s: (was: 3.1.0) 3.2.0 > Remove unnecessary upcasting

[jira] [Comment Edited] (SPARK-34037) Remove unnecessary upcasting for Avg & Sum which handle by themself internally

2021-01-15 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260270#comment-17260270 ] Kent Yao edited comment on SPARK-34037 at 1/15/21, 5:43 PM: Oh, I mean that

[jira] [Updated] (SPARK-34037) Remove unnecessary upcasting for Avg & Sum which handle by themself internally

2021-01-15 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-34037: - Summary: Remove unnecessary upcasting for Avg & Sum which handle by themself internally (was: aggOrder

[jira] [Updated] (SPARK-34037) aggOrder should not be output as a auxiliary internal attribute

2021-01-15 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-34037: - Issue Type: Improvement (was: Bug) > aggOrder should not be output as a auxiliary internal attribute >

[jira] [Updated] (SPARK-34037) aggOrder should not be output as a auxiliary internal attribute

2021-01-15 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-34037: - Description: The type-coercion for numeric types of average and sum is not necessary at all, as the

[jira] [Updated] (SPARK-34037) aggOrder should not be output as a auxiliary internal attribute

2021-01-15 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-34037: - Priority: Major (was: Blocker) > aggOrder should not be output as a auxiliary internal attribute >

[jira] [Assigned] (SPARK-34130) Impove preformace for char varchar padding and length check with StaticInvoke

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34130: Assignee: (was: Apache Spark) > Impove preformace for char varchar padding and

[jira] [Commented] (SPARK-34130) Impove preformace for char varchar padding and length check with StaticInvoke

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266227#comment-17266227 ] Apache Spark commented on SPARK-34130: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34130) Impove preformace for char varchar padding and length check with StaticInvoke

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34130: Assignee: Apache Spark > Impove preformace for char varchar padding and length check

[jira] [Commented] (SPARK-33711) Race condition in Spark k8s Pod lifecycle manager that leads to shutdowns

2021-01-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266217#comment-17266217 ] Dongjoon Hyun commented on SPARK-33711: --- Thank you so much! > Race condition in Spark k8s Pod

[jira] [Created] (SPARK-34130) Impove preformace for char varchar padding and length check with StaticInvoke

2021-01-15 Thread Kent Yao (Jira)
Kent Yao created SPARK-34130: Summary: Impove preformace for char varchar padding and length check with StaticInvoke Key: SPARK-34130 URL: https://issues.apache.org/jira/browse/SPARK-34130 Project: Spark

[jira] [Comment Edited] (SPARK-33790) Reduce the rpc call of getFileStatus in SingleFileEventLogFileReader

2021-01-15 Thread dzcxzl (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17265724#comment-17265724 ] dzcxzl edited comment on SPARK-33790 at 1/15/21, 4:28 PM: --

[jira] [Comment Edited] (SPARK-33790) Reduce the rpc call of getFileStatus in SingleFileEventLogFileReader

2021-01-15 Thread dzcxzl (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17265724#comment-17265724 ] dzcxzl edited comment on SPARK-33790 at 1/15/21, 4:27 PM: -- Thread stack when

[jira] [Comment Edited] (SPARK-33790) Reduce the rpc call of getFileStatus in SingleFileEventLogFileReader

2021-01-15 Thread dzcxzl (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17265724#comment-17265724 ] dzcxzl edited comment on SPARK-33790 at 1/15/21, 4:26 PM: -- Thread stack when

[jira] [Comment Edited] (SPARK-33790) Reduce the rpc call of getFileStatus in SingleFileEventLogFileReader

2021-01-15 Thread dzcxzl (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17265724#comment-17265724 ] dzcxzl edited comment on SPARK-33790 at 1/15/21, 4:25 PM: -- Thread stack when

[jira] [Updated] (SPARK-33790) Reduce the rpc call of getFileStatus in SingleFileEventLogFileReader

2021-01-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33790: - Fix Version/s: 3.1.1 > Reduce the rpc call of getFileStatus in SingleFileEventLogFileReader >

[jira] [Commented] (SPARK-33245) Add built-in UDF - GETBIT

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266127#comment-17266127 ] Apache Spark commented on SPARK-33245: -- User 'beliefer' has created a pull request for this issue:

[jira] [Resolved] (SPARK-32598) Not able to see driver logs in spark history server in standalone mode

2021-01-15 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-32598. -- Fix Version/s: 3.1.1 3.0.2 Assignee: Kevin Wang Resolution:

[jira] [Updated] (SPARK-32598) Not able to see driver logs in spark history server in standalone mode

2021-01-15 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-32598: - Priority: Minor (was: Major) > Not able to see driver logs in spark history server in

[jira] [Assigned] (SPARK-33346) Change the never changed var to val

2021-01-15 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-33346: Assignee: Yang Jie > Change the never changed var to val >

[jira] [Resolved] (SPARK-33346) Change the never changed var to val

2021-01-15 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-33346. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31142

[jira] [Commented] (SPARK-34060) ALTER TABLE .. DROP PARTITION uncaches Hive table while updating table stats

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266072#comment-17266072 ] Apache Spark commented on SPARK-34060: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-34060) ALTER TABLE .. DROP PARTITION uncaches Hive table while updating table stats

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266071#comment-17266071 ] Apache Spark commented on SPARK-34060: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-29890) Unable to fill na with 0 with duplicate columns

2021-01-15 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266063#comment-17266063 ] Peter Toth commented on SPARK-29890: [~imback82], [~cloud_fan], due this change `fill` started to

[jira] [Updated] (SPARK-34129) Add table name to LogicalRelation.simpleString

2021-01-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-34129: Description: Current: {noformat} == Optimized Logical Plan == Project [i_item_sk#7 AS

[jira] [Assigned] (SPARK-34129) Add table name to LogicalRelation.simpleString

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34129: Assignee: Apache Spark > Add table name to LogicalRelation.simpleString >

[jira] [Commented] (SPARK-34129) Add table name to LogicalRelation.simpleString

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266057#comment-17266057 ] Apache Spark commented on SPARK-34129: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34129) Add table name to LogicalRelation.simpleString

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34129: Assignee: (was: Apache Spark) > Add table name to LogicalRelation.simpleString >

[jira] [Assigned] (SPARK-34064) Broadcast job is not aborted even the SQL statement canceled

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34064: Assignee: Lantao Jin (was: Apache Spark) > Broadcast job is not aborted even the SQL

[jira] [Assigned] (SPARK-34064) Broadcast job is not aborted even the SQL statement canceled

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34064: Assignee: Apache Spark (was: Lantao Jin) > Broadcast job is not aborted even the SQL

[jira] [Reopened] (SPARK-34064) Broadcast job is not aborted even the SQL statement canceled

2021-01-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reopened SPARK-34064: - > Broadcast job is not aborted even the SQL statement canceled >

[jira] [Updated] (SPARK-34064) Broadcast job is not aborted even the SQL statement canceled

2021-01-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-34064: Fix Version/s: (was: 3.1.1) > Broadcast job is not aborted even the SQL statement canceled >

[jira] [Updated] (SPARK-34128) Suppress excessive logging of TTransportExceptions in Spark ThriftServer

2021-01-15 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-34128: - Issue Type: Improvement (was: Bug) Priority: Minor (was: Major) > Suppress excessive

[jira] [Commented] (SPARK-33711) Race condition in Spark k8s Pod lifecycle manager that leads to shutdowns

2021-01-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266040#comment-17266040 ] Apache Spark commented on SPARK-33711: -- User 'attilapiros' has created a pull request for this

[jira] [Created] (SPARK-34129) Add table name to LogicalRelation.simpleString

2021-01-15 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-34129: --- Summary: Add table name to LogicalRelation.simpleString Key: SPARK-34129 URL: https://issues.apache.org/jira/browse/SPARK-34129 Project: Spark Issue Type:

[jira] [Commented] (SPARK-34128) Suppress excessive logging of TTransportExceptions in Spark ThriftServer

2021-01-15 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266025#comment-17266025 ] Kent Yao commented on SPARK-34128: -- cc [~dongjoon] [~srowen] [~hyukjin.kwon] > Suppress excessive

[jira] [Updated] (SPARK-34128) Suppress excessive logging of TTransportExceptions in Spark ThriftServer

2021-01-15 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-34128: - Description: Since Spark 3.0, the `libthrift` has been bumped up from 0.9.3 to 0.12.0. Due to

[jira] [Updated] (SPARK-34128) Suppress excessive logging of TTransportExceptions in Spark ThriftServer

2021-01-15 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-34128: - Description: Since Spark 3.0, the `libthrift` has been bumped up from 0.9.3 to 0.12.0. Due to

[jira] [Updated] (SPARK-34128) Suppress excessive logging of TTransportExceptions in Spark ThriftServer

2021-01-15 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-34128: - Description: Since Spark 3.0, the `libthrift` has been bumped up from 0.9.3 to 0.12.0. Due to

[jira] [Updated] (SPARK-34128) Suppress excessive logging of TTransportExceptions in Spark ThriftServer

2021-01-15 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-34128: - Description: Since Spark 3.0, the `libthrift` has been bumped up from 0.9.3 to 0.12.0. Due to, The

[jira] [Updated] (SPARK-34128) Suppress excessive logging of TTransportExceptions in Spark ThriftServer

2021-01-15 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-34128: - Description: Since Spark 3.0, the `libthrift` has been bumped up from 0.9.3 to 0.12.0. Due to, The

[jira] [Created] (SPARK-34128) Suppress excessive logging of TTransportExceptions in Spark ThriftServer

2021-01-15 Thread Kent Yao (Jira)
Kent Yao created SPARK-34128: Summary: Suppress excessive logging of TTransportExceptions in Spark ThriftServer Key: SPARK-34128 URL: https://issues.apache.org/jira/browse/SPARK-34128 Project: Spark

[jira] [Updated] (SPARK-33790) Reduce the rpc call of getFileStatus in SingleFileEventLogFileReader

2021-01-15 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-33790: - Fix Version/s: 3.0.2 > Reduce the rpc call of getFileStatus in SingleFileEventLogFileReader >

[jira] [Updated] (SPARK-34118) Replaces filter and check for emptiness with exists or forall

2021-01-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34118: - Fix Version/s: 3.1.1 3.0.2 > Replaces filter and check for emptiness with

[jira] [Resolved] (SPARK-34118) Replaces filter and check for emptiness with exists or forall

2021-01-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34118. -- Fix Version/s: (was: 3.1.1) (was: 3.2.0) 2.4.8

[jira] [Commented] (SPARK-34127) Support table valued command

2021-01-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17265934#comment-17265934 ] Yuming Wang commented on SPARK-34127: - +1 > Support table valued command >

[jira] [Commented] (SPARK-34098) HadoopVersionInfoSuite faild when maven test in Scala 2.13

2021-01-15 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17265897#comment-17265897 ] Yang Jie commented on SPARK-34098: -- Working on this > HadoopVersionInfoSuite faild when maven test in

[jira] [Created] (SPARK-34127) Support table valued command

2021-01-15 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-34127: -- Summary: Support table valued command Key: SPARK-34127 URL: https://issues.apache.org/jira/browse/SPARK-34127 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-34127) Support table valued command

2021-01-15 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17265874#comment-17265874 ] jiaan.geng commented on SPARK-34127: I'm working on. > Support table valued command >

[jira] [Commented] (SPARK-33711) Race condition in Spark k8s Pod lifecycle manager that leads to shutdowns

2021-01-15 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17265863#comment-17265863 ] Attila Zsolt Piros commented on SPARK-33711: Sure. Working on that. > Race condition in

[jira] [Updated] (SPARK-34118) Replaces filter and check for emptiness with exists or forall

2021-01-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34118: - Fix Version/s: 3.1.1 > Replaces filter and check for emptiness with exists or forall >

[jira] [Assigned] (SPARK-34118) Replaces filter and check for emptiness with exists or forall

2021-01-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-34118: Assignee: Yang Jie > Replaces filter and check for emptiness with exists or forall >

[jira] [Updated] (SPARK-34115) Long runtime on many environment variables

2021-01-15 Thread Norbert Schultz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Norbert Schultz updated SPARK-34115: Affects Version/s: 3.0.1 > Long runtime on many environment variables >

[jira] [Comment Edited] (SPARK-34115) Long runtime on many environment variables

2021-01-15 Thread Norbert Schultz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17265834#comment-17265834 ] Norbert Schultz edited comment on SPARK-34115 at 1/15/21, 8:58 AM: ---

[jira] [Updated] (SPARK-34115) Long runtime on many environment variables

2021-01-15 Thread Norbert Schultz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Norbert Schultz updated SPARK-34115: Component/s: SQL > Long runtime on many environment variables >

  1   2   >