[jira] [Commented] (SPARK-36321) Do not fail application in kubernetes if name is too long

[ https://issues.apache.org/jira/browse/SPARK-36321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770619#comment-17770619 ] Wing Yew Poon commented on SPARK-36321: --- [~dongjoon], is this fixed by SPARK-39614? > Do not fail

[jira] [Commented] (SPARK-38918) Nested column pruning should filter out attributes that do not belong to the current relation

[ https://issues.apache.org/jira/browse/SPARK-38918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17644041#comment-17644041 ] Wing Yew Poon commented on SPARK-38918: --- It seems that this is fixed in 3.2.2

[jira] [Commented] (SPARK-38992) Avoid using bash -c in ShellBasedGroupsMappingProvider

[ https://issues.apache.org/jira/browse/SPARK-38992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17568661#comment-17568661 ] Wing Yew Poon commented on SPARK-38992: --- Does this vulnerability not affect Spark 2.4? > Avoid

[jira] [Updated] (SPARK-34779) ExecutorMetricsPoller should keep stage entry in stageTCMP until a heartbeat occurs

[ https://issues.apache.org/jira/browse/SPARK-34779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wing Yew Poon updated SPARK-34779: -- Summary: ExecutorMetricsPoller should keep stage entry in stageTCMP until a heartbeat occurs

[jira] [Commented] (SPARK-34779) ExecutoMetricsPoller should keep stage entry in stageTCMP until a heartbeat occurs

[ https://issues.apache.org/jira/browse/SPARK-34779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17311068#comment-17311068 ] Wing Yew Poon commented on SPARK-34779: --- I don't think there is a bug as claimed in the

[jira] [Commented] (SPARK-33001) Why am I receiving this warning?

[ https://issues.apache.org/jira/browse/SPARK-33001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17232192#comment-17232192 ] Wing Yew Poon commented on SPARK-33001: --- I may have been the last to touch

[jira] [Commented] (SPARK-31693) Investigate AmpLab Jenkins server network issue

[ https://issues.apache.org/jira/browse/SPARK-31693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163987#comment-17163987 ] Wing Yew Poon commented on SPARK-31693: --- I'm seeing a problem with the .m2 cache on

[jira] [Commented] (SPARK-32054) Flaky test: org.apache.spark.sql.connector.FileDataSourceV2FallBackSuite.Fallback Parquet V2 to V1

[ https://issues.apache.org/jira/browse/SPARK-32054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163976#comment-17163976 ] Wing Yew Poon commented on SPARK-32054: ---

[jira] [Updated] (SPARK-32003) Shuffle files for lost executor are not unregistered if fetch failure occurs after executor is lost

[ https://issues.apache.org/jira/browse/SPARK-32003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wing Yew Poon updated SPARK-32003: -- Affects Version/s: 3.0.0 > Shuffle files for lost executor are not unregistered if fetch

[jira] [Commented] (SPARK-32003) Shuffle files for lost executor are not unregistered if fetch failure occurs after executor is lost

[ https://issues.apache.org/jira/browse/SPARK-32003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17136792#comment-17136792 ] Wing Yew Poon commented on SPARK-32003: --- I will open a PR soon with a solution. > Shuffle files

[jira] [Created] (SPARK-32003) Shuffle files for lost executor are not unregistered if fetch failure occurs after executor is lost

Wing Yew Poon created SPARK-32003: - Summary: Shuffle files for lost executor are not unregistered if fetch failure occurs after executor is lost Key: SPARK-32003 URL:

[jira] [Commented] (SPARK-28626) Spark leaves unencrypted data on local disk, even with encryption turned on (CVE-2019-10099)

[ https://issues.apache.org/jira/browse/SPARK-28626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17062230#comment-17062230 ] Wing Yew Poon commented on SPARK-28626: --- For the record, to assist folks who need to backport

[jira] [Reopened] (SPARK-17398) Failed to query on external JSon Partitioned table

[ https://issues.apache.org/jira/browse/SPARK-17398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wing Yew Poon reopened SPARK-17398: --- This issue was never actually fixed. Evidently the problem still exists. I'll create a PR with

[jira] [Commented] (SPARK-28770) Flaky Tests: Test ReplayListenerSuite.End-to-end replay with compression failed

[ https://issues.apache.org/jira/browse/SPARK-28770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922925#comment-16922925 ] Wing Yew Poon commented on SPARK-28770: --- As [~irashid] noted, following an offline discussion, we

[jira] [Commented] (SPARK-28770) Flaky Tests: Test ReplayListenerSuite.End-to-end replay with compression failed

[ https://issues.apache.org/jira/browse/SPARK-28770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921955#comment-16921955 ] Wing Yew Poon commented on SPARK-28770: --- My earlier analysis of why the test failure occurs was

[jira] [Commented] (SPARK-28770) Flaky Tests: Test ReplayListenerSuite.End-to-end replay with compression failed

[ https://issues.apache.org/jira/browse/SPARK-28770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920234#comment-16920234 ] Wing Yew Poon commented on SPARK-28770: --- I looked into the issue further. In EventLoggingListener,

[jira] [Commented] (SPARK-28770) Flaky Tests: Test ReplayListenerSuite.End-to-end replay with compression failed

[ https://issues.apache.org/jira/browse/SPARK-28770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920212#comment-16920212 ] Wing Yew Poon commented on SPARK-28770: --- On my branch from which

[jira] [Commented] (SPARK-27324) document configurations related to executor metrics

[ https://issues.apache.org/jira/browse/SPARK-27324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806297#comment-16806297 ] Wing Yew Poon commented on SPARK-27324: --- I'd hold off on this until SPARK-26329 is resolved. >

[jira] [Updated] (SPARK-27324) document configurations related to executor metrics

[ https://issues.apache.org/jira/browse/SPARK-27324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wing Yew Poon updated SPARK-27324: -- Description: SPARK-23429 introduced executor memory metrics, and the configuration, 

[jira] [Updated] (SPARK-27324) document configurations related to executor metrics

[ https://issues.apache.org/jira/browse/SPARK-27324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wing Yew Poon updated SPARK-27324: -- Description: SPARK-23429 introduced executor memory metrics, and the configuration, 

[jira] [Created] (SPARK-27324) document configurations related to executor metrics

Wing Yew Poon created SPARK-27324: - Summary: document configurations related to executor metrics Key: SPARK-27324 URL: https://issues.apache.org/jira/browse/SPARK-27324 Project: Spark Issue

[jira] [Commented] (SPARK-26329) ExecutorMetrics should poll faster than heartbeats

[ https://issues.apache.org/jira/browse/SPARK-26329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16745313#comment-16745313 ] Wing Yew Poon commented on SPARK-26329: --- I am working on this. In the executor, we can track what

[jira] [Commented] (SPARK-22403) StructuredKafkaWordCount example fails in YARN cluster mode

[ https://issues.apache.org/jira/browse/SPARK-22403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16227298#comment-16227298 ] Wing Yew Poon commented on SPARK-22403: --- I realize that in a production application, one would set

[jira] [Updated] (SPARK-22403) StructuredKafkaWordCount example fails in YARN cluster mode

[ https://issues.apache.org/jira/browse/SPARK-22403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wing Yew Poon updated SPARK-22403: -- Description: When I run the StructuredKafkaWordCount example in YARN client mode, it runs

[jira] [Updated] (SPARK-22403) StructuredKafkaWordCount example fails in YARN cluster mode

[ https://issues.apache.org/jira/browse/SPARK-22403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wing Yew Poon updated SPARK-22403: -- Description: When I run the StructuredKafkaWordCount example in YARN client mode, it runs

[jira] [Commented] (SPARK-22403) StructuredKafkaWordCount example fails in YARN cluster mode

[ https://issues.apache.org/jira/browse/SPARK-22403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225913#comment-16225913 ] Wing Yew Poon commented on SPARK-22403: --- The simplest change that will solve the problem in this

[jira] [Created] (SPARK-22403) StructuredKafkaWordCount example fails in YARN cluster mode

Wing Yew Poon created SPARK-22403: - Summary: StructuredKafkaWordCount example fails in YARN cluster mode Key: SPARK-22403 URL: https://issues.apache.org/jira/browse/SPARK-22403 Project: Spark

[jira] [Commented] (SPARK-6325) YarnAllocator crash with dynamic allocation on

[ https://issues.apache.org/jira/browse/SPARK-6325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14360894#comment-14360894 ] Wing Yew Poon commented on SPARK-6325: -- The testcase actually came from a similar

[jira] [Created] (SPARK-5485) typo in spark streaming configuration parameter

Wing Yew Poon created SPARK-5485: Summary: typo in spark streaming configuration parameter Key: SPARK-5485 URL: https://issues.apache.org/jira/browse/SPARK-5485 Project: Spark Issue Type:

[jira] [Commented] (SPARK-5485) typo in spark streaming configuration parameter

[ https://issues.apache.org/jira/browse/SPARK-5485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14297449#comment-14297449 ] Wing Yew Poon commented on SPARK-5485: -- It should be