[jira] [Resolved] (SPARK-31337) Support MS Sql Kerberos login in JDBC connector

[ https://issues.apache.org/jira/browse/SPARK-31337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-31337. Fix Version/s: 3.1.0 Assignee: Gabor Somogyi Resolution:

[jira] [Resolved] (SPARK-31559) AM starts with initial fetched tokens in any attempt

[ https://issues.apache.org/jira/browse/SPARK-31559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-31559. Fix Version/s: 3.0.0 Assignee: Jungtaek Lim Resolution:

[jira] [Resolved] (SPARK-31272) Support DB2 Kerberos login in JDBC connector

[ https://issues.apache.org/jira/browse/SPARK-31272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-31272. Fix Version/s: 3.1.0 Assignee: Gabor Somogyi Resolution:

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17089190#comment-17089190 ] Marcelo Masiero Vanzin commented on SPARK-1537: --- Well, the only thing to start with is the

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17089137#comment-17089137 ] Marcelo Masiero Vanzin commented on SPARK-1537: --- [~templedf] sorry forgot to reply. ATSv1

[jira] [Resolved] (SPARK-31021) Support MariaDB Kerberos login in JDBC connector

[ https://issues.apache.org/jira/browse/SPARK-31021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-31021. Fix Version/s: 3.1.0 Assignee: Gabor Somogyi Resolution:

[jira] [Commented] (CRYPTO-139) adding suuport for AARCH64 architecture in common crypto

[ https://issues.apache.org/jira/browse/CRYPTO-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072285#comment-17072285 ] Marcelo Masiero Vanzin commented on CRYPTO-139: --- [~blakegw] closing but I don't really have

[jira] [Resolved] (CRYPTO-139) adding suuport for AARCH64 architecture in common crypto

[ https://issues.apache.org/jira/browse/CRYPTO-139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved CRYPTO-139. --- Fix Version/s: 1.1.0 Resolution: Fixed > adding suuport for AARCH64

[jira] [Resolved] (SPARK-30874) Support Postgres Kerberos login in JDBC connector

[ https://issues.apache.org/jira/browse/SPARK-30874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-30874. Fix Version/s: 3.1.0 Assignee: Gabor Somogyi Resolution:

[jira] [Resolved] (SPARK-30481) Integrate event log compactor into Spark History Server

[ https://issues.apache.org/jira/browse/SPARK-30481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-30481. Fix Version/s: 3.0.0 Assignee: Jungtaek Lim Resolution:

[jira] [Commented] (SPARK-30557) Add public documentation for SPARK_SUBMIT_OPTS

[ https://issues.apache.org/jira/browse/SPARK-30557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018256#comment-17018256 ] Marcelo Masiero Vanzin commented on SPARK-30557: I don't exactly remember what that

[jira] [Resolved] (SPARK-29876) Delete/archive file source completed files in separate thread

[ https://issues.apache.org/jira/browse/SPARK-29876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-29876. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-29876) Delete/archive file source completed files in separate thread

[ https://issues.apache.org/jira/browse/SPARK-29876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-29876: -- Assignee: Gabor Somogyi > Delete/archive file source completed files

[jira] [Commented] (SPARK-27868) Better document shuffle / RPC listen backlog

[ https://issues.apache.org/jira/browse/SPARK-27868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018219#comment-17018219 ] Marcelo Masiero Vanzin commented on SPARK-27868: It's ok for now, it's done. Hopefully

[jira] [Assigned] (SPARK-29950) Deleted excess executors can connect back to driver in K8S with dyn alloc on

[ https://issues.apache.org/jira/browse/SPARK-29950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-29950: -- Assignee: Marcelo Masiero Vanzin > Deleted excess executors can

[jira] [Resolved] (SPARK-29950) Deleted excess executors can connect back to driver in K8S with dyn alloc on

[ https://issues.apache.org/jira/browse/SPARK-29950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-29950. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Commented] (SPARK-27868) Better document shuffle / RPC listen backlog

[ https://issues.apache.org/jira/browse/SPARK-27868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017281#comment-17017281 ] Marcelo Masiero Vanzin commented on SPARK-27868: You shoudn't have reverted the whole

[jira] [Updated] (SPARK-30246) Spark on Yarn External Shuffle Service Memory Leak

[ https://issues.apache.org/jira/browse/SPARK-30246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin updated SPARK-30246: --- Fix Version/s: (was: 2.4.5) 2.4.6 > Spark on Yarn

[jira] [Assigned] (SPARK-30246) Spark on Yarn External Shuffle Service Memory Leak

[ https://issues.apache.org/jira/browse/SPARK-30246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-30246: -- Assignee: Henrique dos Santos Goulart > Spark on Yarn External

[jira] [Resolved] (SPARK-30246) Spark on Yarn External Shuffle Service Memory Leak

[ https://issues.apache.org/jira/browse/SPARK-30246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-30246. Fix Version/s: 3.0.0 2.4.5 Resolution: Fixed

[jira] [Resolved] (SPARK-30495) How to disable 'spark.security.credentials.${service}.enabled' in Structured streaming while connecting to a kafka cluster

[ https://issues.apache.org/jira/browse/SPARK-30495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-30495. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-30495) How to disable 'spark.security.credentials.${service}.enabled' in Structured streaming while connecting to a kafka cluster

[ https://issues.apache.org/jira/browse/SPARK-30495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-30495: -- Assignee: Gabor Somogyi > How to disable

[jira] [Resolved] (SPARK-30479) Apply compaction of event log to SQL events

[ https://issues.apache.org/jira/browse/SPARK-30479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-30479. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-30479) Apply compaction of event log to SQL events

[ https://issues.apache.org/jira/browse/SPARK-30479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-30479: -- Assignee: Jungtaek Lim > Apply compaction of event log to SQL events

[jira] [Assigned] (SPARK-27142) Provide REST API for SQL level information

[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-27142: -- Assignee: Ajith S > Provide REST API for SQL level information >

[jira] [Resolved] (SPARK-27142) Provide REST API for SQL level information

[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-27142. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Resolved] (SPARK-29779) Compact old event log files and clean up

[ https://issues.apache.org/jira/browse/SPARK-29779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-29779. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-29779) Compact old event log files and clean up

[ https://issues.apache.org/jira/browse/SPARK-29779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-29779: -- Assignee: Jungtaek Lim > Compact old event log files and clean up >

[jira] [Assigned] (SPARK-30281) 'archive' option in FileStreamSource misses to consider partitioned and recursive option

[ https://issues.apache.org/jira/browse/SPARK-30281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-30281: -- Assignee: Jungtaek Lim > 'archive' option in FileStreamSource misses

[jira] [Resolved] (SPARK-30281) 'archive' option in FileStreamSource misses to consider partitioned and recursive option

[ https://issues.apache.org/jira/browse/SPARK-30281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-30281. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-30313) Flaky test: MasterSuite.master/worker web ui available with reverseProxy

[ https://issues.apache.org/jira/browse/SPARK-30313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-30313: -- Assignee: Jungtaek Lim > Flaky test: MasterSuite.master/worker web

[jira] [Resolved] (SPARK-30313) Flaky test: MasterSuite.master/worker web ui available with reverseProxy

[ https://issues.apache.org/jira/browse/SPARK-30313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-30313. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-30285) Fix deadlock between LiveListenerBus#stop and AsyncEventQueue#removeListenerOnError

[ https://issues.apache.org/jira/browse/SPARK-30285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-30285: -- Assignee: Wang Shuo > Fix deadlock between LiveListenerBus#stop and

[jira] [Resolved] (SPARK-30285) Fix deadlock between LiveListenerBus#stop and AsyncEventQueue#removeListenerOnError

[ https://issues.apache.org/jira/browse/SPARK-30285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-30285. Fix Version/s: 3.0.0 2.4.5 Resolution: Fixed

[jira] [Commented] (SPARK-30225) "Stream is corrupted at" exception on reading disk-spilled data of a shuffle operation

[ https://issues.apache.org/jira/browse/SPARK-30225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17005838#comment-17005838 ] Marcelo Masiero Vanzin commented on SPARK-30225: The changes in SPARK-23366 may not

[jira] [Assigned] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-18105: -- Assignee: Marcelo Masiero Vanzin (was: Davies Liu) > LZ4 failed to

[jira] [Assigned] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-18105: -- Assignee: (was: Marcelo Masiero Vanzin) > LZ4 failed to

[jira] [Resolved] (SPARK-21869) A cached Kafka producer should not be closed if any task is using it.

[ https://issues.apache.org/jira/browse/SPARK-21869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-21869. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-21869) A cached Kafka producer should not be closed if any task is using it.

[ https://issues.apache.org/jira/browse/SPARK-21869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-21869: -- Assignee: Jungtaek Lim (was: Gabor Somogyi) > A cached Kafka

[jira] [Resolved] (SPARK-26418) Only OpenBlocks without any ChunkFetch for one stream will cause memory leak in ExternalShuffleService

[ https://issues.apache.org/jira/browse/SPARK-26418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-26418. Resolution: Duplicate > Only OpenBlocks without any ChunkFetch for one

[jira] [Resolved] (SPARK-17398) Failed to query on external JSon Partitioned table

[ https://issues.apache.org/jira/browse/SPARK-17398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-17398. Fix Version/s: (was: 2.0.1) 3.0.0

[jira] [Commented] (SPARK-30313) Flaky test: MasterSuite.master/worker web ui available with reverseProxy

[ https://issues.apache.org/jira/browse/SPARK-30313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000493#comment-17000493 ] Marcelo Masiero Vanzin commented on SPARK-30313: >From the logs (in case jenkins cleans

[jira] [Comment Edited] (SPARK-30313) Flaky test: MasterSuite.master/worker web ui available with reverseProxy

[ https://issues.apache.org/jira/browse/SPARK-30313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000493#comment-17000493 ] Marcelo Masiero Vanzin edited comment on SPARK-30313 at 12/20/19 12:02 AM:

[jira] [Created] (SPARK-30313) Flaky test: MasterSuite.master/worker web ui available with reverseProxy

Marcelo Masiero Vanzin created SPARK-30313: -- Summary: Flaky test: MasterSuite.master/worker web ui available with reverseProxy Key: SPARK-30313 URL: https://issues.apache.org/jira/browse/SPARK-30313

[jira] [Assigned] (SPARK-30235) Keeping compatibility with 2.4 external shuffle service regarding host local shuffle blocks reading

[ https://issues.apache.org/jira/browse/SPARK-30235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-30235: -- Assignee: Attila Zsolt Piros > Keeping compatibility with 2.4

[jira] [Resolved] (SPARK-30235) Keeping compatibility with 2.4 external shuffle service regarding host local shuffle blocks reading

[ https://issues.apache.org/jira/browse/SPARK-30235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-30235. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Resolved] (SPARK-30277) NoSuchMethodError in Spark 3.0.0-preview with Delta Lake

[ https://issues.apache.org/jira/browse/SPARK-30277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-30277. Resolution: Not A Problem That's a internal Spark class, which means that

[jira] [Commented] (SPARK-25392) [Spark Job History]Inconsistent behaviour for pool details in spark web UI and history server page

[ https://issues.apache.org/jira/browse/SPARK-25392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16997698#comment-16997698 ] Marcelo Masiero Vanzin commented on SPARK-25392: The fix basically hides pool details

[jira] [Assigned] (SPARK-25392) [Spark Job History]Inconsistent behaviour for pool details in spark web UI and history server page

[ https://issues.apache.org/jira/browse/SPARK-25392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-25392: -- Assignee: shahid > [Spark Job History]Inconsistent behaviour for

[jira] [Resolved] (SPARK-25392) [Spark Job History]Inconsistent behaviour for pool details in spark web UI and history server page

[ https://issues.apache.org/jira/browse/SPARK-25392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-25392. Fix Version/s: 3.0.0 2.4.5 Resolution: Fixed

[jira] [Assigned] (SPARK-29043) [History Server]Only one replay thread of FsHistoryProvider work because of straggler

[ https://issues.apache.org/jira/browse/SPARK-29043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-29043: -- Assignee: feiwang > [History Server]Only one replay thread of

[jira] [Resolved] (SPARK-29043) [History Server]Only one replay thread of FsHistoryProvider work because of straggler

[ https://issues.apache.org/jira/browse/SPARK-29043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-29043. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-29574) spark with user provided hadoop doesn't work on kubernetes

[ https://issues.apache.org/jira/browse/SPARK-29574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-29574: -- Assignee: Shahin Shakeri > spark with user provided hadoop doesn't

[jira] [Resolved] (SPARK-29574) spark with user provided hadoop doesn't work on kubernetes

[ https://issues.apache.org/jira/browse/SPARK-29574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-29574. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Resolved] (SPARK-30167) Log4j configuration for REPL can't override the root logger properly.

[ https://issues.apache.org/jira/browse/SPARK-30167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-30167. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Created] (SPARK-30240) Spark UI redirects do not always work behind (dumb) proxies

Marcelo Masiero Vanzin created SPARK-30240: -- Summary: Spark UI redirects do not always work behind (dumb) proxies Key: SPARK-30240 URL: https://issues.apache.org/jira/browse/SPARK-30240

[jira] [Assigned] (SPARK-29152) Spark Executor Plugin API shutdown is not proper when dynamic allocation enabled

[ https://issues.apache.org/jira/browse/SPARK-29152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-29152: -- Assignee: Rakesh Raushan > Spark Executor Plugin API shutdown is not

[jira] [Resolved] (SPARK-29152) Spark Executor Plugin API shutdown is not proper when dynamic allocation enabled

[ https://issues.apache.org/jira/browse/SPARK-29152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-29152. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Created] (SPARK-30129) New auth engine does not keep client ID in TransportClient after auth

Marcelo Masiero Vanzin created SPARK-30129: -- Summary: New auth engine does not keep client ID in TransportClient after auth Key: SPARK-30129 URL: https://issues.apache.org/jira/browse/SPARK-30129

[jira] [Resolved] (SPARK-27651) Avoid the network when block manager fetches shuffle blocks from the same host

[ https://issues.apache.org/jira/browse/SPARK-27651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-27651. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-27651) Avoid the network when block manager fetches shuffle blocks from the same host

[ https://issues.apache.org/jira/browse/SPARK-27651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-27651: -- Assignee: Attila Zsolt Piros > Avoid the network when block manager

[jira] [Commented] (SPARK-30033) Use Spark plugin support to manage shuffle plugin lifecycle

[ https://issues.apache.org/jira/browse/SPARK-30033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981997#comment-16981997 ] Marcelo Masiero Vanzin commented on SPARK-30033: [~mcheah] [~yifeih] I had plans to do

[jira] [Created] (SPARK-30033) Use Spark plugin support to manage shuffle plugin lifecycle

Marcelo Masiero Vanzin created SPARK-30033: -- Summary: Use Spark plugin support to manage shuffle plugin lifecycle Key: SPARK-30033 URL: https://issues.apache.org/jira/browse/SPARK-30033

[jira] [Updated] (SPARK-29971) Multiple possible buffer leaks in TransportFrameDecoder and TransportCipher

[ https://issues.apache.org/jira/browse/SPARK-29971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin updated SPARK-29971: --- Fix Version/s: 2.4.5 > Multiple possible buffer leaks in

[jira] [Assigned] (SPARK-26260) Task Summary Metrics for Stage Page: Efficient implementation for SHS when using disk store.

[ https://issues.apache.org/jira/browse/SPARK-26260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-26260: -- Assignee: shahid > Task Summary Metrics for Stage Page: Efficient

[jira] [Resolved] (SPARK-26260) Task Summary Metrics for Stage Page: Efficient implementation for SHS when using disk store.

[ https://issues.apache.org/jira/browse/SPARK-26260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-26260. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Resolved] (SPARK-29971) Multiple possible buffer leaks in TransportFrameDecoder and TransportCipher

[ https://issues.apache.org/jira/browse/SPARK-29971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-29971. Fix Version/s: 3.0.0 Assignee: Norman Maurer Resolution:

[jira] [Created] (SPARK-29965) Race in executor shutdown handling can lead to executor never fully unregistering

Marcelo Masiero Vanzin created SPARK-29965: -- Summary: Race in executor shutdown handling can lead to executor never fully unregistering Key: SPARK-29965 URL:

[jira] [Created] (SPARK-29950) Deleted excess executors can connect back to driver in K8S with dyn alloc on

Marcelo Masiero Vanzin created SPARK-29950: -- Summary: Deleted excess executors can connect back to driver in K8S with dyn alloc on Key: SPARK-29950 URL: https://issues.apache.org/jira/browse/SPARK-29950

[jira] [Resolved] (SPARK-29833) Add FileNotFoundException check for spark.yarn.jars

[ https://issues.apache.org/jira/browse/SPARK-29833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-29833. Fix Version/s: 3.0.0 Assignee: ulysses you Resolution:

[jira] [Created] (SPARK-29905) ExecutorPodsLifecycleManager has sub-optimal behavior with dynamic allocation

Marcelo Masiero Vanzin created SPARK-29905: -- Summary: ExecutorPodsLifecycleManager has sub-optimal behavior with dynamic allocation Key: SPARK-29905 URL: https://issues.apache.org/jira/browse/SPARK-29905

[jira] [Resolved] (SPARK-29882) SPARK on Kubernetes is Broken for SPARK with no Hadoop release

[ https://issues.apache.org/jira/browse/SPARK-29882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-29882. Fix Version/s: (was: 2.4.4) Target Version/s: (was: 2.4.4)

[jira] [Created] (SPARK-29865) k8s executor pods all have different prefixes in client mode

Marcelo Masiero Vanzin created SPARK-29865: -- Summary: k8s executor pods all have different prefixes in client mode Key: SPARK-29865 URL: https://issues.apache.org/jira/browse/SPARK-29865

[jira] [Resolved] (SPARK-29487) Ability to run Spark Kubernetes other than from /opt/spark

[ https://issues.apache.org/jira/browse/SPARK-29487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-29487. Resolution: Duplicate This is basically allowing people to customize

[jira] [Assigned] (SPARK-29755) ClassCastException occurs when reading events from SHS

[ https://issues.apache.org/jira/browse/SPARK-29755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-29755: -- Assignee: Jungtaek Lim > ClassCastException occurs when reading

[jira] [Resolved] (SPARK-29755) ClassCastException occurs when reading events from SHS

[ https://issues.apache.org/jira/browse/SPARK-29755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-29755. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Resolved] (SPARK-26154) Stream-stream joins - left outer join gives inconsistent output

[ https://issues.apache.org/jira/browse/SPARK-26154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-26154. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-26154) Stream-stream joins - left outer join gives inconsistent output

[ https://issues.apache.org/jira/browse/SPARK-26154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-26154: -- Assignee: Jungtaek Lim > Stream-stream joins - left outer join gives

[jira] [Resolved] (SPARK-29770) Allow setting spark.app.id when spark-submit for Spark on Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-29770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-29770. Resolution: Won't Fix See comments in PR. > Allow setting spark.app.id

[jira] [Resolved] (SPARK-29790) Add notes about port being required for Kubernetes API URL when set as master

[ https://issues.apache.org/jira/browse/SPARK-29790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-29790. Fix Version/s: 3.0.0 2.4.5 Assignee: Emil

[jira] [Assigned] (SPARK-21869) A cached Kafka producer should not be closed if any task is using it.

[ https://issues.apache.org/jira/browse/SPARK-21869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-21869: -- Assignee: Gabor Somogyi > A cached Kafka producer should not be

[jira] [Resolved] (SPARK-21869) A cached Kafka producer should not be closed if any task is using it.

[ https://issues.apache.org/jira/browse/SPARK-21869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-21869. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Resolved] (SPARK-29635) Deduplicate test suites between Kafka micro-batch sink and Kafka continuous sink

[ https://issues.apache.org/jira/browse/SPARK-29635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-29635. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-29635) Deduplicate test suites between Kafka micro-batch sink and Kafka continuous sink

[ https://issues.apache.org/jira/browse/SPARK-29635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-29635: -- Assignee: Jungtaek Lim > Deduplicate test suites between Kafka

[jira] [Assigned] (SPARK-29642) ContinuousMemoryStream throws error on String type

[ https://issues.apache.org/jira/browse/SPARK-29642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-29642: -- Assignee: Jungtaek Lim > ContinuousMemoryStream throws error on

[jira] [Resolved] (SPARK-29642) ContinuousMemoryStream throws error on String type

[ https://issues.apache.org/jira/browse/SPARK-29642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-29642. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-29603) Support application priority for spark on yarn

[ https://issues.apache.org/jira/browse/SPARK-29603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-29603: -- Assignee: Kent Yao > Support application priority for spark on yarn

[jira] [Resolved] (SPARK-29603) Support application priority for spark on yarn

[ https://issues.apache.org/jira/browse/SPARK-29603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-29603. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Created] (SPARK-29766) Aggregate metrics asynchronously in SQL listener

Marcelo Masiero Vanzin created SPARK-29766: -- Summary: Aggregate metrics asynchronously in SQL listener Key: SPARK-29766 URL: https://issues.apache.org/jira/browse/SPARK-29766 Project: Spark

[jira] [Resolved] (SPARK-29763) Stage UI Page not showing all accumulators in Task Table

[ https://issues.apache.org/jira/browse/SPARK-29763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-29763. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Resolved] (SPARK-20568) Delete files after processing in structured streaming

[ https://issues.apache.org/jira/browse/SPARK-20568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-20568. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-20568) Delete files after processing in structured streaming

[ https://issues.apache.org/jira/browse/SPARK-20568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-20568: -- Assignee: Jungtaek Lim > Delete files after processing in structured

[jira] [Resolved] (SPARK-29397) Create new plugin interface for driver and executor plugins

[ https://issues.apache.org/jira/browse/SPARK-29397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-29397. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-29397) Create new plugin interface for driver and executor plugins

[ https://issues.apache.org/jira/browse/SPARK-29397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-29397: -- Assignee: Marcelo Masiero Vanzin > Create new plugin interface for

[jira] [Resolved] (SPARK-29637) SHS Endpoint /applications//jobs/ doesn't include description

[ https://issues.apache.org/jira/browse/SPARK-29637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-29637. Fix Version/s: 3.0.0 2.4.5 Resolution: Fixed

[jira] [Assigned] (SPARK-29637) SHS Endpoint /applications//jobs/ doesn't include description

[ https://issues.apache.org/jira/browse/SPARK-29637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-29637: -- Assignee: Gabor Somogyi > SHS Endpoint /applications//jobs/ doesn't

[jira] [Resolved] (SPARK-29509) Deduplicate code blocks in Kafka data source

[ https://issues.apache.org/jira/browse/SPARK-29509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-29509. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-29509) Deduplicate code blocks in Kafka data source

[ https://issues.apache.org/jira/browse/SPARK-29509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-29509: -- Assignee: Jungtaek Lim > Deduplicate code blocks in Kafka data

[jira] [Resolved] (SPARK-29472) Mechanism for Excluding Jars at Launch for YARN

[ https://issues.apache.org/jira/browse/SPARK-29472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-29472. Resolution: Won't Fix > Mechanism for Excluding Jars at Launch for YARN >

[jira] [Commented] (SPARK-29472) Mechanism for Excluding Jars at Launch for YARN

[ https://issues.apache.org/jira/browse/SPARK-29472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16959179#comment-16959179 ] Marcelo Masiero Vanzin commented on SPARK-29472: bq. customers sometimes need to debug

  1   2   >