[jira] [Commented] (SPARK-27891) Long running spark jobs fail because of HDFS delegation token expires

[ https://issues.apache.org/jira/browse/SPARK-27891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16852408#comment-16852408 ] Marcelo Vanzin commented on SPARK-27891: {{container_e48_1559242207407_0001_02_01}} tells me

[jira] [Resolved] (SPARK-27891) Long running spark jobs fail because of HDFS delegation token expires

[ https://issues.apache.org/jira/browse/SPARK-27891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27891. Resolution: Not A Problem You have to provide a keytab for Spark for this to work

[jira] [Resolved] (SPARK-27773) Add shuffle service metric for number of exceptions caught in ExternalShuffleBlockHandler

[ https://issues.apache.org/jira/browse/SPARK-27773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27773. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24645 [https

[spark] branch master updated: [SPARK-27773][SHUFFLE] add metrics for number of exceptions caught in ExternalShuffleBlockHandler

This is an automated email from the ASF dual-hosted git repository. vanzin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 568512c [SPARK-27773][SHUFFLE] add metrics

[jira] [Assigned] (SPARK-27773) Add shuffle service metric for number of exceptions caught in ExternalShuffleBlockHandler

[ https://issues.apache.org/jira/browse/SPARK-27773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-27773: -- Assignee: Steven Rand > Add shuffle service metric for number of exceptions cau

[jira] [Resolved] (SPARK-27378) spark-submit requests GPUs in YARN mode

[ https://issues.apache.org/jira/browse/SPARK-27378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27378. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24634 [https

[spark] branch master updated (955eef9 -> 0ced4c0)

This is an automated email from the ASF dual-hosted git repository. vanzin pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 955eef9 Revert "[SPARK-27831][SQL][TEST][test-hadoop3.2] Move Hive test jars to maven dependency" a

[jira] [Assigned] (SPARK-27378) spark-submit requests GPUs in YARN mode

[ https://issues.apache.org/jira/browse/SPARK-27378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-27378: -- Assignee: Thomas Graves > spark-submit requests GPUs in YARN m

[jira] [Created] (SPARK-27868) Better document shuffle / RPC listen backlog

Marcelo Vanzin created SPARK-27868: -- Summary: Better document shuffle / RPC listen backlog Key: SPARK-27868 URL: https://issues.apache.org/jira/browse/SPARK-27868 Project: Spark Issue Type

[jira] [Commented] (SPARK-24149) Automatic namespaces discovery in HDFS federation

[ https://issues.apache.org/jira/browse/SPARK-24149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847926#comment-16847926 ] Marcelo Vanzin commented on SPARK-24149: bq. If they are unrelated the user explicitly provides

[jira] [Commented] (SPARK-24149) Automatic namespaces discovery in HDFS federation

[ https://issues.apache.org/jira/browse/SPARK-24149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847888#comment-16847888 ] Marcelo Vanzin commented on SPARK-24149: bq. I think we are duplicating the logic here Do you

[spark] branch master updated (de13f70 -> 1e87694)

This is an automated email from the ASF dual-hosted git repository. vanzin pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from de13f70 [SPARK-27824][SQL] Make rule EliminateResolvedHint idempotent add 1e87694 [SPARK-27677][CORE] Disable

[jira] [Assigned] (SPARK-27677) Disk-persisted RDD blocks served by shuffle service, and ignored for Dynamic Allocation

[ https://issues.apache.org/jira/browse/SPARK-27677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-27677: -- Assignee: Attila Zsolt Piros > Disk-persisted RDD blocks served by shuffle serv

[jira] [Resolved] (SPARK-27677) Disk-persisted RDD blocks served by shuffle service, and ignored for Dynamic Allocation

[ https://issues.apache.org/jira/browse/SPARK-27677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27677. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24499 [https

[spark] branch master updated (1a68fc3 -> e9f3f62)

This is an automated email from the ASF dual-hosted git repository. vanzin pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 1a68fc3 [SPARK-27816][SQL] make TreeNode tag type safe add e9f3f62 [SPARK-27677][CORE] Serve local disk

[jira] [Resolved] (SPARK-27804) LiveListenerBus#addToQueue : create multiple AsyncEventQueues under race condition

[ https://issues.apache.org/jira/browse/SPARK-27804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27804. Resolution: Not A Problem > LiveListenerBus#addToQueue : create multiple AsyncEventQue

[jira] [Commented] (SPARK-27797) Shuffle service metric "registeredConnections" not tracked correctly

[ https://issues.apache.org/jira/browse/SPARK-27797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16846068#comment-16846068 ] Marcelo Vanzin commented on SPARK-27797: You're right. Looks just like some dead code

[jira] [Created] (SPARK-27797) Shuffle service metric "registeredConnections" not tracked correctly

Marcelo Vanzin created SPARK-27797: -- Summary: Shuffle service metric "registeredConnections" not tracked correctly Key: SPARK-27797 URL: https://issues.apache.org/jira/browse/SPARK-27797

[jira] [Updated] (SPARK-27726) Performance of InMemoryStore suffers under load

[ https://issues.apache.org/jira/browse/SPARK-27726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-27726: --- Fix Version/s: 2.4.4 > Performance of InMemoryStore suffers under l

[jira] [Commented] (SPARK-27726) Performance of InMemoryStore suffers under load

[ https://issues.apache.org/jira/browse/SPARK-27726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845061#comment-16845061 ] Marcelo Vanzin commented on SPARK-27726: [~davidnavas] all of the sub-tasks were handled

[jira] [Assigned] (SPARK-27726) Performance of InMemoryStore suffers under load

[ https://issues.apache.org/jira/browse/SPARK-27726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-27726: -- Assignee: David C Navas > Performance of InMemoryStore suffers under l

[jira] [Resolved] (SPARK-27726) Performance of InMemoryStore suffers under load

[ https://issues.apache.org/jira/browse/SPARK-27726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27726. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24616 [https

[spark] branch master updated (1e0facb -> 9e73be3)

This is an automated email from the ASF dual-hosted git repository. vanzin pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 1e0facb [SQL][DOC][MINOR] update documents for Table and WriteBuilder add 9e73be3 [SPARK-27726][CORE] Fix

[jira] [Resolved] (SPARK-27745) build/mvn take wrong scala version when compile for scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-27745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27745. Resolution: Not A Bug You need to run {{./dev/change-scala-version.sh}} first. Pretty

[spark] branch master updated (bfb3ffe -> d14e2d7)

This is an automated email from the ASF dual-hosted git repository. vanzin pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from bfb3ffe [SPARK-27682][CORE][GRAPHX][MLLIB] Replace use of collections and methods that will be removed in Scala

[jira] [Assigned] (SPARK-27678) Support Knox user impersonation in UI

[ https://issues.apache.org/jira/browse/SPARK-27678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-27678: -- Assignee: Marcelo Vanzin > Support Knox user impersonation in

[jira] [Resolved] (SPARK-27678) Support Knox user impersonation in UI

[ https://issues.apache.org/jira/browse/SPARK-27678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27678. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24582 [https

[jira] [Commented] (SPARK-27681) Use scala.collection.Seq explicitly instead of scala.Seq alias

[ https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839674#comment-16839674 ] Marcelo Vanzin commented on SPARK-27681: bq. 2043 in method parameters, 1488 in method return

[jira] [Commented] (SPARK-27681) Use scala.collection.Seq explicitly instead of scala.Seq alias

[ https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16838799#comment-16838799 ] Marcelo Vanzin commented on SPARK-27681: bq. The case to consider is, roughly, where Seq

[jira] [Commented] (SPARK-27681) Use scala.collection.Seq explicitly instead of scala.Seq alias

[ https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16838766#comment-16838766 ] Marcelo Vanzin commented on SPARK-27681: bq. But, user code that passes a non-immutable Seq

[jira] [Commented] (SPARK-27681) Use scala.collection.Seq explicitly instead of scala.Seq alias

[ https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16838715#comment-16838715 ] Marcelo Vanzin commented on SPARK-27681: bq. varargs methods in Scala will change

[jira] [Commented] (SPARK-27681) Use scala.collection.Seq explicitly instead of scala.Seq alias

[ https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16838693#comment-16838693 ] Marcelo Vanzin commented on SPARK-27681: bq. scala.Seq is just a type def

[jira] [Comment Edited] (SPARK-27681) Use scala.collection.Seq explicitly instead of scala.Seq alias

[ https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16837752#comment-16837752 ] Marcelo Vanzin edited comment on SPARK-27681 at 5/11/19 4:28 AM: - Also

[jira] [Commented] (SPARK-27681) Use scala.collection.Seq explicitly instead of scala.Seq alias

[ https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16837753#comment-16837753 ] Marcelo Vanzin commented on SPARK-27681: bq. we don't need to do this until we support Scala

[jira] [Commented] (SPARK-27681) Use scala.collection.Seq explicitly instead of scala.Seq alias

[ https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16837752#comment-16837752 ] Marcelo Vanzin commented on SPARK-27681: Also the change you're proposing likely would break

[jira] [Commented] (SPARK-27681) Use scala.collection.Seq explicitly instead of scala.Seq alias

[ https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16837749#comment-16837749 ] Marcelo Vanzin commented on SPARK-27681: OOC, what happens if we do nothing? As far as I can

[jira] [Created] (SPARK-27678) Support Knox user impersonation in UI

Marcelo Vanzin created SPARK-27678: -- Summary: Support Knox user impersonation in UI Key: SPARK-27678 URL: https://issues.apache.org/jira/browse/SPARK-27678 Project: Spark Issue Type: New

[jira] [Resolved] (SPARK-26632) Separate Thread Configurations of Driver and Executor

[ https://issues.apache.org/jira/browse/SPARK-26632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26632. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23560 [https

[spark] branch master updated (c71f217 -> fa5dc0a)

This is an automated email from the ASF dual-hosted git repository. vanzin pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from c71f217 [SPARK-27673][SQL] Add `since` info to random, regex, null expressions add fa5dc0a [SPARK-26632][CORE

[jira] [Assigned] (SPARK-26632) Separate Thread Configurations of Driver and Executor

[ https://issues.apache.org/jira/browse/SPARK-26632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26632: -- Assignee: jiafu zhang > Separate Thread Configurations of Driver and Execu

[jira] [Assigned] (SPARK-27294) Multi-cluster Kafka delegation token support

[ https://issues.apache.org/jira/browse/SPARK-27294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-27294: -- Assignee: Gabor Somogyi > Multi-cluster Kafka delegation token supp

[jira] [Resolved] (SPARK-27294) Multi-cluster Kafka delegation token support

[ https://issues.apache.org/jira/browse/SPARK-27294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27294. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24305 [https

[spark] branch master updated: [SPARK-27294][SS] Add multi-cluster Kafka delegation token

This is an automated email from the ASF dual-hosted git repository. vanzin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 2f55809 [SPARK-27294][SS] Add multi-cluster

[jira] [Resolved] (SPARK-27610) Yarn external shuffle service fails to start when spark.shuffle.io.mode=EPOLL

[ https://issues.apache.org/jira/browse/SPARK-27610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27610. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24502 [https

[jira] [Assigned] (SPARK-27610) Yarn external shuffle service fails to start when spark.shuffle.io.mode=EPOLL

[ https://issues.apache.org/jira/browse/SPARK-27610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-27610: -- Assignee: Adrian Muraru > Yarn external shuffle service fails to start w

[spark] branch master updated (d124ce9 -> 8ef4da7)

This is an automated email from the ASF dual-hosted git repository. vanzin pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from d124ce9 [SPARK-27590][CORE] do not consider skipped tasks when scheduling speculative tasks add 8ef4da7

[jira] [Assigned] (SPARK-27194) Job failures when task attempts do not clean up spark-staging parquet files

[ https://issues.apache.org/jira/browse/SPARK-27194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-27194: -- Assignee: (was: Marcelo Vanzin) > Job failures when task attempts do not cl

[jira] [Assigned] (SPARK-27194) Job failures when task attempts do not clean up spark-staging parquet files

[ https://issues.apache.org/jira/browse/SPARK-27194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-27194: -- Assignee: Marcelo Vanzin > Job failures when task attempts do not clean up sp

[jira] [Resolved] (SPARK-16367) Wheelhouse Support for PySpark

[ https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-16367. Resolution: Duplicate This is somewhat similar to SPARK-13587 so let's keep

[jira] [Assigned] (SPARK-13587) Support virtualenv in PySpark

[ https://issues.apache.org/jira/browse/SPARK-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-13587: -- Assignee: Marcelo Vanzin > Support virtualenv in PySp

[jira] [Updated] (SPARK-13587) Support virtualenv in PySpark

[ https://issues.apache.org/jira/browse/SPARK-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-13587: --- Target Version/s: (was: 3.0.0) > Support virtualenv in PySp

[jira] [Assigned] (SPARK-13587) Support virtualenv in PySpark

[ https://issues.apache.org/jira/browse/SPARK-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-13587: -- Assignee: (was: Marcelo Vanzin) > Support virtualenv in PySp

[jira] [Resolved] (SPARK-27575) Spark overwrites existing value of spark.yarn.dist.* instead of merging value

[ https://issues.apache.org/jira/browse/SPARK-27575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27575. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24465 [https

[jira] [Assigned] (SPARK-27575) Spark overwrites existing value of spark.yarn.dist.* instead of merging value

[ https://issues.apache.org/jira/browse/SPARK-27575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-27575: -- Assignee: Jungtaek Lim > Spark overwrites existing value of spark.yarn.d

[spark] branch master updated: [SPARK-27575][CORE][YARN] Yarn file-related confs should merge new value with existing value

This is an automated email from the ASF dual-hosted git repository. vanzin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 75b40a5 [SPARK-27575][CORE][YARN] Yarn file

[jira] [Resolved] (SPARK-23014) Migrate MemorySink fully to v2

[ https://issues.apache.org/jira/browse/SPARK-23014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23014. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24403 [https

[jira] [Assigned] (SPARK-23014) Migrate MemorySink fully to v2

[ https://issues.apache.org/jira/browse/SPARK-23014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23014: -- Assignee: Gabor Somogyi > Migrate MemorySink fully to

[spark] branch master updated: [SPARK-23014][SS] Fully remove V1 memory sink.

This is an automated email from the ASF dual-hosted git repository. vanzin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new fb6b19a [SPARK-23014][SS] Fully remove V1

[jira] [Assigned] (SPARK-27477) Kafka token provider should have provided dependency on Spark

[ https://issues.apache.org/jira/browse/SPARK-27477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-27477: -- Assignee: koert kuipers > Kafka token provider should have provided depende

[jira] [Resolved] (SPARK-27477) Kafka token provider should have provided dependency on Spark

[ https://issues.apache.org/jira/browse/SPARK-27477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27477. Resolution: Fixed Issue resolved by pull request 24384 [https://github.com/apache/spark

[spark] branch master updated: [SPARK-27477][BUILD] Kafka token provider should have provided dependency on Spark

This is an automated email from the ASF dual-hosted git repository. vanzin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 7b367bf [SPARK-27477][BUILD] Kafka token

Re: Is it possible to obtain the full command to be invoked by SparkLauncher?

BTW the SparkLauncher API has hooks to capture the stderr of the spark-submit process into the logging system of the parent process. Check the API javadocs since it's been forever since I looked at that. On Wed, Apr 24, 2019 at 1:58 PM Marcelo Vanzin wrote: > > S

Re: Is it possible to obtain the full command to be invoked by SparkLauncher?

Setting the SPARK_PRINT_LAUNCH_COMMAND env variable to 1 in the launcher env will make Spark code print the command to stderr. Not optimal but I think it's the only current option. On Wed, Apr 24, 2019 at 1:55 PM Jeff Evans wrote: > > The org.apache.spark.launcher.SparkLauncher is used to

[jira] [Resolved] (SPARK-27515) [Deploy] When application master retry after a long time running, the hdfs delegation token may be expired

[ https://issues.apache.org/jira/browse/SPARK-27515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27515. Resolution: Duplicate > [Deploy] When application master retry after a long time runn

[jira] [Commented] (SPARK-11033) Launcher: add support for monitoring standalone/cluster apps

[ https://issues.apache.org/jira/browse/SPARK-11033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16819274#comment-16819274 ] Marcelo Vanzin commented on SPARK-11033: It's definitely not that simple. > Launcher:

[jira] [Commented] (SPARK-11033) Launcher: add support for monitoring standalone/cluster apps

[ https://issues.apache.org/jira/browse/SPARK-11033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16819211#comment-16819211 ] Marcelo Vanzin commented on SPARK-11033: I just filed the bug. I have no intention of working

[jira] [Assigned] (SPARK-27270) Add Kafka dynamic JAAS authentication debug possibility

[ https://issues.apache.org/jira/browse/SPARK-27270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-27270: -- Assignee: Gabor Somogyi > Add Kafka dynamic JAAS authentication debug possibil

[jira] [Resolved] (SPARK-27270) Add Kafka dynamic JAAS authentication debug possibility

[ https://issues.apache.org/jira/browse/SPARK-27270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27270. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24204 [https

[spark] branch master updated: [SPARK-27270][SS] Add Kafka dynamic JAAS authentication debug possibility

This is an automated email from the ASF dual-hosted git repository. vanzin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 94adffa [SPARK-27270][SS] Add Kafka dynamic

[jira] [Assigned] (LIVY-563) RSCConf take no effect in RSCDriver

[ https://issues.apache.org/jira/browse/LIVY-563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned LIVY-563: --- Assignee: Marcelo Vanzin > RSCConf take no effect in RSCDri

[jira] [Resolved] (LIVY-563) RSCConf take no effect in RSCDriver

[ https://issues.apache.org/jira/browse/LIVY-563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved LIVY-563. - Resolution: Fixed Fix Version/s: 0.7.0 Issue resolved by pull request 168 [https

[incubator-livy] branch master updated: [LIVY-563] Propagate RSC configuration when creating sessions.

This is an automated email from the ASF dual-hosted git repository. vanzin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-livy.git The following commit(s) were added to refs/heads/master by this push: new 5abc043 [LIVY-563] Propagate RSC

[jira] [Created] (LIVY-587) Remove Guava dependency

Marcelo Vanzin created LIVY-587: --- Summary: Remove Guava dependency Key: LIVY-587 URL: https://issues.apache.org/jira/browse/LIVY-587 Project: Livy Issue Type: Task Components: Core

[jira] [Resolved] (SPARK-27394) The staleness of UI may last minutes or hours when no tasks start or finish

[ https://issues.apache.org/jira/browse/SPARK-27394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27394. Resolution: Fixed Fix Version/s: 3.0.0 > The staleness of UI may last minu

[spark] branch master updated: [SPARK-27394][WEBUI] Flush LiveEntity if necessary when receiving SparkListenerExecutorMetricsUpdate

This is an automated email from the ASF dual-hosted git repository. vanzin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 5ff39cd [SPARK-27394][WEBUI] Flush LiveEntity

Re: Spark 2.4.0 tests fail with hadoop-3.1 profile: NoClassDefFoundError org.apache.hadoop.hive.conf.HiveConf

dependencies in the classpath, is that correct? > > On Fri, Apr 5, 2019 at 10:57 AM Marcelo Vanzin wrote: >> >> The hadoop-3 profile doesn't really work yet, not even on master. >> That's being worked on still. >> >> On Fri, Apr 5, 2019 at 10:53 AM akirillov >&

Re: Spark 2.4.0 tests fail with hadoop-3.1 profile: NoClassDefFoundError org.apache.hadoop.hive.conf.HiveConf

The hadoop-3 profile doesn't really work yet, not even on master. That's being worked on still. On Fri, Apr 5, 2019 at 10:53 AM akirillov wrote: > > Hi there! I'm trying to run Spark unit tests with the following profiles: > > And 'core' module fails with the following test failing with >

[jira] [Assigned] (LIVY-489) Expose a JDBC endpoint for Livy

[ https://issues.apache.org/jira/browse/LIVY-489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned LIVY-489: --- Assignee: Marco Gaido > Expose a JDBC endpoint for L

Re: [ANNOUNCE] Apache Livy 0.6.0-incubating released

com/artifact/org.apache.livy/livy-api > > On Wed, Apr 3, 2019 at 9:54 AM Marcelo Vanzin wrote: >> >> I closed the bug and will fix the website link. Thanks for pointing it out! >> >> On Wed, Apr 3, 2019 at 4:55 AM Alexey Romanenko >> wrote: >> > >> >

Re: [ANNOUNCE] Apache Livy 0.6.0-incubating released

07:05, Jeff Zhang wrote: > > Thanks Marcelo > > > 于2019年4月3日周三 下午12:15写道： >> >> Nice!! >> >> Sent from my iPhone >> >> On Apr 2, 2019, at 8:21 PM, Ruslan Dautkhanov wrote: >> >> Thanks a lot Marcelo ! >> >> Ruslan >>

[incubator-livy-website] branch master updated: Fix 0.6 tag link.

This is an automated email from the ASF dual-hosted git repository. vanzin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-livy-website.git The following commit(s) were added to refs/heads/master by this push: new f40a695 Fix 0.6 tag link

[incubator-livy-website] branch asf-site updated: Publishing from f40a6959add52283fd6f323bd16f94377a16e728

This is an automated email from the ASF dual-hosted git repository. vanzin pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-livy-website.git The following commit(s) were added to refs/heads/asf-site by this push: new 024e58f Publishing from

[jira] [Resolved] (LIVY-489) Expose a JDBC endpoint for Livy

[ https://issues.apache.org/jira/browse/LIVY-489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved LIVY-489. - Resolution: Done Fix Version/s: 0.6.0 > Expose a JDBC endpoint for L

[jira] [Resolved] (SPARK-11702) Guava ClassLoading Issue When Using Different Hive Metastore Version

[ https://issues.apache.org/jira/browse/SPARK-11702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-11702. Resolution: Cannot Reproduce Pretty sure this does not happen in the 2.x line. > Gu

[jira] [Resolved] (SPARK-25762) Upgrade guava version in spark dependency lists due to CVE issue

[ https://issues.apache.org/jira/browse/SPARK-25762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-25762. Resolution: Duplicate Target Version/s: (was: 3.0.0) > Upgrade guava vers

Re: [ANNOUNCE] Apache Livy 0.6.0-incubating released

(Apologies to all, this is the same announcement as previously, but including the missing disclaimer.) The Apache Livy team is proud to announce the release of Apache Livy 0.6.0-incubating. Livy is web service that exposes a REST interface for managing long running Apache Spark contexts in your

Re: [ANNOUNCE] Apache Livy 0.6.0-incubating released

(Apologies to all, this is the same announcement as previously, but including the missing disclaimer.) The Apache Livy team is proud to announce the release of Apache Livy 0.6.0-incubating. Livy is web service that exposes a REST interface for managing long running Apache Spark contexts in your

Re: [ANNOUNCE] Apache Livy 0.6.0-incubating released

Oops, my bad. Must have missed it when I copy & pasted a previous announcement. Will send an amended e-mail. On Tue, Apr 2, 2019 at 12:03 PM sebb wrote: > > On Tue, 2 Apr 2019 at 19:28, Marcelo Vanzin > wrote: > > > > The Apache Livy team is proud to announce t

[ANNOUNCE] Apache Livy 0.6.0-incubating released

The Apache Livy team is proud to announce the release of Apache Livy 0.6.0-incubating. Livy is web service that exposes a REST interface for managing long running Apache Spark contexts in your cluster. Livy enables programmatic, fault-tolerant, multi-tenant submission of Spark jobs from

[ANNOUNCE] Apache Livy 0.6.0-incubating released

The Apache Livy team is proud to announce the release of Apache Livy 0.6.0-incubating. Livy is web service that exposes a REST interface for managing long running Apache Spark contexts in your cluster. Livy enables programmatic, fault-tolerant, multi-tenant submission of Spark jobs from

[jira] [Resolved] (SPARK-26998) spark.ssl.keyStorePassword in plaintext on 'ps -ef' output of executor processes in Standalone mode

[ https://issues.apache.org/jira/browse/SPARK-26998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26998. Resolution: Fixed Fix Version/s: 2.3.4 2.4.2

[spark] branch branch-2.3 updated: [SPARK-26998][CORE] Remove SSL configuration from executors

This is an automated email from the ASF dual-hosted git repository. vanzin pushed a commit to branch branch-2.3 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-2.3 by this push: new 96c2c3b [SPARK-26998][CORE] Remove SSL

[jira] [Assigned] (SPARK-26998) spark.ssl.keyStorePassword in plaintext on 'ps -ef' output of executor processes in Standalone mode

[ https://issues.apache.org/jira/browse/SPARK-26998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26998: -- Assignee: Gabor Somogyi > spark.ssl.keyStorePassword in plaintext on 'ps -ef' out

[spark] branch branch-2.4 updated: [SPARK-26998][CORE] Remove SSL configuration from executors

This is an automated email from the ASF dual-hosted git repository. vanzin pushed a commit to branch branch-2.4 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-2.4 by this push: new 55e6f7a [SPARK-26998][CORE] Remove SSL

[spark] branch master updated: [SPARK-26998][CORE] Remove SSL configuration from executors

This is an automated email from the ASF dual-hosted git repository. vanzin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 57aff93 [SPARK-26998][CORE] Remove SSL

[incubator-livy] tag v0.6.0-incubating created (now 28be98c)

This is an automated email from the ASF dual-hosted git repository. vanzin pushed a change to tag v0.6.0-incubating in repository https://gitbox.apache.org/repos/asf/incubator-livy.git. at 28be98c (commit) No new revisions were added by this update.

Re: List of Projects that went straight to Top Level Projects

On Mon, Apr 1, 2019 at 1:08 PM Lars Francke wrote: > > Straight to TLP: > > Bahir, DRAT, Kibble, Orc, Serf, STeVe, Whimsy, Zest > > ORC was spun out of Hive as far as I remember. In that sense Bahir was also spun out of Spark, although it added a lot of things after that. -- Marcelo

[incubator-livy] branch master updated: [DOCS] Fix Javadoc 8 errors.

This is an automated email from the ASF dual-hosted git repository. vanzin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-livy.git The following commit(s) were added to refs/heads/master by this push: new d5b27dd [DOCS] Fix Javadoc 8 errors

svn commit: r33383 - /dev/incubator/livy/0.6.0-incubating-rc1/

Author: vanzin Date: Mon Apr 1 18:04:27 2019 New Revision: 33383 Log: Removing stale Livy release packages. Removed: dev/incubator/livy/0.6.0-incubating-rc1/

svn commit: r33382 - /release/incubator/livy/KEYS

Author: vanzin Date: Mon Apr 1 18:04:03 2019 New Revision: 33382 Log: Update Livy release KEYS. Modified: release/incubator/livy/KEYS Modified: release/incubator/livy/KEYS == --- release/incubator/livy/KEYS

<    1   2   3   4   5   6   7   8   9   10   >