svn commit: r27854 - /dev/spark/KEYS

Author: jshao Date: Mon Jul 2 12:18:41 2018 New Revision: 27854 Log: Update KEYS Modified: dev/spark/KEYS Modified: dev/spark/KEYS == --- dev/spark/KEYS (original) +++ dev/spark/KEYS Mon Jul 2 12:18:41 2018

spark git commit: [SPARK-24418][BUILD] Upgrade Scala to 2.11.12 and 2.12.6

Repository: spark Updated Branches: refs/heads/master 4c059ebc6 -> c7967c604 [SPARK-24418][BUILD] Upgrade Scala to 2.11.12 and 2.12.6 ## What changes were proposed in this pull request? Scala is upgraded to `2.11.12` and `2.12.6`. We used `loadFIles()` in `ILoop` as a hook to initialize the

incubator-livy-website git commit: Publishing from 293dcc865131d00fb66d7ed283544758a92653aa

Repository: incubator-livy-website Updated Branches: refs/heads/asf-site 579916d45 -> 281e56f2f Publishing from 293dcc865131d00fb66d7ed283544758a92653aa Project: http://git-wip-us.apache.org/repos/asf/incubator-livy-website/repo Commit:

incubator-livy-website git commit: Add a new link in third-party-projects.md

Repository: incubator-livy-website Updated Branches: refs/heads/master fc83685b3 -> 293dcc865 Add a new link in third-party-projects.md This pr added a new link in `third-party-projects.md`. I'm working on a Spark SQL server based on the PostgreSQL V3 protocol; it uses `Livy` for managing

incubator-livy git commit: [HOTFIX] Fix Base64 not existed in jdk7 issue

Repository: incubator-livy Updated Branches: refs/heads/master 219630273 -> 551dd5c27 [HOTFIX] Fix Base64 not existed in jdk7 issue Project: http://git-wip-us.apache.org/repos/asf/incubator-livy/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-livy/commit/551dd5c2 Tree:

incubator-livy git commit: [Security] Update to support pyspark and sparkr changes in Spark 2.3.1

Repository: incubator-livy Updated Branches: refs/heads/master fe0283f86 -> 219630273 [Security] Update to support pyspark and sparkr changes in Spark 2.3.1 Project: http://git-wip-us.apache.org/repos/asf/incubator-livy/repo Commit:

incubator-livy git commit: [LIVY-477][BUILD] Upgrade Scala version to 2.11.12

Repository: incubator-livy Updated Branches: refs/heads/master 02c5915f3 -> fe0283f86 [LIVY-477][BUILD] Upgrade Scala version to 2.11.12 ## What changes were proposed in this pull request? Scala version below 2.11.12 has CVE (https://scala-lang.org/news/security-update-nov17.html), and

spark git commit: [MINOR][YARN] Add YARN-specific credential providers in debug logging message

Repository: spark Updated Branches: refs/heads/master 21e1fc7d4 -> 2c9c8629b [MINOR][YARN] Add YARN-specific credential providers in debug logging message This PR adds a debugging log for YARN-specific credential providers which is loaded by service loader mechanism. It took me a while to

incubator-livy git commit: [LIVY-475] Support of Hadoop CredentialProvider API

Repository: incubator-livy Updated Branches: refs/heads/master 8b3fcbc8d -> 02c5915f3 [LIVY-475] Support of Hadoop CredentialProvider API In this PR I've added following option to livy.conf: ``` ``` to allow to specify path to Hadoop Credential Provider, which than used in `WebServer.scala`

spark git commit: [SPARK-23991][DSTREAMS] Fix data loss when WAL write fails in allocateBlocksToBatch

Repository: spark Updated Branches: refs/heads/branch-2.3 fec43fe1b -> 49a6c2b91 [SPARK-23991][DSTREAMS] Fix data loss when WAL write fails in allocateBlocksToBatch When blocks tried to get allocated to a batch and WAL write fails then the blocks will be removed from the received block

spark git commit: [SPARK-23991][DSTREAMS] Fix data loss when WAL write fails in allocateBlocksToBatch

Repository: spark Updated Branches: refs/heads/master 23db600c9 -> aca65c63c [SPARK-23991][DSTREAMS] Fix data loss when WAL write fails in allocateBlocksToBatch When blocks tried to get allocated to a batch and WAL write fails then the blocks will be removed from the received block queue.

incubator-livy git commit: [LIVY-473][TEST] Minor refactor of integration test to remove legacy code

Repository: incubator-livy Updated Branches: refs/heads/master ca4cad229 -> 8b3fcbc8d [LIVY-473][TEST] Minor refactor of integration test to remove legacy code Integration test has some legacy codes related to different cluster type (mini or real). Since now we already use real Spark package

incubator-livy git commit: [LIVY-472][SERVER] Improve the logs for fail-to-create session

Repository: incubator-livy Updated Branches: refs/heads/master 6f1756cff -> ca4cad229 [LIVY-472][SERVER] Improve the logs for fail-to-create session ## What changes were proposed in this pull request? Livy currently doesn't give a very clear log about the fail-to-create session, it only

incubator-livy git commit: [LIVY-472][SERVER] Improve the logs for fail-to-create session

Repository: incubator-livy Updated Branches: refs/heads/branch-0.5 8c9698113 -> 916255a9f [LIVY-472][SERVER] Improve the logs for fail-to-create session ## What changes were proposed in this pull request? Livy currently doesn't give a very clear log about the fail-to-create session, it only

incubator-livy git commit: [LIVY-452] Differentiate FAILED and KILLED states

Repository: incubator-livy Updated Branches: refs/heads/master e3f45a057 -> 551cc5309 [LIVY-452] Differentiate FAILED and KILLED states Currently, it's not possible to distinguish between two states - SparkApp.State.KILLED and SparkApp.State.FAILED. In both cases the session state will be

spark git commit: [SPARK-24241][SUBMIT] Do not fail fast when dynamic resource allocation enabled with 0 executor

Repository: spark Updated Branches: refs/heads/master 80c6d35a3 -> 4a2b15f0a [SPARK-24241][SUBMIT] Do not fail fast when dynamic resource allocation enabled with 0 executor ## What changes were proposed in this pull request? ``` ~/spark-2.3.0-bin-hadoop2.7$ bin/spark-sql --num-executors 0

spark git commit: [SPARK-24188][CORE] Restore "/version" API endpoint.

Repository: spark Updated Branches: refs/heads/master cd12c5c3e -> 05eb19b6e [SPARK-24188][CORE] Restore "/version" API endpoint. It was missing the jax-rs annotation. Author: Marcelo Vanzin Closes #21245 from vanzin/SPARK-24188. Change-Id:

spark git commit: [SPARK-24188][CORE] Restore "/version" API endpoint.

Repository: spark Updated Branches: refs/heads/branch-2.3 4dc6719e9 -> aba52f449 [SPARK-24188][CORE] Restore "/version" API endpoint. It was missing the jax-rs annotation. Author: Marcelo Vanzin Closes #21245 from vanzin/SPARK-24188. Change-Id:

spark git commit: [SPARK-24136][SS] Fix MemoryStreamDataReader.next to skip sleeping if record is available

Repository: spark Updated Branches: refs/heads/master 0c23e254c -> 7f1b6b182 [SPARK-24136][SS] Fix MemoryStreamDataReader.next to skip sleeping if record is available ## What changes were proposed in this pull request? Avoid unnecessary sleep (10 ms) in each invocation of

incubator-livy git commit: [LIVY-466][RSC] Fix RSCDriver exception during RPC shutdown

Repository: incubator-livy Updated Branches: refs/heads/master 9d381bdf0 -> e3f45a057 [LIVY-466][RSC] Fix RSCDriver exception during RPC shutdown ## What changes were proposed in this pull request? During RSCDriver's shutdown, it will first shutdown RPC server, and then all the RPC clients.

incubator-livy git commit: [LIVY-466][RSC] Fix RSCDriver exception during RPC shutdown

Repository: incubator-livy Updated Branches: refs/heads/branch-0.5 bdb954798 -> 8c9698113 [LIVY-466][RSC] Fix RSCDriver exception during RPC shutdown ## What changes were proposed in this pull request? During RSCDriver's shutdown, it will first shutdown RPC server, and then all the RPC

spark git commit: [SPARK-24110][THRIFT-SERVER] Avoid UGI.loginUserFromKeytab in STS

Repository: spark Updated Branches: refs/heads/master e4c91c089 -> bf4352ca6 [SPARK-24110][THRIFT-SERVER] Avoid UGI.loginUserFromKeytab in STS ## What changes were proposed in this pull request? Spark ThriftServer will call UGI.loginUserFromKeytab twice in initialization. This is

spark git commit: [SPARK-22732][SS][FOLLOW-UP] Fix MemorySinkV2 toString error

Repository: spark Updated Branches: refs/heads/master ad94e8592 -> 4df51361a [SPARK-22732][SS][FOLLOW-UP] Fix MemorySinkV2 toString error ## What changes were proposed in this pull request? Fix `MemorySinkV2` toString() error ## How was this patch tested? N/A Author: Yuming Wang

spark git commit: [SPARK-23688][SS] Refactor tests away from rate source

Repository: spark Updated Branches: refs/heads/master 8614edd44 -> 1fb46f30f [SPARK-23688][SS] Refactor tests away from rate source ## What changes were proposed in this pull request? Replace rate source with memory source in continuous mode test suite. Keep using "rate" source if the tests

spark git commit: [SPARK-23830][YARN] added check to ensure main method is found

Repository: spark Updated Branches: refs/heads/master 8aa1d7b0e -> 109935fc5 [SPARK-23830][YARN] added check to ensure main method is found ## What changes were proposed in this pull request? When a user specifies the wrong class -- or, in fact, a class instead of an object -- Spark throws

spark git commit: [SPARK-24062][THRIFT SERVER] Fix SASL encryption cannot enabled issue in thrift server

Repository: spark Updated Branches: refs/heads/master cd10f9df8 -> ffaf0f9fd [SPARK-24062][THRIFT SERVER] Fix SASL encryption cannot enabled issue in thrift server ## What changes were proposed in this pull request? For the details of the exception please see

spark git commit: [SPARK-24062][THRIFT SERVER] Fix SASL encryption cannot enabled issue in thrift server

Repository: spark Updated Branches: refs/heads/branch-2.3 096defdd7 -> 07ec75ca0 [SPARK-24062][THRIFT SERVER] Fix SASL encryption cannot enabled issue in thrift server ## What changes were proposed in this pull request? For the details of the exception please see

incubator-livy git commit: [LIVY-460][SERVER] Remove misleading log prefix

Repository: incubator-livy Updated Branches: refs/heads/master 71c3feed9 -> 9d381bdf0 [LIVY-460][SERVER] Remove misleading log prefix ## What changes were proposed in this pull request? Removing misleading log prefix "stdout: " for the captured stdout/stderr of spark-submit process ## How

spark git commit: [SPARK-24014][PYSPARK] Add onStreamingStarted method to StreamingListener

Repository: spark Updated Branches: refs/heads/master 0c94e48bc -> 8bb0df2c6 [SPARK-24014][PYSPARK] Add onStreamingStarted method to StreamingListener ## What changes were proposed in this pull request? The `StreamingListener` in PySpark side seems to be lack of `onStreamingStarted` method.

spark git commit: [SPARK-24014][PYSPARK] Add onStreamingStarted method to StreamingListener

Repository: spark Updated Branches: refs/heads/branch-2.3 130641102 -> 32bec6ca3 [SPARK-24014][PYSPARK] Add onStreamingStarted method to StreamingListener ## What changes were proposed in this pull request? The `StreamingListener` in PySpark side seems to be lack of `onStreamingStarted`

incubator-livy git commit: [LIVY-458][BUILD] Upgrade jackson version to 2.9.5

Repository: incubator-livy Updated Branches: refs/heads/master cd8b11261 -> 71c3feed9 [LIVY-458][BUILD] Upgrade jackson version to 2.9.5 ## What changes were proposed in this pull request? Due to several security issues of jackson databind module (CVE-2018-5968, CVE-2017-17485,

incubator-livy git commit: [LIVY-457][REPL] Fix SQLContext is not initialized correctly issue

Repository: incubator-livy Updated Branches: refs/heads/branch-0.5 88d66a95f -> bdb954798 [LIVY-457][REPL] Fix SQLContext is not initialized correctly issue ## What changes were proposed in this pull request? The signature of SQLContext's constructor is changed in Spark2, but we're still

incubator-livy git commit: [LIVY-457][REPL] Fix SQLContext is not initialized correctly issue

Repository: incubator-livy Updated Branches: refs/heads/master 7e4bb3bd6 -> cd8b11261 [LIVY-457][REPL] Fix SQLContext is not initialized correctly issue ## What changes were proposed in this pull request? The signature of SQLContext's constructor is changed in Spark2, but we're still using

spark git commit: [SPARK-23956][YARN] Use effective RPC port in AM registration

Repository: spark Updated Branches: refs/heads/master 73f28530d -> c0964935d [SPARK-23956][YARN] Use effective RPC port in AM registration ## What changes were proposed in this pull request? We propose not to hard-code the RPC port in the AM registration. ## How was this patch tested?

incubator-livy git commit: [LIVY-455][REPL] Fix json4s doesn't support java.math.BigDecimal issue

Repository: incubator-livy Updated Branches: refs/heads/branch-0.5 4783fbc25 -> 88d66a95f [LIVY-455][REPL] Fix json4s doesn't support java.math.BigDecimal issue ## What changes were proposed in this pull request? Livy's SQLInterpreter will throw exception when rows contain

incubator-livy git commit: [LIVY-455][REPL] Fix json4s doesn't support java.math.BigDecimal issue

Repository: incubator-livy Updated Branches: refs/heads/master 06cfa7bce -> 7e4bb3bd6 [LIVY-455][REPL] Fix json4s doesn't support java.math.BigDecimal issue ## What changes were proposed in this pull request? Livy's SQLInterpreter will throw exception when rows contain java.math.BigDecimal

incubator-livy git commit: [MINOR] Fix travis builds

Repository: incubator-livy Updated Branches: refs/heads/branch-0.5 1f5841ee7 -> 4783fbc25 [MINOR] Fix travis builds Each individual commit has a more detailed description of what's being changed and why. At the moment, Travis builds don't work: *

incubator-livy git commit: [MINOR] Fix travis builds

Repository: incubator-livy Updated Branches: refs/heads/master 10373b6e2 -> 06cfa7bce [MINOR] Fix travis builds Each individual commit has a more detailed description of what's being changed and why. At the moment, Travis builds don't work: *

spark git commit: [SPARK-23743][SQL] Changed a comparison logic from containing 'slf4j' to starting with 'org.slf4j'

Repository: spark Updated Branches: refs/heads/master b34890119 -> df05fb63a [SPARK-23743][SQL] Changed a comparison logic from containing 'slf4j' to starting with 'org.slf4j' ## What changes were proposed in this pull request? isSharedClass returns if some classes can/should be shared or

spark git commit: [SPARK-23787][TESTS] Fix file download test in SparkSubmitSuite for Hadoop 2.9.

Repository: spark Updated Branches: refs/heads/master 087fb3142 -> eb48edf9c [SPARK-23787][TESTS] Fix file download test in SparkSubmitSuite for Hadoop 2.9. This particular test assumed that Hadoop libraries did not support http as a file system. Hadoop 2.9 does, so the test failed. The test

spark git commit: [SPARK-23361][YARN] Allow AM to restart after initial tokens expire.

Repository: spark Updated Branches: refs/heads/master b2edc30db -> 5fa438471 [SPARK-23361][YARN] Allow AM to restart after initial tokens expire. Currently, the Spark AM relies on the initial set of tokens created by the submission client to be able to talk to HDFS and other services that

spark git commit: [SPARK-23644][CORE][UI][BACKPORT-2.3] Use absolute path for REST call in SHS

Repository: spark Updated Branches: refs/heads/branch-2.3 5c1c03d08 -> 2f82c037d [SPARK-23644][CORE][UI][BACKPORT-2.3] Use absolute path for REST call in SHS ## What changes were proposed in this pull request? SHS is using a relative path for the REST API call to get the list of the

spark git commit: [SPARK-23708][CORE] Correct comment for function addShutDownHook in ShutdownHookManager

Repository: spark Updated Branches: refs/heads/master 61487b308 -> 745c8c090 [SPARK-23708][CORE] Correct comment for function addShutDownHook in ShutdownHookManager ## What changes were proposed in this pull request? Minor modification.Comment below is not right. ``` /** * Adds a shutdown

spark git commit: [SPARK-23635][YARN] AM env variable should not overwrite same name env variable set through spark.executorEnv.

Repository: spark Updated Branches: refs/heads/master ca83526de -> c95200048 [SPARK-23635][YARN] AM env variable should not overwrite same name env variable set through spark.executorEnv. ## What changes were proposed in this pull request? In the current Spark on YARN code, AM always will

spark git commit: [SPARK-23644][CORE][UI] Use absolute path for REST call in SHS

Repository: spark Updated Branches: refs/heads/master c2632edeb -> ca83526de [SPARK-23644][CORE][UI] Use absolute path for REST call in SHS ## What changes were proposed in this pull request? SHS is using a relative path for the REST API call to get the list of the application is a relative

incubator-livy git commit: [LIVY-446][BUILD] Livy to Support Spark 2.3

Repository: incubator-livy Updated Branches: refs/heads/master b9c2f1019 -> 10373b6e2 [LIVY-446][BUILD] Livy to Support Spark 2.3 ## What changes were proposed in this pull request? This PR aims to support Spark 2.3 in Livy, several changes due to internal changes of Spark 2.3 ## How was

incubator-livy git commit: [LIVY-448] Assembly format is duplicated in pom.xml and assembly.xml

Repository: incubator-livy Updated Branches: refs/heads/master 6d2ffdd54 -> b9c2f1019 [LIVY-448] Assembly format is duplicated in pom.xml and assembly.xml ## What changes were proposed in this pull request? The assembly format for Livy is specified twice: 1 in `assembly/pom.xml`: ```xml

spark git commit: [MINOR][YARN] Add disable yarn.nodemanager.vmem-check-enabled option to memLimitExceededLogMessage

Repository: spark Updated Branches: refs/heads/master 4c587eb48 -> 04e71c316 [MINOR][YARN] Add disable yarn.nodemanager.vmem-check-enabled option to memLimitExceededLogMessage My spark application sometimes will throw `Container killed by YARN for exceeding memory limits`. Even I increased

spark git commit: Revert "[SPARK-23200] Reset Kubernetes-specific config on Checkpoint restore"

Repository: spark Updated Branches: refs/heads/master b6b50efc8 -> 4b7cd479a Revert "[SPARK-23200] Reset Kubernetes-specific config on Checkpoint restore" This reverts commit d1721816d26bedee3c72eeb75db49da500568376. The patch is not fully tested and out-of-date. So revert it. Project:

spark git commit: [SPARK-23279][SS] Avoid triggering distributed job for Console sink

Repository: spark Updated Branches: refs/heads/master ca04c3ff2 -> 8c6a9c90a [SPARK-23279][SS] Avoid triggering distributed job for Console sink ## What changes were proposed in this pull request? Console sink will redistribute collected local data and trigger a distributed job in each

spark git commit: [SPARK-23279][SS] Avoid triggering distributed job for Console sink

Repository: spark Updated Branches: refs/heads/branch-2.3 b8778321b -> ab5a51055 [SPARK-23279][SS] Avoid triggering distributed job for Console sink ## What changes were proposed in this pull request? Console sink will redistribute collected local data and trigger a distributed job in each

spark git commit: [SPARK-23088][CORE] History server not showing incomplete/running applications

Repository: spark Updated Branches: refs/heads/master f235df66a -> 31bd1dab1 [SPARK-23088][CORE] History server not showing incomplete/running applications ## What changes were proposed in this pull request? History server not showing incomplete/running applications when

spark git commit: [SPARK-23200] Reset Kubernetes-specific config on Checkpoint restore

Repository: spark Updated Branches: refs/heads/master 70a68b328 -> d1721816d [SPARK-23200] Reset Kubernetes-specific config on Checkpoint restore ## What changes were proposed in this pull request? When using the Kubernetes cluster-manager and spawning a Streaming workload, it is important

spark git commit: [MINOR][DOC] Fix the path to the examples jar

Repository: spark Updated Branches: refs/heads/branch-2.3 57c320a0d -> cf078a205 [MINOR][DOC] Fix the path to the examples jar ## What changes were proposed in this pull request? The example jar file is now in ./examples/jars directory of Spark distribution. Author: Arseniy Tashoyan

spark git commit: [MINOR][DOC] Fix the path to the examples jar

Repository: spark Updated Branches: refs/heads/master ec2289761 -> 60175e959 [MINOR][DOC] Fix the path to the examples jar ## What changes were proposed in this pull request? The example jar file is now in ./examples/jars directory of Spark distribution. Author: Arseniy Tashoyan

spark git commit: [SPARK-22976][CORE] Cluster mode driver dir removed while running

Repository: spark Updated Branches: refs/heads/branch-2.3 7520491bf -> 5781fa79e [SPARK-22976][CORE] Cluster mode driver dir removed while running ## What changes were proposed in this pull request? The clean up logic on the worker perviously determined the liveness of a particular

spark git commit: [SPARK-22976][CORE] Cluster mode driver dir removed while running

Repository: spark Updated Branches: refs/heads/master 602c6d82d -> 11daeb833 [SPARK-22976][CORE] Cluster mode driver dir removed while running ## What changes were proposed in this pull request? The clean up logic on the worker perviously determined the liveness of a particular applicaiton

spark git commit: [SPARK-22587] Spark job fails if fs.defaultFS and application jar are different url

Repository: spark Updated Branches: refs/heads/branch-2.3 551ccfba5 -> 317b0aaed [SPARK-22587] Spark job fails if fs.defaultFS and application jar are different url ## What changes were proposed in this pull request? Two filesystems comparing does not consider the authority of URI. This is

spark git commit: [SPARK-22587] Spark job fails if fs.defaultFS and application jar are different url

Repository: spark Updated Branches: refs/heads/master 9b33dfc40 -> a6647ffbf [SPARK-22587] Spark job fails if fs.defaultFS and application jar are different url ## What changes were proposed in this pull request? Two filesystems comparing does not consider the authority of URI. This is

incubator-livy git commit: [LIVY-429] preserve url fragment in spark.yarn.dist.archives

Repository: incubator-livy Updated Branches: refs/heads/master e54318314 -> 85e8fba3c [LIVY-429] preserve url fragment in spark.yarn.dist.archives ## What changes were proposed in this pull request? Preserve url fragment identifier in spark.yarn.dist.archives setting when there is no schema

incubator-livy git commit: [LIVY-19][REPL] Add SQL interpreter for Livy interactive session

Repository: incubator-livy Updated Branches: refs/heads/master 1efc80180 -> e54318314 [LIVY-19][REPL] Add SQL interpreter for Livy interactive session ## What changes were proposed in this pull request? With the requirement to support executing SQL query in Livy interactive session. Here

incubator-livy git commit: [LIVY-415][CORE][FOLLOWUP] Fix static object creation issue for SessionState

Repository: incubator-livy Updated Branches: refs/heads/master cd777e209 -> 1efc80180 [LIVY-415][CORE][FOLLOWUP] Fix static object creation issue for SessionState ## What changes were proposed in this pull request? In LIVY-415, we changed `SessionState` to static object instead of case

incubator-livy git commit: [LIVY-399][TEST] Enable real test for PySpark and SparkR interpreters

Repository: incubator-livy Updated Branches: refs/heads/master f893d1991 -> cd777e209 [LIVY-399][TEST] Enable real test for PySpark and SparkR interpreters Currently because we lack pyspark and sparkr dependencies in our environment, so we neglect the pyspark and sparkr integration tests,

incubator-livy git commit: [LIVY-416] Upgrade the version of Jackson from 2.4.4 to 2.9.2

Repository: incubator-livy Updated Branches: refs/heads/master 64d71aa75 -> 1f59e102b [LIVY-416] Upgrade the version of Jackson from 2.4.4 to 2.9.2 ## What changes were proposed in this pull request? https://issues.apache.org/jira/browse/LIVY-416

incubator-livy git commit: [LIVY-418] Respect spark.pyspark.python for PythonInterpreter

Repository: incubator-livy Updated Branches: refs/heads/master 54715447e -> 64d71aa75 [LIVY-418] Respect spark.pyspark.python for PythonInterpreter ## What changes were proposed in this pull request? `spark.pyspark.python` is introduced in spark 2.1, livy also need to update its code for

spark git commit: [SPARK-22172][CORE] Worker hangs when the external shuffle service port is already in use

Repository: spark Updated Branches: refs/heads/master 556b5d215 -> 96798d14f [SPARK-22172][CORE] Worker hangs when the external shuffle service port is already in use ## What changes were proposed in this pull request? Handling the NonFatal exceptions while starting the external shuffle

spark git commit: [SPARK-21840][CORE] Add trait that allows conf to be directly set in application.

Repository: spark Updated Branches: refs/heads/master 592cfeab9 -> 3073344a2 [SPARK-21840][CORE] Add trait that allows conf to be directly set in application. Currently SparkSubmit uses system properties to propagate configuration to applications. This makes it hard to implement features

incubator-livy git commit: [MINOR][HOTFIX] Fix python code style issue

Repository: incubator-livy Updated Branches: refs/heads/branch-0.4 939c97673 -> 7556d5f6a [MINOR][HOTFIX] Fix python code style issue ## What changes were proposed in this pull request? Travis complains "E722 do not use bare except" in python-client style check. Minor changes to fix it.

spark git commit: [SPARK-22319][CORE][BACKPORT-2.2] call loginUserFromKeytab before accessing hdfs

Repository: spark Updated Branches: refs/heads/branch-2.2 f8c83fdc5 -> bf8163f5b [SPARK-22319][CORE][BACKPORT-2.2] call loginUserFromKeytab before accessing hdfs In SparkSubmit, call loginUserFromKeytab before attempting to make RPC calls to the NameNode. Same as

spark git commit: [SPARK-22319][CORE] call loginUserFromKeytab before accessing hdfs

Repository: spark Updated Branches: refs/heads/master ca2a780e7 -> 57accf6e3 [SPARK-22319][CORE] call loginUserFromKeytab before accessing hdfs In `SparkSubmit`, call `loginUserFromKeytab` before attempting to make RPC calls to the NameNode. I manually tested this patch by: 1. Confirming

incubator-livy git commit: [LIVY-245][RSC] Add support shared variables across Jobs

Repository: incubator-livy Updated Branches: refs/heads/master 1bbefe601 -> bef5b4f6d [LIVY-245][RSC] Add support shared variables across Jobs Currently we cannot share variables across different Jobs in Livy, so here propose to add a cache layer in RSC to store shared objects. This cache

spark git commit: [SPARK-22290][CORE] Avoid creating Hive delegation tokens when not necessary.

Repository: spark Updated Branches: refs/heads/master 6f1d0dea1 -> dc2714da5 [SPARK-22290][CORE] Avoid creating Hive delegation tokens when not necessary. Hive delegation tokens are only needed when the Spark driver has no access to the kerberos TGT. That happens only in two situations: -

incubator-livy git commit: [LIVY-7] added autocompletion api and implementation for scala

Repository: incubator-livy Updated Branches: refs/heads/master ec069e32f -> 8056e4682 [LIVY-7] added autocompletion api and implementation for scala I started an implementation of the very old feature request (LIVY-7) for code autocompletion. This implementation works with scala 2.11 and

spark git commit: [SPARK-22074][CORE] Task killed by other attempt task should not be resubmitted

Repository: spark Updated Branches: refs/heads/master c998a2ae0 -> fe7b219ae [SPARK-22074][CORE] Task killed by other attempt task should not be resubmitted ## What changes were proposed in this pull request? As the detail scenario described in

incubator-livy git commit: [LIVY-408][RSC] Update Netty version to 4.0.37.Final

Repository: incubator-livy Updated Branches: refs/heads/master 0ca16d5d2 -> ff6c2ebbb [LIVY-408][RSC] Update Netty version to 4.0.37.Final Netty version below 4.0.37.Final has some potential security issue (https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-4970), it is fixed in this

spark git commit: [SPARK-22135][MESOS] metrics in spark-dispatcher not being registered properly

Repository: spark Updated Branches: refs/heads/branch-2.2 42e172744 -> 12a74e352 [SPARK-22135][MESOS] metrics in spark-dispatcher not being registered properly ## What changes were proposed in this pull request? Fix a trivial bug with how metrics are registered in the mesos dispatcher. Bug

spark git commit: [SPARK-22135][MESOS] metrics in spark-dispatcher not being registered properly

Repository: spark Updated Branches: refs/heads/master 3b117d631 -> f20be4d70 [SPARK-22135][MESOS] metrics in spark-dispatcher not being registered properly ## What changes were proposed in this pull request? Fix a trivial bug with how metrics are registered in the mesos dispatcher. Bug

spark git commit: [SPARK-22123][CORE] Add latest failure reason for task set blacklist

Repository: spark Updated Branches: refs/heads/master 7bf4da8a3 -> 3b117d631 [SPARK-22123][CORE] Add latest failure reason for task set blacklist ## What changes were proposed in this pull request? This patch add latest failure reason for task set blacklist.Which can be showed on spark ui

spark git commit: [SAPRK-20785][WEB-UI][SQL] Spark should provide jump links and add (count) in the SQL web ui.

Repository: spark Updated Branches: refs/heads/master 74daf622d -> d2b8b63b9 [SAPRK-20785][WEB-UI][SQL] Spark should provide jump links and add (count) in the SQL web ui. ## What changes were proposed in this pull request? propose: it provide links that jump to Running Queries,Completed

incubator-livy git commit: [LIVY-406][LIVY-406][FOLLOWUP][SERVER] Fix UT failure in branch 0.4 and apply to Master branch for consistency

Repository: incubator-livy Updated Branches: refs/heads/branch-0.4 3d70104cf -> 939c97673 [LIVY-406][LIVY-406][FOLLOWUP][SERVER] Fix UT failure in branch 0.4 and apply to Master branch for consistency Because we changed the AppState logic in #39 , so the UT which passed in master branch

incubator-livy git commit: [LIVY-406][SERVER] Fix Livy cannot app id in yarn client mode issue

Repository: incubator-livy Updated Branches: refs/heads/master 1bd92b9f4 -> 4a537e24d [LIVY-406][SERVER] Fix Livy cannot app id in yarn client mode issue In our `SparkYarnApp` logic, we have to wait for process to exit before to query app id. Since the process is never exited in yarn client

spark git commit: [SPARK-21934][CORE] Expose Shuffle Netty memory usage to MetricsSystem

Repository: spark Updated Branches: refs/heads/master 352bea545 -> 1da5822e6 [SPARK-21934][CORE] Expose Shuffle Netty memory usage to MetricsSystem ## What changes were proposed in this pull request? This is a followup work of SPARK-9104 to expose the Netty memory usage to MetricsSystem.

spark git commit: [SPARK-22030][CORE] GraphiteSink fails to re-connect to Graphite instances behind an ELB or any other auto-scaled LB

Repository: spark Updated Branches: refs/heads/master c66d64b3d -> 94f7e046a [SPARK-22030][CORE] GraphiteSink fails to re-connect to Graphite instances behind an ELB or any other auto-scaled LB ## What changes were proposed in this pull request? Upgrade codahale metrics library so that

incubator-livy git commit: [MINOR] Update Travis Python Requests version to fix build error

Repository: incubator-livy Updated Branches: refs/heads/branch-0.4 118d10d77 -> 844275ccb [MINOR] Update Travis Python Requests version to fix build error The issue is from #39 , seems like `requests` version doesn't meet, so here update `requests` version to fix it. Author: jerryshao

spark git commit: [SPARK-21902][CORE] Print root cause for BlockManager#doPut

Repository: spark Updated Branches: refs/heads/master 88661747f -> 22b111ef9 [SPARK-21902][CORE] Print root cause for BlockManager#doPut ## What changes were proposed in this pull request? As logging below, actually exception will be hidden when removeBlockInternal throw an exception.

spark git commit: [SPARK-21922] Fix duration always updating when task failed but status is still RUN…

Repository: spark Updated Branches: refs/heads/master 4e6fc6901 -> 4b88393cb [SPARK-21922] Fix duration always updating when task failed but status is still RUN… …NING ## What changes were proposed in this pull request? When driver quit abnormally which cause executor shutdown and task

spark git commit: [SPARK-21939][TEST] Use TimeLimits instead of Timeouts

Repository: spark Updated Branches: refs/heads/master e00f1a1da -> c26976fe1 [SPARK-21939][TEST] Use TimeLimits instead of Timeouts Since ScalaTest 3.0.0, `org.scalatest.concurrent.Timeouts` is deprecated. This PR replaces the deprecated one with `org.scalatest.concurrent.TimeLimits`.

spark git commit: [SPARK-18061][THRIFTSERVER] Add spnego auth support for ThriftServer thrift/http protocol

Repository: spark Updated Branches: refs/heads/master 9e451bcf3 -> 6a2325448 [SPARK-18061][THRIFTSERVER] Add spnego auth support for ThriftServer thrift/http protocol Spark ThriftServer doesn't support spnego auth for thrift/http protocol, this mainly used for knox+thriftserver scenario.

incubator-livy-website git commit: [MINOR] Updated tar.gz to zip on download page

Repository: incubator-livy-website Updated Branches: refs/heads/master 46e39ee66 -> e3317a823 [MINOR] Updated tar.gz to zip on download page Followup to #8 Accidentally left the tar.gz naming after switching the file type to zip in the links. Author: Alex Bozarth

incubator-livy-website git commit: [LIVY-389][RELEASE] 0.4.0-incubating release update

Repository: incubator-livy-website Updated Branches: refs/heads/master 5a044aefa -> 46e39ee66 [LIVY-389][RELEASE] 0.4.0-incubating release update Website updates for 0.4.0-incubating, including download page, history page and 0.4.0-incubating release post. Author: Alex Bozarth

[1/2] incubator-livy-website git commit: Publishing from 5a044aefa4de8b31a1d94f65afdd3eaaef70911d

Repository: incubator-livy-website Updated Branches: refs/heads/asf-site 0ac7686f6 -> 98e1c540b http://git-wip-us.apache.org/repos/asf/incubator-livy-website/blob/98e1c540/content/docs/latest/rest-api.html -- diff --git

[2/2] incubator-livy-website git commit: Publishing from 5a044aefa4de8b31a1d94f65afdd3eaaef70911d

Publishing from 5a044aefa4de8b31a1d94f65afdd3eaaef70911d Project: http://git-wip-us.apache.org/repos/asf/incubator-livy-website/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-livy-website/commit/98e1c540 Tree:

incubator-livy-website git commit: [LIVY-384][DOCS] Include local build of Docs

Repository: incubator-livy-website Updated Branches: refs/heads/master 315b22db2 -> 5a044aefa [LIVY-384][DOCS] Include local build of Docs [LIVY-384](https://issues.cloudera.org/browse/LIVY-384) Included a copy of the 0.4.0-incubating build of the Livy Docs in the website. The process for

incubator-livy git commit: [LIVY-394][BUILD] Update branch 0.4 to version 0.4.1-incubating-SNAPSHOT

Repository: incubator-livy Updated Branches: refs/heads/branch-0.4 0e30800a4 -> 5fa53c3d5 [LIVY-394][BUILD] Update branch 0.4 to version 0.4.1-incubating-SNAPSHOT This PR updates branch-0.4 Livy version to 0.4.1-incubating-SNAPSHOT. Author: jerryshao Closes #41 from

incubator-livy git commit: [LIVY-394][BUILD] Update Livy master to 0.5.0-incubating-SNAPSHOT

Repository: incubator-livy Updated Branches: refs/heads/master c1aafeb6c -> a23969721 [LIVY-394][BUILD] Update Livy master to 0.5.0-incubating-SNAPSHOT This PR updates Livy master branch version to 0.5.0-incubating-SNAPSHOT, also changes the Livy package name to livy--bin.zip. Author:

[incubator-livy] Git Push Summary

Repository: incubator-livy Updated Tags: refs/tags/v0.4.0-incubating-rc1 [deleted] 02e4030c7

[incubator-livy] Git Push Summary

Repository: incubator-livy Updated Tags: refs/tags/v0.4.0-incubating-rc2 [deleted] 61ad80b43

[incubator-livy] Git Push Summary

Repository: incubator-livy Updated Branches: refs/heads/gh-pages [deleted] fd20a236a

[incubator-livy] Git Push Summary

Repository: incubator-livy Updated Branches: refs/heads/revert-237-LIVY-255 [deleted] 2da435ad1

[incubator-livy] Git Push Summary

Repository: incubator-livy Updated Branches: refs/heads/rel-0.2.0-refresh [deleted] e9256c27f

<    1   2   3   4   5   >