Repository: spark
Updated Branches:
refs/heads/master 19530da69 -> 81012546e
[SPARK-11872] Prevent the call to SparkContext#stop() in the listener bus's
thread
This is continuation of SPARK-11761
Andrew suggested adding this protection. See tail of
https://github.com/apache/spark/pull/9741
Repository: spark
Updated Branches:
refs/heads/branch-1.6 927070d6d -> 0419fd361
[SPARK-11872] Prevent the call to SparkContext#stop() in the listener bus's
thread
This is continuation of SPARK-11761
Andrew suggested adding this protection. See tail of
https://github.com/apache/spark/pull/9
rt/junit/org.apache.spark.streaming.util/BatchedWriteAheadLogWithCloseFileAfterWriteSuite/BatchedWriteAheadLog___clean_old_logs/
The reason the test fails is in `afterEach`, `writeAheadLog.close` is called,
and there may still be async deletes in flight.
tdas zsxwing
Author: Burak Yavuz
Closes #9953 from brky
rt/junit/org.apache.spark.streaming.util/BatchedWriteAheadLogWithCloseFileAfterWriteSuite/BatchedWriteAheadLog___clean_old_logs/
The reason the test fails is in `afterEach`, `writeAheadLog.close` is called,
and there may still be async deletes in flight.
tdas zsxwing
Author: Burak Yavuz
Closes #9953 fro
Repository: spark
Updated Branches:
refs/heads/master 151d7c2ba -> 216988688
[SPARK-11979][STREAMING] Empty TrackStateRDD cannot be checkpointed and
recovered from checkpoint file
This solves the following exception caused when empty state RDD is checkpointed
and recovered. The root cause is
Repository: spark
Updated Branches:
refs/heads/branch-1.6 68bcb9b33 -> 7f030aa42
[SPARK-11979][STREAMING] Empty TrackStateRDD cannot be checkpointed and
recovered from checkpoint file
This solves the following exception caused when empty state RDD is checkpointed
and recovered. The root caus
978 from zsxwing/cached-threadpool.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/d3ef6933
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/d3ef6933
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/d3ef6933
Bran
978 from zsxwing/cached-threadpool.
(cherry picked from commit d3ef693325f91a1ed340c9756c81244a80398eb2)
Signed-off-by: Shixiong Zhu
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/0df6becc
Tree: http://git-wip-us.apache.o
978 from zsxwing/cached-threadpool.
(cherry picked from commit d3ef693325f91a1ed340c9756c81244a80398eb2)
Signed-off-by: Shixiong Zhu
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/7900d192
Tree: http://git-wip-us.apache.o
978 from zsxwing/cached-threadpool.
(cherry picked from commit d3ef693325f91a1ed340c9756c81244a80398eb2)
Signed-off-by: Shixiong Zhu
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/f5af299a
Tree: http://git-wip-us.apache.o
ad calls `ssc.stop()`, `StreamingContextStoppingCollector` may call
`ssc.stop()` in the listener bus thread, which is a dead-lock. This PR updated
`StreamingContextStoppingCollector` to only call `ssc.stop()` in the first
batch to avoid the dead-lock.
Author: Shixiong Zhu
Closes #10011 from zsxwing/fix-test-deadloc
ad calls `ssc.stop()`, `StreamingContextStoppingCollector` may call
`ssc.stop()` in the listener bus thread, which is a dead-lock. This PR updated
`StreamingContextStoppingCollector` to only call `ssc.stop()` in the first
batch to avoid the dead-lock.
Author: Shixiong Zhu
Closes #10011 from zsxwing/fix-test-deadlock.
P
oot)/KinesisStreamTests/test_kinesis_stream/
Because Streaming Python didnât work when merging
https://github.com/apache/spark/pull/9403, the PR build didnât report the
Python test failure actually.
This PR just disabled the test to unblock #10039
Author: Shixiong Zhu
Closes #10047 from zsxwing/disa
Zhu
Closes #10051 from zsxwing/SPARK-12060.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/14011665
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/14011665
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/14011
ong Zhu
Closes #10051 from zsxwing/SPARK-12060.
(cherry picked from commit 1401166576c7018c5f9c31e0a6703d5fb16ea339)
Signed-off-by: Shixiong Zhu
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/9b99b2b4
Tree: http://git-
Repository: spark
Updated Branches:
refs/heads/master 60b541ee1 -> 328b757d5
Revert "[SPARK-12060][CORE] Avoid memory copy in
JavaSerializerInstance.serialize"
This reverts commit 1401166576c7018c5f9c31e0a6703d5fb16ea339.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: ht
Repository: spark
Updated Branches:
refs/heads/branch-1.6 81db8d086 -> 21909b8ac
Revert "[SPARK-12060][CORE] Avoid memory copy in
JavaSerializerInstance.serialize"
This reverts commit 9b99b2b46c452ba396e922db5fc7eec02c45b158.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit
Repository: spark
Updated Branches:
refs/heads/master e76431f88 -> f292018f8
[SPARK-12002][STREAMING][PYSPARK] Fix python direct stream checkpoint recovery
issue
Fixed a minor race condition in #10017
Closes #10017
Author: jerryshao
Author: Shixiong Zhu
Closes #10074 from zsxwing/rev
Repository: spark
Updated Branches:
refs/heads/branch-1.6 5647774b0 -> 012de2ce5
[SPARK-12002][STREAMING][PYSPARK] Fix python direct stream checkpoint recovery
issue
Fixed a minor race condition in #10017
Closes #10017
Author: jerryshao
Author: Shixiong Zhu
Closes #10074 from zsxw
Repository: spark
Updated Branches:
refs/heads/branch-1.5 4f07a590c -> 0d57a4ae1
[SPARK-12087][STREAMING] Create new JobConf for every batch in saveAsHadoopFiles
The JobConf object created in `DStream.saveAsHadoopFiles` is used concurrently
in multiple places:
* The JobConf is updated by `RDD
Repository: spark
Updated Branches:
refs/heads/branch-1.6 a5743affc -> 1f42295b5
[SPARK-12087][STREAMING] Create new JobConf for every batch in saveAsHadoopFiles
The JobConf object created in `DStream.saveAsHadoopFiles` is used concurrently
in multiple places:
* The JobConf is updated by `RDD
Repository: spark
Updated Branches:
refs/heads/master 96691feae -> 8a75a3049
[SPARK-12087][STREAMING] Create new JobConf for every batch in saveAsHadoopFiles
The JobConf object created in `DStream.saveAsHadoopFiles` is used concurrently
in multiple places:
* The JobConf is updated by `RDD.sav
Repository: spark
Updated Branches:
refs/heads/branch-1.4 f5af299ab -> b6ba2dab2
[SPARK-12087][STREAMING] Create new JobConf for every batch in saveAsHadoopFiles
The JobConf object created in `DStream.saveAsHadoopFiles` is used concurrently
in multiple places:
* The JobConf is updated by `RDD
he fixed `ThreadUtils.newDaemonCachedThreadPool`.
Author: Shixiong Zhu
Closes #10108 from zsxwing/fix-threadpool.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/649be4fa
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/649be4fa
Di
he fixed `ThreadUtils.newDaemonCachedThreadPool`.
Author: Shixiong Zhu
Closes #10108 from zsxwing/fix-threadpool.
(cherry picked from commit 649be4fa4532dcd3001df8345f9f7e970a3fbc65)
Signed-off-by: Shixiong Zhu
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/r
ses #10167 from zsxwing/merge-SPARK-12060.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/3f4efb5c
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/3f4efb5c
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/3f4ef
Repository: spark
Updated Branches:
refs/heads/branch-1.5 93a0510a5 -> 3868ab644
[SPARK-12101][CORE] Fix thread pools that cannot cache tasks in Worker and
AppClient (backport 1.5)
backport #10108 to branch 1.5
Author: Shixiong Zhu
Closes #10135 from zsxwing/fix-threadpool-1.5.
Proj
Repository: spark
Updated Branches:
refs/heads/master 6cb06e871 -> 75c60bf4b
[SPARK-12074] Avoid memory copy involving
ByteBuffer.wrap(ByteArrayOutputStream.toByteArray)
SPARK-12060 fixed JavaSerializerInstance.serialize
This PR applies the same technique on two other classes.
zsxw
Repository: spark
Updated Branches:
refs/heads/master 2166c2a75 -> bd2cd4f53
http://git-wip-us.apache.org/repos/asf/spark/blob/bd2cd4f5/streaming/src/test/java/org/apache/spark/streaming/JavaTrackStateByKeySuite.java
--
diff --
[SPARK-12244][SPARK-12245][STREAMING] Rename trackStateByKey to mapWithState
and change tracking function signature
SPARK-12244:
Based on feedback from early users and personal experience attempting to
explain it, the name trackStateByKey had two problem.
"trackState" is a completely new term w
[SPARK-12244][SPARK-12245][STREAMING] Rename trackStateByKey to mapWithState
and change tracking function signature
SPARK-12244:
Based on feedback from early users and personal experience attempting to
explain it, the name trackStateByKey had two problem.
"trackState" is a completely new term w
Repository: spark
Updated Branches:
refs/heads/branch-1.6 699f497cf -> f6d866173
http://git-wip-us.apache.org/repos/asf/spark/blob/f6d86617/streaming/src/test/java/org/apache/spark/streaming/JavaTrackStateByKeySuite.java
--
dif
red to Scala/Java, so here changing the
description to make it more precise.
zsxwing tdas , please review, thanks a lot.
Author: jerryshao
Closes #10246 from jerryshao/direct-kafka-doc-update.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/
ies compared to Scala/Java, so here changing the
description to make it more precise.
zsxwing tdas , please review, thanks a lot.
Author: jerryshao
Closes #10246 from jerryshao/direct-kafka-doc-update.
(cherry picked from commit 24d3357d66e14388faf8709b368edca70ea96432)
Signed-off-by: Shixiong
ies compared to Scala/Java, so here changing the
description to make it more precise.
zsxwing tdas , please review, thanks a lot.
Author: jerryshao
Closes #10246 from jerryshao/direct-kafka-doc-update.
(cherry picked from commit 24d3357d66e14388faf8709b368edca70ea96432)
Signed-off-by: Shixiong
ies compared to Scala/Java, so here changing the
description to make it more precise.
zsxwing tdas , please review, thanks a lot.
Author: jerryshao
Closes #10246 from jerryshao/direct-kafka-doc-update.
(cherry picked from commit 24d3357d66e14388faf8709b368edca70ea96432)
Signed-off-by: Shixiong
Repository: spark
Updated Branches:
refs/heads/master aa305dcaf -> 713e6959d
[SPARK-12273][STREAMING] Make Spark Streaming web UI list Receivers in order
Currently the Streaming web UI does NOT list Receivers in order; however, it
seems more convenient for the users if Receivers are listed in
Repository: spark
Updated Branches:
refs/heads/master 98b212d36 -> 8af2f8c61
[SPARK-12267][CORE] Store the remote RpcEnv address to send the correct
disconnetion message
Author: Shixiong Zhu
Closes #10261 from zsxwing/SPARK-12267.
Project: http://git-wip-us.apache.org/repos/asf/sp
Repository: spark
Updated Branches:
refs/heads/branch-1.6 e05364baa -> d7e3bfd7d
[SPARK-12267][CORE] Store the remote RpcEnv address to send the correct
disconnetion message
Author: Shixiong Zhu
Closes #10261 from zsxwing/SPARK-12267.
(cherry picked from com
java:745)
```
Author: Shixiong Zhu
Closes #10269 from zsxwing/executor-state.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/2aecda28
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/2aecda28
Diff: http:/
ead.java:745)
```
Author: Shixiong Zhu
Closes #10269 from zsxwing/executor-state.
(cherry picked from commit 2aecda284e22ec608992b6221e2f5ffbd51fcd24)
Signed-off-by: Shixiong Zhu
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spar
Repository: spark
Updated Branches:
refs/heads/master c59df8c51 -> bc1ff9f4a
[STREAMING][MINOR] Fix typo in function name of StateImpl
cc\ tdas zsxwing , please review. Thanks a lot.
Author: jerryshao
Closes #10305 from jerryshao/fix-typo-state-impl.
Project: http://git-wip-us.apache.
Repository: spark
Updated Branches:
refs/heads/branch-1.6 352a0c80f -> 23c884605
[STREAMING][MINOR] Fix typo in function name of StateImpl
cc\ tdas zsxwing , please review. Thanks a lot.
Author: jerryshao
Closes #10305 from jerryshao/fix-typo-state-impl.
(cherry picked from com
Repository: spark
Updated Branches:
refs/heads/master ca0690b5e -> d52bf47e1
[SPARK-12304][STREAMING] Make Spark Streaming web UI display more friâ¦
â¦endly Receiver graphs
Currently, the Spark Streaming web UI uses the same maxY when displays 'Input
Rate Times& Histograms' and 'Per-Receiv
Repository: spark
Updated Branches:
refs/heads/branch-1.6 552b38f87 -> 638b89bc3
[MINOR] Add missing interpolation in NettyRPCEnv
```
Exception in thread "main" org.apache.spark.rpc.RpcTimeoutException:
Cannot receive any reply in ${timeout.duration}. This timeout is controlled by
spark.rpc.a
Repository: spark
Updated Branches:
refs/heads/master 27b98e99d -> 861549acd
[MINOR] Add missing interpolation in NettyRPCEnv
```
Exception in thread "main" org.apache.spark.rpc.RpcTimeoutException:
Cannot receive any reply in ${timeout.duration}. This timeout is controlled by
spark.rpc.askTi
Repository: spark
Updated Branches:
refs/heads/master 97678edea -> 437583f69
[SPARK-11904][PYSPARK] reduceByKeyAndWindow does not require checkpointing when
invFunc is None
when invFunc is None, `reduceByKeyAndWindow(func, None, winsize, slidesize)` is
equivalent to
reduceByKey(func).w
Repository: spark
Updated Branches:
refs/heads/master 6e0771665 -> 86e405f35
[SPARK-12220][CORE] Make Utils.fetchFile support files that contain special
characters
This PR encodes and decodes the file name to fix the issue.
Author: Shixiong Zhu
Closes #10208 from zsxwing/uri.
Proj
Repository: spark
Updated Branches:
refs/heads/branch-1.6 41ad8aced -> 1fbca4120
[SPARK-12220][CORE] Make Utils.fetchFile support files that contain special
characters
This PR encodes and decodes the file name to fix the issue.
Author: Shixiong Zhu
Closes #10208 from zsxwing/uri.
(che
Repository: spark
Updated Branches:
refs/heads/master 818456881 -> 540b5aead
[SPARK-12410][STREAMING] Fix places that use '.' and '|' directly in split
String.split accepts a regular expression, so we should escape "." and "|".
Author: Shixio
Repository: spark
Updated Branches:
refs/heads/branch-1.6 88bbb5429 -> c0ab14fbe
[SPARK-12410][STREAMING] Fix places that use '.' and '|' directly in split
String.split accepts a regular expression, so we should escape "." and "|".
Author: Shixio
Repository: spark
Updated Branches:
refs/heads/branch-1.5 0fdf5542b -> a8d14cc06
[SPARK-12410][STREAMING] Fix places that use '.' and '|' directly in split
String.split accepts a regular expression, so we should escape "." and "|".
Author: Shixio
Repository: spark
Updated Branches:
refs/heads/branch-1.4 43f02e41e -> 28adc45d5
[SPARK-12410][STREAMING] Fix places that use '.' and '|' directly in split
String.split accepts a regular expression, so we should escape "." and "|".
Author: Shixio
Repository: spark
Updated Branches:
refs/heads/master e096a652b -> ed6ebda5c
[SPARK-12376][TESTS] Spark Streaming Java8APISuite fails in
assertOrderInvariantEquals method
org.apache.spark.streaming.Java8APISuite.java is failing due to trying to sort
immutable list in assertOrderInvariantEqua
Repository: spark
Updated Branches:
refs/heads/branch-1.6 48dcee484 -> 4df1dd403
[SPARK-12376][TESTS] Spark Streaming Java8APISuite fails in
assertOrderInvariantEquals method
org.apache.spark.streaming.Java8APISuite.java is failing due to trying to sort
immutable list in assertOrderInvariant
Repository: spark
Updated Branches:
refs/heads/master 658f66e62 -> f4346f612
[SPARK-11749][STREAMING] Duplicate creating the RDD in file stream when
recovering from checkpoint data
Add a transient flag `DStream.restoredFromCheckpointData` to control the
restore processing in DStream to avoid
Repository: spark
Updated Branches:
refs/heads/branch-1.6 4df1dd403 -> 9177ea383
[SPARK-11749][STREAMING] Duplicate creating the RDD in file stream when
recovering from checkpoint data
Add a transient flag `DStream.restoredFromCheckpointData` to control the
restore processing in DStream to a
hu
Closes #10363 from zsxwing/hide-log.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/0370abdf
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/0370abdf
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/0370abdf
Br
Repository: spark
Updated Branches:
refs/heads/master 6eba65525 -> 2377b707f
[SPARK-11985][STREAMING][KINESIS][DOCS] Update Kinesis docs
- Provide example on `message handler`
- Provide bit on KPL record de-aggregation
- Fix typos
Author: Burak Yavuz
Closes #9970 from brkyvz/kinesis-docs
Repository: spark
Updated Branches:
refs/heads/branch-1.6 bd33d4ee8 -> eca401ee5
[SPARK-11985][STREAMING][KINESIS][DOCS] Update Kinesis docs
- Provide example on `message handler`
- Provide bit on KPL record de-aggregation
- Fix typos
Author: Burak Yavuz
Closes #9970 from brkyvz/kinesis-
Repository: spark
Updated Branches:
refs/heads/master ea4aab7e8 -> 1e9781395
[SPARK-12396][CORE] Modify the function scheduleAtFixedRate to schedule.
Instead of just cancel the registrationRetryTimer to avoid driver retry connect
to master, change the function to schedule.
It is no need to re
ock`.
Author: Shixiong Zhu
Closes #10440 from zsxwing/findbugs.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/710b4117
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/710b4117
Diff: http://git-wip-us.apache.org/re
and `ReentrantLock`.
Author: Shixiong Zhu
Closes #10440 from zsxwing/findbugs.
(cherry picked from commit 710b41172958a0b3a2b70c48821aefc81893731b)
Signed-off-by: Shixiong Zhu
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/fd202
f8b-39df08426bf8.png";>
This PR just added a class for the new style and only applied them to the paged
tables.
Author: Shixiong Zhu
Closes #10517 from zsxwing/fix-streaming-ui.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/s
result. `submitJobThreadPool` was a workaround in `ReceiverTracker` to run
these waiting-job-result threads. Now #9264 has been merged to master and
resolved this blocking issue, `submitJobThreadPool` can be removed now.
Author: Shixiong Zhu
Closes #10560 from zsxwing/remove-submitJobThreadPool.
ses #10621 from zsxwing/SPARK-12617-2.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/1e6648d6
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/1e6648d6
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/1e6648d6
Bra
ses #10621 from zsxwing/SPARK-12617-2.
(cherry picked from commit 1e6648d62fb82b708ea54c51cd23bfe4f542856e)
Signed-off-by: Shixiong Zhu
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/d821fae0
Tree: http://git-wip-us.apache.
ses #10621 from zsxwing/SPARK-12617-2.
(cherry picked from commit 1e6648d62fb82b708ea54c51cd23bfe4f542856e)
Signed-off-by: Shixiong Zhu
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/598a5c2c
Tree: http://git-wip-us.apache.
Repository: spark
Updated Branches:
refs/heads/master 1e6648d62 -> 19e4e9feb
[SPARK-12672][STREAMING][UI] Use the uiRoot function instead of default root
path to gain the streaming batch url.
Author: huangzhaowei
Closes #10617 from SaintBacchus/SPARK-12672.
Project: http://git-wip-us.apac
Repository: spark
Updated Branches:
refs/heads/branch-1.6 d821fae0e -> 8f0ead3e7
[SPARK-12672][STREAMING][UI] Use the uiRoot function instead of default root
path to gain the streaming batch url.
Author: huangzhaowei
Closes #10617 from SaintBacchus/SPARK-12672.
Project: http://git-wip-us.
Repository: spark
Updated Branches:
refs/heads/branch-1.5 598a5c2cc -> fb421af08
[SPARK-12672][STREAMING][UI] Use the uiRoot function instead of default root
path to gain the streaming batch url.
Author: huangzhaowei
Closes #10617 from SaintBacchus/SPARK-12672.
Project: http://git-wip-us.
Repository: spark
Updated Branches:
refs/heads/master 19e4e9feb -> cbaea9591
Revert "[SPARK-12672][STREAMING][UI] Use the uiRoot function instead of default
root path to gain the streaming batch url."
This reverts commit 19e4e9febf9bb4fd69f6d7bc13a54844e4e096f1. Will merge #10618
instead.
Repository: spark
Updated Branches:
refs/heads/branch-1.6 8f0ead3e7 -> 39b0a3480
Revert "[SPARK-12672][STREAMING][UI] Use the uiRoot function instead of default
root path to gain the streaming batch url."
This reverts commit 8f0ead3e79beb2c5f2731ceaa34fe1c133763386. Will merge #10618
instead
Repository: spark
Updated Branches:
refs/heads/branch-1.5 fb421af08 -> d10b9d572
Revert "[SPARK-12672][STREAMING][UI] Use the uiRoot function instead of default
root path to gain the streaming batch url."
This reverts commit fb421af08de73e4ae6b04a576721109cae561865. Will merge #10618
instead
Repository: spark
Updated Branches:
refs/heads/branch-1.5 5e86c0cce -> f2bc02ec4
[SPARK-12673][UI] Add missing uri prepending for job description
Otherwise the url will be failed to proxy to the right one if in YARN mode.
Here is the screenshot:
Signed-off-by: Shixiong Z
Repository: spark
Updated Branches:
refs/heads/master 9559ac5f7 -> b313badaa
[STREAMING][MINOR] Typo fixes
Author: Jacek Laskowski
Closes #10698 from jaceklaskowski/streaming-kafka-typo-fixes.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/r
634 and 27634 to reduce the possibility of port
conflicts.
- Make `service1` use `service0.port` to bind to avoid the above race condition.
## How was this patch tested?
Jenkins unit tests.
Author: Shixiong Zhu
Closes #12477 from zsxwing/SPARK-14713.
Project: http://git-wip-us.apache.org/re
Repository: spark
Updated Branches:
refs/heads/master 9ee95b6ec -> e89633605
[SPARK-13904] Add exit code parameter to exitExecutor()
## What changes were proposed in this pull request?
This PR adds exit code parameter to exitExecutor() so that caller can specify
different exit code.
## How
ition, Outbox should not close the client since it will be reused by
others. This PR fixes it as well.
## How was this patch tested?
test("SPARK-14699: RpcEnv.shutdown should not fire onDisconnected events")
Author: Shixiong Zhu
Closes #12481 from zsxwing/SPARK-14699.
Project: http:
Repository: spark
Updated Branches:
refs/heads/master 3647120a5 -> fde1340c7
[SPARK-14701][STREAMING] First stop the event loop, then stop the checkpoint
writer in JobGenerator
Currently if we call `streamingContext.stop` (e.g. in a
`StreamingListener.onBatchCompleted` callback) when a batch
Repository: spark
Updated Branches:
refs/heads/master c25b97fcc -> c431a76d0
[SPARK-14832][SQL][STREAMING] Refactor DataSource to ensure schema is inferred
only once when creating a file stream
## What changes were proposed in this pull request?
When creating a file stream using sqlContext.w
Repository: spark
Updated Branches:
refs/heads/master ba5e0b87a -> 285385965
[SPARK-14833][SQL][STREAMING][TEST] Refactor StreamTests to test for source
fault-tolerance correctly.
## What changes were proposed in this pull request?
Current StreamTest allows testing of a streaming Dataset gen
Repository: spark
Updated Branches:
refs/heads/master 5503e453b -> 5bd9a2f69
[SPARK-14884][SQL][STREAMING][WEBUI] Fix call site for continuous queries
## What changes were proposed in this pull request?
Since we've been processing continuous queries in separate threads, the call
sites are th
Repository: spark
Updated Branches:
refs/heads/branch-2.0 17996e7d0 -> 45bc65519
[SPARK-14884][SQL][STREAMING][WEBUI] Fix call site for continuous queries
## What changes were proposed in this pull request?
Since we've been processing continuous queries in separate threads, the call
sites ar
Repository: spark
Updated Branches:
refs/heads/master ca813330c -> 439e36101
[SPARK-9819][STREAMING][DOCUMENTATION] Clarify doc for invReduceFunc in
incremental versions of reduceByWindow
- that reduceFunc and invReduceFunc should be associative
- that the intermediate result in iterated appl
Repository: spark
Updated Branches:
refs/heads/branch-2.0 2b715251d -> 69e030226
[SPARK-9819][STREAMING][DOCUMENTATION] Clarify doc for invReduceFunc in
incremental versions of reduceByWindow
- that reduceFunc and invReduceFunc should be associative
- that the intermediate result in iterated
Repository: spark
Updated Branches:
refs/heads/branch-2.0 a08d2c954 -> 50adfcdce
[SPARK-15059][CORE] Remove fine-grained lock in ChildFirstURLClassLoader to
avoid dead lock
## What changes were proposed in this pull request?
In some cases, fine-grained lock have race condition with class-loa
Repository: spark
Updated Branches:
refs/heads/master 84b3a4a87 -> f5623b460
[SPARK-15059][CORE] Remove fine-grained lock in ChildFirstURLClassLoader to
avoid dead lock
## What changes were proposed in this pull request?
In some cases, fine-grained lock have race condition with class-loader
Repository: spark
Updated Branches:
refs/heads/branch-2.0 50adfcdce -> 4bacdebe9
[SPARK-14234][CORE] Executor crashes for TaskRunner thread interruption
## What changes were proposed in this pull request?
Resetting the task interruption status before updating the task status.
## How was this
Repository: spark
Updated Branches:
refs/heads/master f5623b460 -> 659f635d3
[SPARK-14234][CORE] Executor crashes for TaskRunner thread interruption
## What changes were proposed in this pull request?
Resetting the task interruption status before updating the task status.
## How was this patc
Repository: spark
Updated Branches:
refs/heads/master a45647746 -> e597ec6f1
[SPARK-15022][SPARK-15023][SQL][STREAMING] Add support for testing against the
`ProcessingTime(intervalMS > 0)` trigger and `ManualClock`
## What changes were proposed in this pull request?
Currently in `StreamTest`
Repository: spark
Updated Branches:
refs/heads/branch-2.0 ae79032dc -> 343c28504
[SPARK-15022][SPARK-15023][SQL][STREAMING] Add support for testing against the
`ProcessingTime(intervalMS > 0)` trigger and `ManualClock`
## What changes were proposed in this pull request?
Currently in `StreamT
Repository: spark
Updated Branches:
refs/heads/master ef55e46c9 -> bde27b89a
[SPARK-15131][SQL] Shutdown StateStore management thread when SparkContext has
been shutdown
## What changes were proposed in this pull request?
Make sure that whenever the StateStoreCoordinator cannot be contacted,
Repository: spark
Updated Branches:
refs/heads/branch-2.0 689b0fc81 -> e12ec46c6
[SPARK-15131][SQL] Shutdown StateStore management thread when SparkContext has
been shutdown
## What changes were proposed in this pull request?
Make sure that whenever the StateStoreCoordinator cannot be contac
Repository: spark
Updated Branches:
refs/heads/master 93353b011 -> 89f73f674
[SPARK-14642][SQL] import org.apache.spark.sql.expressions._ breaks udf under
functions
## What changes were proposed in this pull request?
PR fixes the import issue which breaks udf functions.
The following code s
1 - 100 of 788 matches
Mail list logo
