This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 7074e4fee7e [SPARK-41040][SS] Fix self-union
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new ef402edff91 [SPARK-41045][SQL] Pre-compute
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 878cc0e [SPARK-32896][SS][FOLLOW-UP] Rename
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 6fa797e [SPARK-32896][SS][FOLLOW-UP] Rename
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 4f96670 [SPARK-31953][SS] Add Spark Structured
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new 48017cc [SPARK-31923][CORE] Ignore
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new b00ac30 [SPARK-31923][CORE] Ignore
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new b333ed0 [SPARK-31923][CORE] Ignore internal
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 5a258b0 [SPARK-30915][SS
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new 5343059 [SPARK-30943][SS] Show "
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 9ea6c0a [SPARK-30943][SS] Show "bat
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new 1385fc0 [SPARK-29748][DOCS][FOLLOW-UP
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new b343757 [SPARK-29748][DOCS][FOLLOW-UP] Add
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new f9b8637 [SPARK-29543][SS][FOLLOWUP
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new a4912ce [SPARK-29543][SS][FOLLOWUP] Move
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 481e521 [SPARK-30657][SPARK-30658][SS] Fixed
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new f56ba37 [SPARK-30656][SS] Support
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 7173786 [SPARK-29543][SS][UI] Structured
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a change to branch 2.0
in repository https://gitbox.apache.org/repos/asf/spark.git.
at 0f3d744c [MINOR][TESTS] Remove unsupported `header` option in AvroSuite
No new revisions were added by this update
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new cfd7ca9 Revert "[SPARK-21869][SS] Apply A
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 25431d7 [SPARK-29953][SS] Don't clean up
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new b094774 Revert "[SPARK-29494][SQL
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new d72f398 [SPARK-27254][SS] Cleanup complete
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 84a4d3a [SPARK-28976][CORE] Use KeyLock
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 8980093 [SPARK-3137][CORE] Replace the global
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 406c533 [SPARK-28025][SS] Fix
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new aff5e2b [SPARK-28650][SS][DOC] Correct
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new b37c8d5 [SPARK-28650][SS][DOC] Correct
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new c212c9d [SPARK-28574][CORE] Allow to config
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 04f142d [SPARK-20547][REPL] Throw
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new a8a2ba1 [SPARK-27394][WEBUI] Flush
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 78d546f [SPARK-27210][SS] Cleanup incomplete
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch branch-2.3
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.3 by this push:
new 4d1d0a4 [SPARK-27111][SS] Fix a race
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new 53590f2 [SPARK-27111][SS] Fix a race
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 6e1c082 [SPARK-27111][SS] Fix a race
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 77b99af [SPARK-26824][SS] Fix the checkpoint
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch branch-2.2
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.2 by this push:
new 7c7d7f6 [SPARK-26806][SS
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch branch-2.3
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.3 by this push:
new a5d22da [SPARK-26806][SS
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new bd4ce51 [SPARK-26806][SS
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 03a928c [SPARK-26806][SS] EventTimeStats.merge
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/spark-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new fb1a7b4 Add Jose Torres
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch branch-2.3
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.3 by this push:
new b88067b [SPARK-26665][CORE] Fix a bug
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new 9814108 [SPARK-26665][CORE] Fix a bug
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 66450bb [SPARK-26665][CORE] Fix a bug
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch branch-2.3
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.3 by this push:
new c0fc6d0 Revert "[SPARK-2662
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch branch-2.3
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.3 by this push:
new 5a50ae3 [SPARK-26629][SS] Fixed error
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new 1843c16 [SPARK-26629][SS] Fixed error
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 06d5b17 [SPARK-26629][SS] Fixed error
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 2ebb79b [SPARK-26350][FOLLOWUP] Add actual
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new bbf61eb [SPARK-26586][SS] Fix race
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new ae382c9 [SPARK-26586][SS] Fix race condition
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new faa4c28 [SPARK-26267][SS] Retry when
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 8e76d66 [SPARK-26267][SS] Retry when detecting
so that the
`metadata` file will never be a partial file.
## How was this patch tested?
Jenkins
Closes #23060 from zsxwing/SPARK-26092.
Authored-by: Shixiong Zhu
Signed-off-by: Shixiong Zhu
(cherry picked from commit 058c4602b000b24deb764a810ef8b43c41fe63ae)
Signed-off-by: Shixiong Zhu
Project: h
so that the
`metadata` file will never be a partial file.
## How was this patch tested?
Jenkins
Closes #23060 from zsxwing/SPARK-26092.
Authored-by: Shixiong Zhu
Signed-off-by: Shixiong Zhu
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/com
32e/common/network-common/src/test/java/org/apache/spark/network/RpcIntegrationSuite.java#L217
This PR fixes the above issue and also improves the test failure messages of
`assertErrorAndClosed`.
## How was this patch tested?
Jenkins
Closes #23041 from zsxwing/SPARK-26069.
Authored-by: Shixiong
mon/network-common/src/test/java/org/apache/spark/network/RpcIntegrationSuite.java#L217
This PR fixes the above issue and also improves the test failure messages of
`assertErrorAndClosed`.
## How was this patch tested?
Jenkins
Closes #23041 from zsxwing/SPARK-26069.
Authored-by: Shixiong
ing to
initialize `executedPlan` when `isRDD` is running, this thread will hang
forever.
This PR just materializes `executedPlan` so that accessing it when `toRdd` is
running doesn't need to wait for a lock
## How was this patch tested?
Jenkins
Closes #23023 from zsxwing/SPARK-26042.
Autho
ing to
initialize `executedPlan` when `isRDD` is running, this thread will hang
forever.
This PR just materializes `executedPlan` so that accessing it when `toRdd` is
running doesn't need to wait for a lock
## How was this patch tested?
Jenkins
Closes #23023 from zsxwing/SPARK-26042.
Autho
lso fixes two minor issues while I'm touching DAGScheduler:
- Invalid spark.job.interruptOnCancel should not crash DAGScheduler.
- Non fatal errors should not crash DAGScheduler.
## How was this patch tested?
The new unit tests.
Closes #22771 from zsxwing/SPARK-25773.
Lead-authored-by: Shixiong
ock.
2. `createSimpleWorker` misses `synchronized` when updating `simpleWorkers`.
Other changes are just to improve the code style to make the thread-safe
contract clear.
## How was this patch tested?
Jenkins
Closes #22770 from zsxwing/pwf.
Authored-by: Shixiong Zhu
Signed-off-by: Shixiong
her `scala.Long`.
## How was this patch tested?
New java test.
Closes #22633 from zsxwing/fix-java-foreachbatch.
Authored-by: Shixiong Zhu
Signed-off-by: Shixiong Zhu
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/0a70afdc
Tree: h
ong`.
## How was this patch tested?
New java test.
Closes #22633 from zsxwing/fix-java-foreachbatch.
Authored-by: Shixiong Zhu
Signed-off-by: Shixiong Zhu
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/7dcc90fb
Tree: h
Repository: spark
Updated Branches:
refs/heads/master a28146568 -> 9362c5cc2
[SPARK-25449][CORE] Heartbeat shouldn't include accumulators for zero metrics
## What changes were proposed in this pull request?
Heartbeat shouldn't include accumulators for zero metrics.
Heartbeats sent from
ise it will cause inconsistent cached data and may
make Kafka connector return wrong results.
## How was this patch tested?
The new unit test.
Closes #22507 from zsxwing/fix-kafka-reset.
Lead-authored-by: Shixiong Zhu
Co-authored-by: Shixiong Zhu
Signed-off-by: Shixiong Zhu
(cherry picked f
ise it will cause inconsistent cached data and may
make Kafka connector return wrong results.
## How was this patch tested?
The new unit test.
Closes #22507 from zsxwing/fix-kafka-reset.
Lead-authored-by: Shixiong Zhu
Co-authored-by: Shixiong Zhu
Signed-off-by: Shixiong Zhu
Project: http://git-
can see a specified offset before checking the result.
## How was this patch tested?
Jenkins
Closes #22293 from zsxwing/SPARK-25288.
Authored-by: Shixiong Zhu
Signed-off-by: Shixiong Zhu
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/
tch
They are all covered by the new unit tests.
## How was this patch tested?
The new unit tests.
Closes #22042 from zsxwing/kafka-transaction-read.
Authored-by: Shixiong Zhu
Signed-off-by: Shixiong Zhu
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.
ces for all
types of errors.
## How was this patch tested?
Jenkins
Closes #22210 from zsxwing/SPARK-25218.
Authored-by: Shixiong Zhu
Signed-off-by: Shixiong Zhu
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/592e3a42
Tree: h
Repository: spark
Updated Branches:
refs/heads/master 381a967a7 -> 810d59ce4
[SPARK-24882][FOLLOWUP] Fix flaky synchronization in Kafka tests.
## What changes were proposed in this pull request?
Fix flaky synchronization in Kafka tests - we need to use the scan config that
was persisted
fix a potential flaky test.
`processAllAvailable` doesn't work for continuous processing so we should not
use it for a continuous query.
## How was this patch tested?
Jenkins.
Closes #22230 from zsxwing/SPARK-25214-2.
Authored-by: Shixiong Zhu
Signed-off-by: Shixiong Zhu
Project: http://
may return duplicated records
when `failOnDataLoss=false` because it doesn't skip missing offsets.
This PR fixes the issue and also adds regression tests for all Kafka readers.
## How was this patch tested?
New tests.
Closes #22207 from zsxwing/SPARK-25214.
Authored-by: Shixiong Zhu
Signed-
Repository: spark
Updated Branches:
refs/heads/master 310632498 -> 49a1993b1
[SPARK-25163][SQL] Fix flaky test:
o.a.s.util.collection.ExternalAppendOnlyMapSuiteCheck
## What changes were proposed in this pull request?
`ExternalAppendOnlyMapSuiteCheck` test is flaky.
We use a
Repository: spark
Updated Branches:
refs/heads/master 2381953ab -> 68ec4d641
[SPARK-25181][CORE] Limit Thread Pool size in BlockManager Master and Slave
endpoints
## What changes were proposed in this pull request?
Limit Thread Pool size in BlockManager Master and Slave endpoints.
ion to node 0 could not
be established. Broker may not be available.
```
I also reverted
https://github.com/apache/spark/pull/22097/commits/b5eb54244ed573c8046f5abf7bf087f5f08dba58
introduced by #22097 since it doesn't help.
## How was this patch tested?
Jenkins
Closes #22106 from zsxwing/SP
ost` to make sure zookeeper
will never use an ipv6 address.
## How was this patch tested?
Jenkins
Closes #22097 from zsxwing/fix-zookeeper-connect.
Authored-by: Shixiong Zhu
Signed-off-by: Shixiong Zhu
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.
allocateArray` to fix the issue.
## How was this patch tested?
The new unit test will make JVM crash without the fix.
Closes #22062 from zsxwing/SPARK-25081.
Authored-by: Shixiong Zhu
Signed-off-by: Shixiong Zhu
(cherry picked from commit f5aba657396bd4e2e03dd06491a2d169a99592a7)
Signed-off-by:
Array` to fix the issue.
## How was this patch tested?
The new unit test will make JVM crash without the fix.
Closes #22062 from zsxwing/SPARK-25081.
Authored-by: Shixiong Zhu
Signed-off-by: Shixiong Zhu
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apa
Repository: spark
Updated Branches:
refs/heads/master 3c96937c7 -> 87ca7396c
[SPARK-24161][SS] Enable debug package feature on structured streaming
## What changes were proposed in this pull request?
Currently, debug package has a implicit class "DebugQuery" which matches
Dataset to provide
Repository: spark
Updated Branches:
refs/heads/master 8c14276c3 -> 4c27663cb
[SPARK-18057][FOLLOW-UP][SS] Update Kafka client version from 0.10.0.1 to 2.0.0
## What changes were proposed in this pull request?
Increase ZK timeout and harmonize configs across Kafka tests to resolâ¦ve
Repository: spark
Updated Branches:
refs/heads/master efef55388 -> d0bc3ed67
[SPARK-24896][SQL] Uuid should produce different values for each execution in
streaming query
## What changes were proposed in this pull request?
`Uuid`'s results depend on random seed given during analysis. Thus
wip-us.apache.org/repos/asf/spark/diff/e82784d1
Branch: refs/heads/master
Commit: e82784d13fac7d45164dfadb00d3fa43e64e0bde
Parents: 1223a20
Author: tedyu
Authored: Tue Jul 31 13:14:14 2018 -0700
Committer: zsxwing
Committed: Tue Jul 31 13:14:14 2018 -0
Repository: spark
Updated Branches:
refs/heads/master f6e6899a8 -> f71e8da5e
[SPARK-24566][CORE] Fix spark.storage.blockManagerSlaveTimeoutMs default config
This PR use spark.network.timeout in place of
spark.storage.blockManagerSlaveTimeoutMs when it is not configured, as
configuration doc
Repository: spark
Updated Branches:
refs/heads/branch-2.3 d687d97b1 -> 8928de3cd
[SPARK-24578][CORE] Cap sub-region's size of returned nio buffer
## What changes were proposed in this pull request?
This PR tries to fix the performance regression introduced by SPARK-21517.
In our production
Repository: spark
Updated Branches:
refs/heads/master c5a0d1132 -> 3f4bda728
[SPARK-24578][CORE] Cap sub-region's size of returned nio buffer
## What changes were proposed in this pull request?
This PR tries to fix the performance regression introduced by SPARK-21517.
In our production job,
Repository: spark
Updated Branches:
refs/heads/master 13092d733 -> 2cb976355
[SPARK-24565][SS] Add API for in Structured Streaming for exposing output rows
of each microbatch as a DataFrame
## What changes were proposed in this pull request?
Currently, the micro-batches in the
Repository: spark
Updated Branches:
refs/heads/master 299d297e2 -> 1b46f41c5
[SPARK-24235][SS] Implement continuous shuffle writer for single reader
partition.
## What changes were proposed in this pull request?
Repository: spark
Updated Branches:
refs/heads/master 98909c398 -> 6039b1323
[SPARK-24351][SS] offsetLog/commitLog purge thresholdBatchId should be computed
with current committed epoch but not currentBatchId in CP mode
## What changes were proposed in this pull request?
Compute the
sue that
reading "spark.network.timeout" using a wrong time unit when the user doesn't
specify a time out.
## How was this patch tested?
Jenkins
Author: Shixiong Zhu <zsxw...@gmail.com>
Closes #21382 from zsxwing/fix-network-timeout-conf.
Project: http://git-wip-us.apache.org/r
Repository: spark
Updated Branches:
refs/heads/master 7696b9de0 -> 807ba44cb
[SPARK-24159][SS] Enable no-data micro batches for streaming mapGroupswithState
## What changes were proposed in this pull request?
Enabled no-data batches in flatMapGroupsWithState in following two cases.
- When
Repository: spark
Updated Branches:
refs/heads/master 0cf59fcbe -> 7696b9de0
[SPARK-20538][SQL] Wrap Dataset.reduce with withNewRddExecutionId.
## What changes were proposed in this pull request?
Wrap Dataset.reduce with `withNewExecutionId`.
Author: Soham Aurangabadkar
Repository: spark
Updated Branches:
refs/heads/master d610d2a3f -> 3fabbc576
[SPARK-24040][SS] Support single partition aggregates in continuous processing.
## What changes were proposed in this pull request?
Support aggregates with exactly 1 partition in continuous processing.
A few small
t; to provide the SparkSession parameter
otherwise TreeNode.toJSON cannot get the full constructor parameter list.
## How was this patch tested?
The new unit test.
Author: Shixiong Zhu <zsxw...@gmail.com>
Closes #21275 from zsxwing/SPARK-24214.
(cherry
t; to provide the SparkSession parameter
otherwise TreeNode.toJSON cannot get the full constructor parameter list.
## How was this patch tested?
The new unit test.
Author: Shixiong Zhu <zsxw...@gmail.com>
Closes #21275 from zsxwing/SPARK-24214.
Project: http://git-wip-us.apache.org/repos/as
Repository: spark
Updated Branches:
refs/heads/master b857fb549 -> 7bbec0dce
[SPARK-24061][SS] Add TypedFilter support for continuous processing
## What changes were proposed in this pull request?
Add TypedFilter support for continuous processing application.
## How was this patch tested?
Repository: spark
Updated Branches:
refs/heads/master 109935fc5 -> 2824f12b8
[SPARK-23565][SS] New error message for structured streaming sources assertion
## What changes were proposed in this pull request?
A more informative message to tell you why a structured streaming query cannot
Repository: spark
Updated Branches:
refs/heads/branch-2.2 85ab72b59 -> 6b5f9c374
[SPARK-23788][SS] Fix race in StreamingQuerySuite
## What changes were proposed in this pull request?
The serializability test uses the same MemoryStream instance for 3 different
queries. If any of those
Repository: spark
Updated Branches:
refs/heads/branch-2.3 ea44783ad -> 523fcafc5
[SPARK-23788][SS] Fix race in StreamingQuerySuite
## What changes were proposed in this pull request?
The serializability test uses the same MemoryStream instance for 3 different
queries. If any of those
Repository: spark
Updated Branches:
refs/heads/master a33655348 -> 816a5496b
[SPARK-23788][SS] Fix race in StreamingQuerySuite
## What changes were proposed in this pull request?
The serializability test uses the same MemoryStream instance for 3 different
queries. If any of those queries
Repository: spark
Updated Branches:
refs/heads/branch-2.3 21b6de459 -> 6937571ab
[SPARK-23623][SS] Avoid concurrent use of cached consumers in
CachedKafkaConsumer (branch-2.3)
This is a backport of #20767 to branch 2.3
## What changes were proposed in this pull request?
CacheKafkaConsumer
1 - 100 of 751 matches
Mail list logo