[GitHub] spark pull request #20587: Branch 2.2

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/20587


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20587: Branch 2.2

[zhuge134]

GitHub user zhuge134 opened a pull request:

https://github.com/apache/spark/pull/20587

Branch 2.2



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/spark branch-2.2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/20587.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #20587


commit 4e53a4edd72e372583f243c660bbcc0572205716
Author: Tathagata Das 
Date:   2017-07-06T07:20:26Z

[SS][MINOR] Fix flaky test in DatastreamReaderWriterSuite. temp checkpoint 
dir should be deleted

## What changes were proposed in this pull request?

Stopping query while it is being initialized can throw interrupt exception, 
in which case temporary checkpoint directories will not be deleted, and the 
test will fail.

Author: Tathagata Das 

Closes #18442 from tdas/DatastreamReaderWriterSuite-fix.

(cherry picked from commit 60043f22458668ac7ecba94fa78953f23a6bdcec)
Signed-off-by: Tathagata Das 

commit 576fd4c3a67b4affc5ac50979e27ae929472f0d9
Author: Tathagata Das 
Date:   2017-07-07T00:28:20Z

[SPARK-21267][SS][DOCS] Update Structured Streaming Documentation

## What changes were proposed in this pull request?

Few changes to the Structured Streaming documentation
- Clarify that the entire stream input table is not materialized
- Add information for Ganglia
- Add Kafka Sink to the main docs
- Removed a couple of leftover experimental tags
- Added more associated reading material and talk videos.

In addition, https://github.com/apache/spark/pull/16856 broke the link to 
the RDD programming guide in several places while renaming the page. This PR 
fixes those sameeragarwal cloud-fan.
- Added a redirection to avoid breaking internal and possible external 
links.
- Removed unnecessary redirection pages that were there since the separate 
scala, java, and python programming guides were merged together in 2013 or 2014.

## How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration 
tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, 
remove this)

Please review http://spark.apache.org/contributing.html before opening a 
pull request.

Author: Tathagata Das 

Closes #18485 from tdas/SPARK-21267.

(cherry picked from commit 0217dfd26f89133f146197359b556c9bf5aca172)
Signed-off-by: Shixiong Zhu 

commit ab12848d624f6b74d401e924255c0b4fcc535231
Author: Prashant Sharma 
Date:   2017-07-08T06:33:12Z

[SPARK-21069][SS][DOCS] Add rate source to programming guide.

## What changes were proposed in this pull request?

SPARK-20979 added a new structured streaming source: Rate source. This 
patch adds the corresponding documentation to programming guide.

## How was this patch tested?

Tested by running jekyll locally.

Author: Prashant Sharma 
Author: Prashant Sharma 

Closes #18562 from ScrapCodes/spark-21069/rate-source-docs.

(cherry picked from commit d0bfc6733521709e453d643582df2bdd68f28de7)
Signed-off-by: Shixiong Zhu 

commit 7d0b1c927d92cc2a4932262514ffd12c47593b80
Author: Bogdan Raducanu 
Date:   2017-07-08T12:14:59Z

[SPARK-21228][SQL][BRANCH-2.2] InSet incorrect handling of structs

## What changes were proposed in this pull request?

This is backport of https://github.com/apache/spark/pull/18455
When data type is struct, InSet now uses TypeUtils.getInterpretedOrdering 
(similar to EqualTo) to build a TreeSet. In other cases it will use a HashSet 
as before (which should be faster). Similarly, In.eval uses Ordering.equiv 
instead of equals.

## How was this patch tested?
New test in SQLQuerySuite.

Author: Bogdan Raducanu 

Closes #18563 from bogdanrdc/SPARK-21228-BRANCH2.2.

commit a64f10800244a8057f7f32c3d2f4a719c5080d05
Author: Dongjoon Hyun 
Date:   2017-07-08T12:16:47Z

[SPARK-21345][SQL][TEST][TEST-MAVEN] SparkSessionBuilderSuite should clean 
up stopped sessions.

`SparkSessionBuilderSuite` should clean up stopped sessions. Otherwise, it 
leaves behind some stopped `SparkContext`s interfereing with other test suites 
using `ShardSQLContext`.

Recently, master branch fails consequtively.
- 
https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/

Pass the Jenkins with a updated suite.