[GitHub] spark pull request #20569: Branch 2.2

[MohammedLayeeq]

GitHub user MohammedLayeeq opened a pull request:

https://github.com/apache/spark/pull/20569

Branch 2.2

## What changes were proposed in this pull request?

(Please fill in changes proposed in this fix)

## How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration 
tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, 
remove this)

Please review http://spark.apache.org/contributing.html before opening a 
pull request.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/spark branch-2.2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/20569.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #20569


commit 6e1081cbeac58826526b6ff7f2938a556b31ca9e
Author: Sumedh Wale 
Date:   2017-07-06T06:47:22Z

[SPARK-21312][SQL] correct offsetInBytes in UnsafeRow.writeToStream

## What changes were proposed in this pull request?

Corrects offsetInBytes calculation in UnsafeRow.writeToStream. Known 
failures include writes to some DataSources that have own SparkPlan 
implementations and cause EXCHANGE in writes.

## How was this patch tested?

Extended UnsafeRowSuite.writeToStream to include an UnsafeRow over byte 
array having non-zero offset.

Author: Sumedh Wale 

Closes #18535 from sumwale/SPARK-21312.

(cherry picked from commit 14a3bb3a008c302aac908d7deaf0942a98c63be7)
Signed-off-by: Wenchen Fan 

commit 4e53a4edd72e372583f243c660bbcc0572205716
Author: Tathagata Das 
Date:   2017-07-06T07:20:26Z

[SS][MINOR] Fix flaky test in DatastreamReaderWriterSuite. temp checkpoint 
dir should be deleted

## What changes were proposed in this pull request?

Stopping query while it is being initialized can throw interrupt exception, 
in which case temporary checkpoint directories will not be deleted, and the 
test will fail.

Author: Tathagata Das 

Closes #18442 from tdas/DatastreamReaderWriterSuite-fix.

(cherry picked from commit 60043f22458668ac7ecba94fa78953f23a6bdcec)
Signed-off-by: Tathagata Das 

commit 576fd4c3a67b4affc5ac50979e27ae929472f0d9
Author: Tathagata Das 
Date:   2017-07-07T00:28:20Z

[SPARK-21267][SS][DOCS] Update Structured Streaming Documentation

## What changes were proposed in this pull request?

Few changes to the Structured Streaming documentation
- Clarify that the entire stream input table is not materialized
- Add information for Ganglia
- Add Kafka Sink to the main docs
- Removed a couple of leftover experimental tags
- Added more associated reading material and talk videos.

In addition, https://github.com/apache/spark/pull/16856 broke the link to 
the RDD programming guide in several places while renaming the page. This PR 
fixes those sameeragarwal cloud-fan.
- Added a redirection to avoid breaking internal and possible external 
links.
- Removed unnecessary redirection pages that were there since the separate 
scala, java, and python programming guides were merged together in 2013 or 2014.

## How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration 
tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, 
remove this)

Please review http://spark.apache.org/contributing.html before opening a 
pull request.

Author: Tathagata Das 

Closes #18485 from tdas/SPARK-21267.

(cherry picked from commit 0217dfd26f89133f146197359b556c9bf5aca172)
Signed-off-by: Shixiong Zhu 

commit ab12848d624f6b74d401e924255c0b4fcc535231
Author: Prashant Sharma 
Date:   2017-07-08T06:33:12Z

[SPARK-21069][SS][DOCS] Add rate source to programming guide.

## What changes were proposed in this pull request?

SPARK-20979 added a new structured streaming source: Rate source. This 
patch adds the corresponding documentation to programming guide.

## How was this patch tested?

Tested by running jekyll locally.

Author: Prashant Sharma 
Author: Prashant Sharma 

Closes #18562 from ScrapCodes/spark-21069/rate-source-docs.

(cherry picked from commit d0bfc6733521709e453d643582df2bdd68f28de7)
Signed-off-by: Shixiong Zhu 

commit 7d0b1c927d92cc2a4932262514ffd12c47593b80
Author: Bogdan Raducanu 
Date:   

[GitHub] spark pull request #20569: Branch 2.2

[MohammedLayeeq]

Github user MohammedLayeeq closed the pull request at:

https://github.com/apache/spark/pull/20569


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org