[GitHub] spark pull request #20957: Branch 2.3

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/20957


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20957: Branch 2.3

[rameshch16]

GitHub user rameshch16 opened a pull request:

https://github.com/apache/spark/pull/20957

Branch 2.3

## What changes were proposed in this pull request?

(Please fill in changes proposed in this fix)

## How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration 
tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, 
remove this)

Please review http://spark.apache.org/contributing.html before opening a 
pull request.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/spark branch-2.3

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/20957.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #20957


commit acf3b70d16cc4d2416b4ce3f42b3cf95836170ed
Author: Tathagata Das 
Date:   2018-01-19T00:29:45Z

[SPARK-23142][SS][DOCS] Added docs for continuous processing

## What changes were proposed in this pull request?

Added documentation for continuous processing. Modified two locations.
- Modified the overview to have a mention of Continuous Processing.
- Added a new section on Continuous Processing at the end.


![image](https://user-images.githubusercontent.com/663212/35083551-a3dd23f6-fbd4-11e7-9e7e-90866f131ca9.png)

![image](https://user-images.githubusercontent.com/663212/35083618-d844027c-fbd4-11e7-9fde-75992cc517bd.png)

## How was this patch tested?
N/A

Author: Tathagata Das 

Closes #20308 from tdas/SPARK-23142.

(cherry picked from commit 4cd2ecc0c7222fef1337e04f1948333296c3be86)
Signed-off-by: Tathagata Das 

commit 225b1afdd1582cd4087e7cb98834505eaf16743e
Author: brandonJY 
Date:   2018-01-19T00:57:49Z

[DOCS] change to dataset for java code in 
structured-streaming-kafka-integration document

## What changes were proposed in this pull request?

In latest structured-streaming-kafka-integration document, Java code 
example for Kafka integration is using `DataFrame`, shouldn't it be 
changed to `DataSet`?

## How was this patch tested?

manual test has been performed to test the updated example Java code in 
Spark 2.2.1 with Kafka 1.0

Author: brandonJY 

Closes #20312 from brandonJY/patch-2.

(cherry picked from commit 6121e91b7f5c9513d68674e4d5edbc3a4a5fd5fd)
Signed-off-by: Sean Owen 

commit 541dbc00b24f17d83ea2531970f2e9fe57fe3718
Author: Takuya UESHIN 
Date:   2018-01-19T03:37:08Z

[SPARK-23054][SQL][PYSPARK][FOLLOWUP] Use sqlType casting when casting 
PythonUserDefinedType to String.

## What changes were proposed in this pull request?

This is a follow-up of #20246.

If a UDT in Python doesn't have its corresponding Scala UDT, cast to string 
will be the raw string of the internal value, e.g. 
`"org.apache.spark.sql.catalyst.expressions.UnsafeArrayData"` if the 
internal type is `ArrayType`.

This pr fixes it by using its `sqlType` casting.

## How was this patch tested?

Added a test and existing tests.

Author: Takuya UESHIN 

Closes #20306 from ueshin/issues/SPARK-23054/fup1.

(cherry picked from commit 568055da93049c207bb830f244ff9b60c638837c)
Signed-off-by: Wenchen Fan 

commit 54c1fae12df654c7713ac5e7eb4da7bb2f785401
Author: Sameer Agarwal 
Date:   2018-01-19T09:38:08Z

[BUILD][MINOR] Fix java style check issues

## What changes were proposed in this pull request?

This patch fixes a few recently introduced java style check errors in 
master and release branch.

As an aside, given that [java linting currently 
fails](https://github.com/apache/spark/pull/10763
) on machines with a clean maven cache, it'd be great to find another 
workaround to [re-enable the java style 
checks](https://github.com/apache/spark/blob/3a07eff5af601511e97a05e6fea0e3d48f74c4f0/dev/run-tests.py#L577)
 as part of Spark PRB.

/cc zsxwing JoshRosen srowen for any suggestions

## How was this patch tested?

Manual Check

Author: Sameer Agarwal 

Closes #20323 from sameeragarwal/java.

(cherry picked from commit 9c4b99861cda3f9ec44ca8c1adc81a293508190c)
Signed-off-by: Sameer Agarwal 

commit e58223171ecae6450482aadf4e7994c3b8d8a58d
Author: Nick Pentreath 
Date:   2018-01-19T10:43:23Z

[SPARK-23127][DOC] Update FeatureHasher guide for categoricalCols parameter

Update user