[GitHub] spark pull request #15513: [SPARK-17963][SQL][Documentation] Add examples (e...

[HyukjinKwon]

Github user HyukjinKwon commented on a diff in the pull request:

https://github.com/apache/spark/pull/15513#discussion_r85626369
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/nullExpressions.scala
 ---
@@ -88,7 +97,17 @@ case class Coalesce(children: Seq[Expression]) extends 
Expression {
 }
 
 
-@ExpressionDescription(usage = "_FUNC_(a,b) - Returns b if a is null, or a 
otherwise.")
+@ExpressionDescription(
+  usage = "_FUNC_(expr1, expr2) - Returns `expr2` if `expr1` is null, or 
`expr1` otherwise.",
+  extended = """
+Arguments:
+  expr1 - an expression of any type.
+  expr2 - an expression of any type.
--- End diff --

```sql
spark-sql> SELECT ifnull(array(1), array(1)), ifnull(struct(1), struct(1)), 
ifnull(map(1,1), map(1, 1));
[1] {"col1":1}  {1:1}
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15513: [SPARK-17963][SQL][Documentation] Add examples (e...

[HyukjinKwon]

Github user HyukjinKwon commented on a diff in the pull request:

https://github.com/apache/spark/pull/15513#discussion_r85626603
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/nullExpressions.scala
 ---
@@ -126,7 +155,17 @@ case class NullIf(left: Expression, right: Expression) 
extends RuntimeReplaceabl
 }
 
 
-@ExpressionDescription(usage = "_FUNC_(a,b) - Returns b if a is null, or a 
otherwise.")
+@ExpressionDescription(
+  usage = "_FUNC_(expr1, expr2) - Returns `expr2` if `expr1` is null, or 
`expr1` otherwise.",
+  extended = """
+Arguments:
+  expr1 - an expression of any type.
+  expr2 - an expression of any type.
--- End diff --

```sql
spark-sql> SELECT nvl(array(1), array(1)), nvl(struct(1), struct(1)), 
nvl(map(1,1), map(1, 1));
[1] {"col1":1}  {1:1}
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15513: [SPARK-17963][SQL][Documentation] Add examples (e...

[HyukjinKwon]

Github user HyukjinKwon commented on a diff in the pull request:

https://github.com/apache/spark/pull/15513#discussion_r85625648
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala
 ---
@@ -234,7 +259,16 @@ case class CreateStruct(children: Seq[Expression]) 
extends Expression {
  */
 // scalastyle:off line.size.limit
 @ExpressionDescription(
-  usage = "_FUNC_(name1, val1, name2, val2, ...) - Creates a struct with 
the given field names and values.")
+  usage = "_FUNC_(name1, val1, name2, val2, ...) - Creates a struct with 
the given field names and values.",
+  extended = """
+Arguments:
+  name - a string expression literal that represents the field name.
+  val - an expression of any type.
--- End diff --

```sql
spark-sql> SELECT named_struct('1', array(1)), named_struct('1', 
struct(1)), named_struct('1', map(1, 1));
{"1":[1]}   {"1":{"col1":1}}{"1":{1:1}}
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15513: [SPARK-17963][SQL][Documentation] Add examples (e...

[HyukjinKwon]

Github user HyukjinKwon commented on a diff in the pull request:

https://github.com/apache/spark/pull/15513#discussion_r85625439
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala
 ---
@@ -28,7 +28,15 @@ import org.apache.spark.unsafe.types.UTF8String
  * Returns an Array containing the evaluation of all children expressions.
  */
 @ExpressionDescription(
-  usage = "_FUNC_(n0, ...) - Returns an array with the given elements.")
+  usage = "_FUNC_(expr, ...) - Returns an array with the given elements.",
+  extended = """
+Arguments:
+  expr - an expression of any type.
--- End diff --

```sql
spark-sql> SELECT array(array(1)), array(struct(1)), array(map(1, 1));
[[1]]   [{"col1":1}][{1:1}]
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15513: [SPARK-17963][SQL][Documentation] Add examples (e...

[HyukjinKwon]

Github user HyukjinKwon commented on a diff in the pull request:

https://github.com/apache/spark/pull/15513#discussion_r85626234
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala
 ---
@@ -631,7 +682,11 @@ case class CurrentDatabase() extends LeafExpression 
with Unevaluable {
  * TODO: Support Decimal and date related types
  */
 @ExpressionDescription(
-  usage = "_FUNC_(a1, a2, ...) - Returns a hash value of the arguments.")
+  usage = "_FUNC_(expr1, expr2, ...) - Returns a hash value of the 
arguments.",
+  extended = """
+Arguments:
+  expr - an expression of any type.
--- End diff --

```sql
spark-sql> SELECT hash(struct(1));
-559580957
Time taken: 0.027 seconds, Fetched 1 row(s)
spark-sql> SELECT hash(array(1));
-559580957
Time taken: 0.021 seconds, Fetched 1 row(s)
spark-sql> SELECT hash(map(1, 1));
245521047
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15513: [SPARK-17963][SQL][Documentation] Add examples (e...

[HyukjinKwon]

Github user HyukjinKwon commented on a diff in the pull request:

https://github.com/apache/spark/pull/15513#discussion_r85625713
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Max.scala
 ---
@@ -23,7 +23,11 @@ import org.apache.spark.sql.catalyst.util.TypeUtils
 import org.apache.spark.sql.types._
 
 @ExpressionDescription(
-  usage = "_FUNC_(expr) - Returns the maximum value of expr.")
+  usage = "_FUNC_(expr) - Returns the maximum value of `expr`.",
+  extended = """
+Arguments:
+  expr - an expression of any type.
+  """)
--- End diff --

```sql
SELECT max(array(1)), max(struct(1));
[1] {"col1":1}
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15513: [SPARK-17963][SQL][Documentation] Add examples (e...

[HyukjinKwon]

Github user HyukjinKwon commented on a diff in the pull request:

https://github.com/apache/spark/pull/15513#discussion_r85624958
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/HyperLogLogPlusPlus.scala
 ---
@@ -47,10 +47,16 @@ import org.apache.spark.sql.types._
  */
 // scalastyle:on
 @ExpressionDescription(
-  usage = """_FUNC_(expr) - Returns the estimated cardinality by 
HyperLogLog++.
-_FUNC_(expr, relativeSD=0.05) - Returns the estimated cardinality by 
HyperLogLog++
-  with relativeSD, the maximum estimation error allowed.
-""")
+  usage = """
+_FUNC_(expr[, relativeSD]) - Returns the estimated cardinality by 
HyperLogLog++.
+  `relativeSD` defines the maximum estimation error allowed.
+  """,
+  extended = """
+Arguments:
+  expr - an expression of any type that represents data to count.
--- End diff --

```sql
spark-sql> SELECT approx_count_distinct(array(1)), 
approx_count_distinct(struct(1)), approx_count_distinct(map(1,1));
1   1   1
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15513: [SPARK-17963][SQL][Documentation] Add examples (e...

[HyukjinKwon]

Github user HyukjinKwon commented on a diff in the pull request:

https://github.com/apache/spark/pull/15513#discussion_r85626188
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala
 ---
@@ -102,8 +102,17 @@ case class UserDefinedGenerator(
  * }}}
  */
 @ExpressionDescription(
-  usage = "_FUNC_(n, v1, ..., vk) - Separate v1, ..., vk into n rows.",
-  extended = "> SELECT _FUNC_(2, 1, 2, 3);\n  [1,2]\n  [3,null]")
+  usage = "_FUNC_(n, expr1, ..., exprk) - Separates `expr1`, ..., `exprk` 
into `n` rows.",
+  extended = """
+Arguments:
+  n - an integer literal that represents the number of output rows.
+  expr - an expression of any type.
--- End diff --

```sql
spark-sql> SELECT stack(2, array(1));
[1]
NULL
spark-sql> SELECT stack(2, map(1, 1));
{1:1}
NULL
spark-sql> SELECT stack(2, struct(1));
{"col1":1}
NULL
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15513: [SPARK-17963][SQL][Documentation] Add examples (e...

[HyukjinKwon]

Github user HyukjinKwon commented on a diff in the pull request:

https://github.com/apache/spark/pull/15513#discussion_r85624910
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/First.scala
 ---
@@ -29,10 +29,16 @@ import org.apache.spark.sql.types._
  * a single partition, and we use a single reducer to do the aggregation.).
  */
 @ExpressionDescription(
-  usage = """_FUNC_(expr) - Returns the first value of `child` for a group 
of rows.
-_FUNC_(expr,isIgnoreNull=false) - Returns the first value of `child` 
for a group of rows.
-  If isIgnoreNull is true, returns only non-null values.
-""")
+  usage = """
+_FUNC_(expr[, isIgnoreNull]) - Returns the first value of `expr` for a 
group of rows.
+  If `isIgnoreNull` is true, returns only non-null values.
+  """,
+  extended = """
+Arguments:
+  expr - an expression of any type that represents data to collect the 
first.
--- End diff --

```sql
spark-sql> SELECT first(array(1)), first(struct(1)), first(map(1,1));
[1] {"col1":1}  {1:1}
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15513: [SPARK-17963][SQL][Documentation] Add examples (e...

[HyukjinKwon]

Github user HyukjinKwon commented on a diff in the pull request:

https://github.com/apache/spark/pull/15513#discussion_r85626782
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala
 ---
@@ -114,7 +118,11 @@ case class Not(child: Expression)
  * Evaluates to `true` if `list` contains `value`.
  */
 @ExpressionDescription(
-  usage = "expr _FUNC_(val1, val2, ...) - Returns true if expr equals to 
any valN.")
+  usage = "expr1 _FUNC_(expr2, expr3, ...) - Returns true if `expr` equals 
to any valN.",
+  extended = """
+Arguments:
+  expr - an expression of any type.
--- End diff --

```sql
spark-sql> SELECT array(1) in(array(1)), struct(1) in(struct(1)), map(1,1) 
in(map(1, 1));
truetruefalse
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15513: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15513
  
**[Test build #67732 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67732/consoleFull)**
 for PR 15513 at commit 
[`2b437fe`](https://github.com/apache/spark/commit/2b437fe169080b53215f280c8987ff1d8e779df8).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15513: [SPARK-17963][SQL][Documentation] Add examples (e...

[HyukjinKwon]

Github user HyukjinKwon closed the pull request at:

https://github.com/apache/spark/pull/15513


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15513: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

[HyukjinKwon]

Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/15513
  
Let me close and reopen another. It is really messy.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15677: [SPARK-17963][SQL][Documentation] Add examples (e...

[HyukjinKwon]

GitHub user HyukjinKwon opened a pull request:

https://github.com/apache/spark/pull/15677

[SPARK-17963][SQL][Documentation] Add examples (extend) in each expression 
and improve documentation with arguments

## What changes were proposed in this pull request?

This PR proposes to change the documentation for functions. Please refer 
the discussion from https://github.com/apache/spark/pull/15513

The changes include

 - Re-indent the documentation
 - Add examples/arguments in `extended` where the arguments are multiple or 
specific format (e.g. xml/ json).

For examples, the documentation was updated as below:

### Functions with single line usage

**Before**
  - `pow`

```sql
Usage: pow(x1, x2) - Raise x1 to the power of x2.
Extended Usage:
> SELECT pow(2, 3);
 8.0
```

  - `current_timestamp`

```sql
Usage: current_timestamp() - Returns the current timestamp at the start 
of query evaluation.
Extended Usage:
No example for current_timestamp.
```

**After**

  - `pow`

```sql
Usage: pow(expr1, expr2) - Raise expr1 to the power of expr2.
Extended Usage:
Arguments:
  expr1 - a numeric expression.
  expr2 - a numeric expression.

Examples:
  > SELECT pow(2, 3);
   8.0
```

  - `current_timestamp`

```sql
Usage: current_timestamp() - Returns the current timestamp at the start 
of query evaluation.
Extended Usage:
No example/arguemnt for current_timestamp.
```


### Functions with (already) multiple line usage

**Before**

  - `approx_count_distinct`

```sql
Usage: approx_count_distinct(expr) - Returns the estimated cardinality 
by HyperLogLog++.
approx_count_distinct(expr, relativeSD=0.05) - Returns the 
estimated cardinality by HyperLogLog++
  with relativeSD, the maximum estimation error allowed.

Extended Usage:
No example for approx_count_distinct.
```

  - `percentile_approx`

```sql
Usage:
  percentile_approx(col, percentage [, accuracy]) - Returns the 
approximate percentile value of numeric
  column `col` at the given percentage. The value of percentage 
must be between 0.0
  and 1.0. The `accuracy` parameter (default: 1) is a positive 
integer literal which
  controls approximation accuracy at the cost of memory. Higher 
value of `accuracy` yields
  better accuracy, `1.0/accuracy` is the relative error of the 
approximation.

  percentile_approx(col, array(percentage1 [, percentage2]...) [, 
accuracy]) - Returns the approximate
  percentile array of column `col` at the given percentage array. 
Each value of the
  percentage array must be between 0.0 and 1.0. The `accuracy` 
parameter (default: 1) is
  a positive integer literal which controls approximation accuracy 
at the cost of memory.
  Higher value of `accuracy` yields better accuracy, `1.0/accuracy` 
is the relative error of
  the approximation.

Extended Usage:
No example for percentile_approx.
```

**After**

  - `approx_count_distinct`

```sql
Usage:
approx_count_distinct(expr[, relativeSD]) - Returns the estimated 
cardinality by HyperLogLog++.
  relativeSD defines the maximum estimation error allowed.

Extended Usage:
Arguments:
  expr - an expression of any type that represents data to count.
  relativeSD - a numeric literal that defines the maximum 
estimation error allowed.
```

  - `percentile_approx`

```sql
Usage:
percentile_approx(col, percentage [, accuracy]) - Returns the 
approximate percentile value of numeric
  column `col` at the given percentage. The value of `percentage` 
must be between 0.0
  and 1.0. The `accuracy` parameter (default: 1) is a positive 
integer literal which
  controls approximation accuracy at the cost of memory. Higher 
value of `accuracy` yields
  better accuracy, `1.0/accuracy` is the relative error of the 
approximation.
  When `percentage` is an array, each value of the percentage array 
must be between 0.0 and 1.0.

Extended Usage:
Arguments:
  col - a numeric expression.
  percentage - a numeric literal or an array literal of numeric 
type that defines the
percentile. For example, 0.5 means 50-percentile.
  accuracy - a numeric literal.

Examples:
 

[GitHub] spark issue #15677: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15677
  
**[Test build #67733 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67733/consoleFull)**
 for PR 15677 at commit 
[`2b437fe`](https://github.com/apache/spark/commit/2b437fe169080b53215f280c8987ff1d8e779df8).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15677: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

[HyukjinKwon]

Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/15677
  
cc @srowen, @rxin, @jodersky, @gatorsmile . I closed the previous one and 
reopened it here.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15677: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

[HyukjinKwon]

Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/15677
  
@gatorsmile I double-checked the type ones again and tried to describe the 
types more specifically. Could you please take another look?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15678: Merge pull request #1 from apache/master

[wwjiang007]

GitHub user wwjiang007 opened a pull request:

https://github.com/apache/spark/pull/15678

Merge pull request #1 from apache/master

## What changes were proposed in this pull request?

(Please fill in changes proposed in this fix)

## How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration 
tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, 
remove this)

Please review 
https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark before 
opening a pull request.

update from spark

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/wwjiang007/spark master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/15678.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #15678


commit c093771be56b29c703c147cd6ce3b6f72ee0d194
Author: wwjiang007 
Date:   2016-10-26T15:16:31Z

Merge pull request #1 from apache/master

update from spark




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15678: Merge pull request #1 from apache/master

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15678
  
Can one of the admins verify this patch?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15219: [SPARK-14098][SQL] Generate Java code to build CachedCol...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15219
  
**[Test build #67726 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67726/consoleFull)**
 for PR 15219 at commit 
[`73b2760`](https://github.com/apache/spark/commit/73b27608ffee3e67fe55f052cadcddcd533cd12f).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15219: [SPARK-14098][SQL] Generate Java code to build CachedCol...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15219
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67726/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15219: [SPARK-14098][SQL] Generate Java code to build CachedCol...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15219
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15677: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

[HyukjinKwon]

Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/15677
  
BTW, I hope we can fix up all the minor comments together once as a final 
look if each does not block each other.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15663: [SPARK-18143][SQL]Ignore Structured Streaming event logs...

[zsxwing]

Github user zsxwing commented on the issue:

https://github.com/apache/spark/pull/15663
  
Updated. 2.0.1 event logs don't include the `"stackTrace"` field.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15663: [SPARK-18143][SQL]Ignore Structured Streaming event logs...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15663
  
**[Test build #67734 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67734/consoleFull)**
 for PR 15663 at commit 
[`d30a3a5`](https://github.com/apache/spark/commit/d30a3a5591a485c2a7b6e12c2b006f79ab955db9).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15626
  
**[Test build #67723 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67723/consoleFull)**
 for PR 15626 at commit 
[`4663d5e`](https://github.com/apache/spark/commit/4663d5ed1b1dbfa4e154c1d16d51ef4c63db7c41).
 * This patch **fails from timeout after a configured wait of \`250m\`**.
 * This patch merges cleanly.
 * This patch adds the following public classes _(experimental)_:
  * `class KafkaSourceOffsetSuite extends OffsetSuite with SharedSQLContext 
`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15626
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67723/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15626
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15679: [SPARK-16312][Follow-up][STREAMING][KAFKA][DOC] A...

[lw-lin]

GitHub user lw-lin opened a pull request:

https://github.com/apache/spark/pull/15679

[SPARK-16312][Follow-up][STREAMING][KAFKA][DOC] Add java code snippet for 
Kafka 0.10 integration doc

## What changes were proposed in this pull request?

added java code snippet for Kafka 0.10 integration doc

## How was this patch tested?

SKIP_API=1 jekyll build

## Screenshot


![kafka-doc](https://cloud.githubusercontent.com/assets/15843379/19826272/bf0d8a4c-9db8-11e6-9e40-1396723df4bc.png)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/lw-lin/spark kafka-010-examples

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/15679.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #15679


commit 9dc8e0fa6e679eb511b70a1796d9ae127661a95e
Author: Liwei Lin 
Date:   2016-10-28T09:07:42Z

Add Java code snippet for Kafka guide




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15354: [SPARK-17764][SQL] Add `to_json` supporting to co...

[HyukjinKwon]

Github user HyukjinKwon commented on a diff in the pull request:

https://github.com/apache/spark/pull/15354#discussion_r85630778
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala
 ---
@@ -494,3 +495,46 @@ case class JsonToStruct(schema: StructType, options: 
Map[String, String], child:
 
   override def inputTypes: Seq[AbstractDataType] = StringType :: Nil
 }
+
+/**
+ * Converts a [[StructType]] to a json output string.
+ */
+case class StructToJson(options: Map[String, String], child: Expression)
+  extends Expression with CodegenFallback with ExpectsInputTypes {
+  override def nullable: Boolean = true
+
+  @transient
+  lazy val writer = new CharArrayWriter()
+
+  @transient
+  lazy val gen =
+new JacksonGenerator(child.dataType.asInstanceOf[StructType], writer)
+
+  override def dataType: DataType = StringType
+  override def children: Seq[Expression] = child :: Nil
+
+  override def checkInputDataTypes(): TypeCheckResult = {
+if (StructType.acceptsType(child.dataType)) {
+  try {
--- End diff --

Ah, yes, makes sense but if `verifySchema` returns a boolean, we could not 
find which field and type are problematic.

Maybe, I can make do one of the below:
  - this logic in `verifySchema` into `checkInputDataTypes`
  - `verifySchema` returns the unsupported fields. and types.
  - Just fix the exception message without the information of unsupported 
fields and types.

If you pick one, I will follow (or please let me know if there is a better 
way)!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15679: [SPARK-16312][Follow-up][STREAMING][KAFKA][DOC] Add java...

[lw-lin]

Github user lw-lin commented on the issue:

https://github.com/apache/spark/pull/15679
  
@koeninger @zsxwing it'd be great if you can take a look, thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15679: [SPARK-16312][Follow-up][STREAMING][KAFKA][DOC] Add java...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15679
  
**[Test build #67735 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67735/consoleFull)**
 for PR 15679 at commit 
[`9dc8e0f`](https://github.com/apache/spark/commit/9dc8e0fa6e679eb511b70a1796d9ae127661a95e).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15354
  
**[Test build #67736 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67736/consoleFull)**
 for PR 15354 at commit 
[`b76a08e`](https://github.com/apache/spark/commit/b76a08ec0513b34c540db745e7511330d7fcb56b).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15618: [SPARK-14914][CORE] Fix Resource not closed after...

[HyukjinKwon]

Github user HyukjinKwon commented on a diff in the pull request:

https://github.com/apache/spark/pull/15618#discussion_r85630990
  
--- Diff: 
core/src/test/scala/org/apache/spark/scheduler/TaskResultGetterSuite.scala ---
@@ -209,7 +209,8 @@ class TaskResultGetterSuite extends SparkFunSuite with 
BeforeAndAfter with Local
 
   assert(expectedFailure.findFirstMatchIn(exceptionMessage).isDefined)
   assert(unknownFailure.findFirstMatchIn(exceptionMessage).isEmpty)
-} finally {
+} {
+  loader.close()
   Thread.currentThread.setContextClassLoader(originalClassLoader)
--- End diff --

Sure, makes sense.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to be pip i...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #67729 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67729/consoleFull)**
 for PR 15659 at commit 
[`ab8ca53`](https://github.com/apache/spark/commit/ab8ca53465cdac5e2af4b365830f8904d166e3da).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to be pip i...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to be pip i...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67729/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15679: [SPARK-16312][Follow-up][STREAMING][KAFKA][DOC] Add java...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15679
  
**[Test build #67735 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67735/consoleFull)**
 for PR 15679 at commit 
[`9dc8e0f`](https://github.com/apache/spark/commit/9dc8e0fa6e679eb511b70a1796d9ae127661a95e).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15679: [SPARK-16312][Follow-up][STREAMING][KAFKA][DOC] Add java...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15679
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15679: [SPARK-16312][Follow-up][STREAMING][KAFKA][DOC] Add java...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15679
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67735/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15679: [SPARK-16312][Follow-up][STREAMING][KAFKA][DOC] Add java...

[lw-lin]

Github user lw-lin commented on the issue:

https://github.com/apache/spark/pull/15679
  
Oh there's a on-going 2.0.2 RC cut. It'd be great if we can make this into 
2.0.2. So also cc @srowen to take a look, thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15674: [SPARK-18164][SQL]ForeachSink should fail the Spark job ...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15674
  
**[Test build #67730 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67730/consoleFull)**
 for PR 15674 at commit 
[`a214755`](https://github.com/apache/spark/commit/a2147558e4aec77c3f461ebe6eb40fa58b8b1092).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15674: [SPARK-18164][SQL]ForeachSink should fail the Spark job ...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15674
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67730/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15675: [SPARK-18144][SQL] logging StreamingQueryListener$QueryS...

[CodingCat]

Github user CodingCat commented on the issue:

https://github.com/apache/spark/pull/15675
  
Sorry, I misunderstood the real reason of the bug

The real reason is actually very simple...the original postToAll() is 
actually calling `StreamingQueryListenerBus.postToAll()` which has no listener 
at allwe shall post by `sparkListenerBus.postToAll(s)` and 
`this.postToAll()` to trigger local listeners as well as the listeners 
registered in LiveListenerBus


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15674: [SPARK-18164][SQL]ForeachSink should fail the Spark job ...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15674
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15626
  
**[Test build #67724 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67724/consoleFull)**
 for PR 15626 at commit 
[`d62a877`](https://github.com/apache/spark/commit/d62a87782b1fc204000d490a4200cd1afc86f803).
 * This patch **fails from timeout after a configured wait of \`250m\`**.
 * This patch merges cleanly.
 * This patch adds the following public classes _(experimental)_:
  * `class OffsetSeqLog(sparkSession: SparkSession, path: String)`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15675: [SPARK-18144][SQL] logging StreamingQueryListener$QueryS...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15675
  
**[Test build #67737 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67737/consoleFull)**
 for PR 15675 at commit 
[`24bcd7b`](https://github.com/apache/spark/commit/24bcd7b700627dff961ac6e405922babf5d1851f).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15626
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67724/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15678: Merge pull request #1 from apache/master

[HyukjinKwon]

Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/15678
  
@wwjiang007 Could you please close this? it seems mistakenly opened.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15626
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13771: [SPARK-13748][PYSPARK][DOC] Add the description for expl...

[HyukjinKwon]

Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/13771
  
gentle ping @davies (If you are not sure of this change, I can close and 
take an action for the JIRA).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14627: [SPARK-16975][SQL][FOLLOWUP] Do not duplicately check fi...

[HyukjinKwon]

Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/14627
  
(gentle ping @liancheng)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14660: [SPARK-17071][SQL] Add an option to support for reading ...

[HyukjinKwon]

Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/14660
  
(ping @liancheng ..)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15361: [SPARK-17765][SQL] Support for writing out user-defined ...

[HyukjinKwon]

Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/15361
  
(gentle ping @chenghao-intel @davies)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15618: [SPARK-14914][CORE] Fix Resource not closed after using,...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15618
  
**[Test build #67738 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67738/consoleFull)**
 for PR 15618 at commit 
[`3949dbe`](https://github.com/apache/spark/commit/3949dbeda632e215c64dcc089365bdc7334dacaf).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15663: [SPARK-18143][SQL]Ignore Structured Streaming event logs...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15663
  
**[Test build #67731 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67731/consoleFull)**
 for PR 15663 at commit 
[`729d50d`](https://github.com/apache/spark/commit/729d50d5aa81e3b47df9d4f2f55204cdce2f2c29).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15663: [SPARK-18143][SQL]Ignore Structured Streaming event logs...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15663
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67731/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15663: [SPARK-18143][SQL]Ignore Structured Streaming event logs...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15663
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15626
  
**[Test build #67739 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67739/consoleFull)**
 for PR 15626 at commit 
[`9298022`](https://github.com/apache/spark/commit/9298022eabb9f1a48b0d90f415e3a658f702c420).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15612: [SPARK-18078] Add zipPartitionsWithPreferredLocation fun...

[WeichenXu123]

Github user WeichenXu123 commented on the issue:

https://github.com/apache/spark/pull/15612
  
Jenkins, test this please.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15612: [SPARK-18078] Add zipPartitionsWithPreferredLocation fun...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15612
  
**[Test build #67740 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67740/consoleFull)**
 for PR 15612 at commit 
[`439def9`](https://github.com/apache/spark/commit/439def9e5fd46f0650cba24c9232c3f62349023e).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15513: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15513
  
**[Test build #67732 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67732/consoleFull)**
 for PR 15513 at commit 
[`2b437fe`](https://github.com/apache/spark/commit/2b437fe169080b53215f280c8987ff1d8e779df8).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15513: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15513
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15513: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15513
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67732/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15663: [SPARK-18143][SQL]Ignore Structured Streaming event logs...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15663
  
**[Test build #67734 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67734/consoleFull)**
 for PR 15663 at commit 
[`d30a3a5`](https://github.com/apache/spark/commit/d30a3a5591a485c2a7b6e12c2b006f79ab955db9).
 * This patch **fails PySpark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15663: [SPARK-18143][SQL]Ignore Structured Streaming event logs...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15663
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15663: [SPARK-18143][SQL]Ignore Structured Streaming event logs...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15663
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67734/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15677: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15677
  
**[Test build #67733 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67733/consoleFull)**
 for PR 15677 at commit 
[`2b437fe`](https://github.com/apache/spark/commit/2b437fe169080b53215f280c8987ff1d8e779df8).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15677: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15677
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67733/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15677: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15677
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15671: [SPARK-14567][ML]Add instrumentation logs to MLlib train...

[zhengruifeng]

Github user zhengruifeng commented on the issue:

https://github.com/apache/spark/pull/15671
  
@sethah Thanks. I have revert those changes in Tree-Algos.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15671: [SPARK-14567][ML]Add instrumentation logs to MLlib train...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15671
  
**[Test build #67741 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67741/consoleFull)**
 for PR 15671 at commit 
[`05e2f4d`](https://github.com/apache/spark/commit/05e2f4d26d4f160334abfa3da1ecd985f0d138ad).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15663: [SPARK-18143][SQL]Ignore Structured Streaming event logs...

[zsxwing]

Github user zsxwing commented on the issue:

https://github.com/apache/spark/pull/15663
  
retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15674: [SPARK-18164][SQL]ForeachSink should fail the Spark job ...

[zsxwing]

Github user zsxwing commented on the issue:

https://github.com/apache/spark/pull/15674
  
Merging to master and 2.0. Thanks, @tdas 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15674: [SPARK-18164][SQL]ForeachSink should fail the Spa...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/15674


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15663: [SPARK-18143][SQL]Ignore Structured Streaming event logs...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15663
  
**[Test build #67742 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67742/consoleFull)**
 for PR 15663 at commit 
[`d30a3a5`](https://github.com/apache/spark/commit/d30a3a5591a485c2a7b6e12c2b006f79ab955db9).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15680: sync

[zhangxinyu1]

GitHub user zhangxinyu1 opened a pull request:

https://github.com/apache/spark/pull/15680

sync

## What changes were proposed in this pull request?

(Please fill in changes proposed in this fix)

## How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration 
tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, 
remove this)

Please review 
https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark before 
opening a pull request.

sync

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/zhangxinyu1/spark master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/15680.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #15680


commit a0683680ac86791cb9b5fe5aa644b562e1a4c08e
Author: zhangxinyu1 <342689...@qq.com>
Date:   2016-10-20T10:37:16Z

Merge pull request #1 from apache/master

sync




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15680: sync

[zhangxinyu1]

Github user zhangxinyu1 closed the pull request at:

https://github.com/apache/spark/pull/15680


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15354
  
**[Test build #67736 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67736/consoleFull)**
 for PR 15354 at commit 
[`b76a08e`](https://github.com/apache/spark/commit/b76a08ec0513b34c540db745e7511330d7fcb56b).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15354
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67736/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15354
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15471: [SPARK-17919] Make timeout to RBackend configurable in S...

[falaki]

Github user falaki commented on the issue:

https://github.com/apache/spark/pull/15471
  
@shivaram is there a chance this makes it to the 2.0.2 release?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15618: [SPARK-14914][CORE] Fix Resource not closed after using,...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15618
  
**[Test build #67738 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67738/consoleFull)**
 for PR 15618 at commit 
[`3949dbe`](https://github.com/apache/spark/commit/3949dbeda632e215c64dcc089365bdc7334dacaf).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15618: [SPARK-14914][CORE] Fix Resource not closed after using,...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15618
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67738/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15618: [SPARK-14914][CORE] Fix Resource not closed after using,...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15618
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15471: [SPARK-17919] Make timeout to RBackend configurable in S...

[shivaram]

Github user shivaram commented on the issue:

https://github.com/apache/spark/pull/15471
  
Taking another look now


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15671: [SPARK-14567][ML]Add instrumentation logs to MLlib train...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15671
  
**[Test build #67741 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67741/consoleFull)**
 for PR 15671 at commit 
[`05e2f4d`](https://github.com/apache/spark/commit/05e2f4d26d4f160334abfa3da1ecd985f0d138ad).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15671: [SPARK-14567][ML]Add instrumentation logs to MLlib train...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15671
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67741/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15671: [SPARK-14567][ML]Add instrumentation logs to MLlib train...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15671
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15675: [SPARK-18144][SQL] logging StreamingQueryListener$QueryS...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15675
  
**[Test build #67737 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67737/consoleFull)**
 for PR 15675 at commit 
[`24bcd7b`](https://github.com/apache/spark/commit/24bcd7b700627dff961ac6e405922babf5d1851f).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15675: [SPARK-18144][SQL] logging StreamingQueryListener$QueryS...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15675
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15618: [SPARK-14914][CORE] Fix Resource not closed after using,...

[HyukjinKwon]

Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/15618
  
retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15675: [SPARK-18144][SQL] logging StreamingQueryListener$QueryS...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15675
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67737/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15618: [SPARK-14914][CORE] Fix Resource not closed after using,...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15618
  
**[Test build #67743 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67743/consoleFull)**
 for PR 15618 at commit 
[`3949dbe`](https://github.com/apache/spark/commit/3949dbeda632e215c64dcc089365bdc7334dacaf).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15666: [SPARK-11421] [Core][Python][R] Added ability for addJar...

[shivaram]

Github user shivaram commented on the issue:

https://github.com/apache/spark/pull/15666
  
Jenkins, ok to test


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #12257: [SPARK-14483][WEBUI] Display user name for each job and ...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/12257
  
**[Test build #67745 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67745/consoleFull)**
 for PR 12257 at commit 
[`8a99f50`](https://github.com/apache/spark/commit/8a99f504df93732368ffcd255af4d846449dc757).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15666: [SPARK-11421] [Core][Python][R] Added ability for addJar...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15666
  
**[Test build #67744 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67744/consoleFull)**
 for PR 15666 at commit 
[`2b1e98e`](https://github.com/apache/spark/commit/2b1e98e50feb7180b94f7b9e304634566f163718).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15666: [SPARK-11421] [Core][Python][R] Added ability for addJar...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15666
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67744/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15666: [SPARK-11421] [Core][Python][R] Added ability for addJar...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15666
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15666: [SPARK-11421] [Core][Python][R] Added ability for addJar...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15666
  
**[Test build #67744 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67744/consoleFull)**
 for PR 15666 at commit 
[`2b1e98e`](https://github.com/apache/spark/commit/2b1e98e50feb7180b94f7b9e304634566f163718).
 * This patch **fails R style tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org