[GitHub] spark issue #20225: [SPARK-23033] Don't use task level retry for continuous ...

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20225 **[Test build #85997 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85997/testReport)** for PR 20225 at commit

[GitHub] spark issue #20243: [SPARK-23052][SS] Migrate ConsoleSink to data source V2 ...

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20243 **[Test build #86018 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86018/testReport)** for PR 20243 at commit

[GitHub] spark issue #20243: [SPARK-23052][SS] Migrate ConsoleSink to data source V2 ...

2018-01-11 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20243 I split off PackedRowWriterFactory with the intent to refactor MemorySinkV2 to use it later, but I just realized no refactoring is actually needed. So I've slotted it in. ---

[GitHub] spark pull request #20183: [SPARK-22986][Core] Use a cache to avoid instanti...

2018-01-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20183#discussion_r161134415 --- Diff: core/src/main/scala/org/apache/spark/broadcast/BroadcastManager.scala --- @@ -52,6 +54,10 @@ private[spark] class BroadcastManager(

[GitHub] spark issue #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpark

2018-01-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20217 To be consistent with Java/Scala API, I think we should only add this new API to `UDFRegistration`. We should also move the `registerFunction` to there too, and deprecate it in `Catalog`.

[GitHub] spark issue #20243: [SPARK-23052] Migrate ConsoleSink to data source V2 api.

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20243 **[Test build #86017 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86017/testReport)** for PR 20243 at commit

[GitHub] spark pull request #20243: [SPARK-23052] Migrate ConsoleSink to data source ...

2018-01-11 Thread jose-torres
GitHub user jose-torres opened a pull request: https://github.com/apache/spark/pull/20243 [SPARK-23052] Migrate ConsoleSink to data source V2 api. ## What changes were proposed in this pull request? Migrate ConsoleSink to data source V2 api. Note that this includes

[GitHub] spark pull request #20183: [SPARK-22986][Core] Use a cache to avoid instanti...

2018-01-11 Thread ho3rexqj
Github user ho3rexqj commented on a diff in the pull request: https://github.com/apache/spark/pull/20183#discussion_r161133496 --- Diff: core/src/main/scala/org/apache/spark/broadcast/BroadcastManager.scala --- @@ -52,6 +54,10 @@ private[spark] class BroadcastManager(

[GitHub] spark issue #20242: [MINOR][BUILD] Fix Java linter errors

2018-01-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20242 Thank you for review and approval, @HyukjinKwon . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #20183: [SPARK-22986][Core] Use a cache to avoid instanti...

2018-01-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20183#discussion_r161132849 --- Diff: core/src/main/scala/org/apache/spark/broadcast/BroadcastManager.scala --- @@ -52,6 +54,10 @@ private[spark] class BroadcastManager(

[GitHub] spark issue #20183: [SPARK-22986][Core] Use a cache to avoid instantiating m...

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20183 **[Test build #86016 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86016/testReport)** for PR 20183 at commit

[GitHub] spark issue #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpark

2018-01-11 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20217 Yea, @gatorsmile's https://github.com/apache/spark/pull/20217#issuecomment-357131129 this was exactly what was on my mind. I have few arguments to

[GitHub] spark pull request #20183: [SPARK-22986][Core] Use a cache to avoid instanti...

2018-01-11 Thread ho3rexqj
Github user ho3rexqj commented on a diff in the pull request: https://github.com/apache/spark/pull/20183#discussion_r161132399 --- Diff: core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala --- @@ -206,37 +206,51 @@ private[spark] class TorrentBroadcast[T:

[GitHub] spark issue #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpark

2018-01-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20217 Deprecating `registerFunction(name, f, returnType)` is not acceptable in Spark 2.x releases. They still call `registerFunction(name, f)` when `f` is a UDF based on my above comment.

[GitHub] spark issue #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpark

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20217 **[Test build #86011 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86011/testReport)** for PR 20217 at commit

[GitHub] spark issue #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpark

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20217 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86011/ Test PASSed. ---

[GitHub] spark issue #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpark

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20217 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20183: [SPARK-22986][Core] Use a cache to avoid instanti...

2018-01-11 Thread ho3rexqj
Github user ho3rexqj commented on a diff in the pull request: https://github.com/apache/spark/pull/20183#discussion_r161132203 --- Diff: core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala --- @@ -206,37 +206,51 @@ private[spark] class TorrentBroadcast[T:

[GitHub] spark pull request #20183: [SPARK-22986][Core] Use a cache to avoid instanti...

2018-01-11 Thread ho3rexqj
Github user ho3rexqj commented on a diff in the pull request: https://github.com/apache/spark/pull/20183#discussion_r161132057 --- Diff: core/src/main/scala/org/apache/spark/broadcast/BroadcastManager.scala --- @@ -52,6 +54,10 @@ private[spark] class BroadcastManager(

[GitHub] spark issue #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpark

2018-01-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20217 Another way is to change the default value of `returnType` of `registerFunction` to None. To avoid the behavior change, we can set `returnType` for Python functions to `StringType` internally.

[GitHub] spark issue #20222: [SPARK-23028] Bump master branch version to 2.4.0-SNAPSH...

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20222 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85996/ Test FAILed. ---

[GitHub] spark issue #20222: [SPARK-23028] Bump master branch version to 2.4.0-SNAPSH...

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20222 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20222: [SPARK-23028] Bump master branch version to 2.4.0-SNAPSH...

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20222 **[Test build #85996 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85996/testReport)** for PR 20222 at commit

[GitHub] spark issue #20242: [MINOR][BUILD] Fix Java linter errors

2018-01-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20242 cc @srowen , @HyukjinKwon , @sameeragarwal . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20242: [MINOR][BUILD] Fix Java linter errors

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20242 **[Test build #86015 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86015/testReport)** for PR 20242 at commit

[GitHub] spark issue #20189: [SPARK-22975][SS] MetricsReporter should not throw excep...

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20189 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20189: [SPARK-22975][SS] MetricsReporter should not throw excep...

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20189 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85998/ Test FAILed. ---

[GitHub] spark issue #20189: [SPARK-22975][SS] MetricsReporter should not throw excep...

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20189 **[Test build #85998 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85998/testReport)** for PR 20189 at commit

[GitHub] spark pull request #20242: [MINOR][BUILD] Fix Java linter errors

2018-01-11 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/20242 [MINOR][BUILD] Fix Java linter errors ## What changes were proposed in this pull request? This PR cleans up the java-lint errors (for v2.3.0-rc1 tag). Hopefully, this will be the

[GitHub] spark issue #20240: [SPARK-23049][SQL] `spark.sql.files.ignoreCorruptFiles` ...

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20240 **[Test build #86014 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86014/testReport)** for PR 20240 at commit

[GitHub] spark issue #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpark

2018-01-11 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/20217 My two cents: I kind of like the PySpark API `registerUDF`. I think the API is simple and clear, comparing to the alternative (merging this API to `registerFunction` and throwing an

[GitHub] spark pull request #20214: [SPARK-23023][SQL] Cast field data to strings in ...

2018-01-11 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20214#discussion_r161131086 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -237,13 +237,18 @@ class Dataset[T] private[sql]( private[sql] def

[GitHub] spark issue #19001: [SPARK-19256][SQL] Hive bucketing support

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19001 **[Test build #86013 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86013/testReport)** for PR 19001 at commit

[GitHub] spark issue #20240: [SPARK-23049][SQL] `spark.sql.files.ignoreCorruptFiles` ...

2018-01-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20240 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20240: [SPARK-23049][SQL] `spark.sql.files.ignoreCorruptFiles` ...

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20240 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpark

2018-01-11 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20217 Also, I am fine with having a discussion later within 2.3.0 timeline and doing that in a followup if needed. --- - To

[GitHub] spark issue #20240: [SPARK-23049][SQL] `spark.sql.files.ignoreCorruptFiles` ...

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20240 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85999/ Test FAILed. ---

[GitHub] spark issue #20240: [SPARK-23049][SQL] `spark.sql.files.ignoreCorruptFiles` ...

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20240 **[Test build #85999 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85999/testReport)** for PR 20240 at commit

[GitHub] spark issue #19001: [SPARK-19256][SQL] Hive bucketing support

2018-01-11 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/19001 Jenkins retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpark

2018-01-11 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20217 Just wanted to make sure and check other possibilities (which is usually not quite good to check so late like this ... ). I am not against this implementation and fine as is too. ---

[GitHub] spark issue #20214: [SPARK-23023][SQL] Cast field data to strings in showStr...

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20214 **[Test build #86012 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86012/testReport)** for PR 20214 at commit

[GitHub] spark pull request #20194: [SPARK-22999][SQL]'show databases like command' c...

2018-01-11 Thread guoxiaolongzte
Github user guoxiaolongzte commented on a diff in the pull request: https://github.com/apache/spark/pull/20194#discussion_r161130352 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -141,7 +141,7 @@ statement (LIKE?

[GitHub] spark issue #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpark

2018-01-11 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20217 Yup, I am looking at it. `def register(name: String, udf: UserDefinedFunction)` and `def register[RT: TypeTag](name: String, func: Function0[RT])`, etc. The usual way to resemble

[GitHub] spark pull request #20214: [SPARK-23023][SQL] Cast field data to strings in ...

2018-01-11 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20214#discussion_r161130049 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -237,13 +237,19 @@ class Dataset[T] private[sql]( private[sql] def

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20204 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20204 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85994/ Test FAILed. ---

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20204 **[Test build #85994 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85994/testReport)** for PR 20204 at commit

[GitHub] spark pull request #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpar...

2018-01-11 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/20217#discussion_r161129504 --- Diff: python/pyspark/sql/context.py --- @@ -203,18 +203,46 @@ def registerFunction(self, name, f, returnType=StringType()): >>> _ =

[GitHub] spark issue #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpark

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20217 **[Test build #86011 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86011/testReport)** for PR 20217 at commit

[GitHub] spark issue #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpark

2018-01-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20217 `registerUDF` is more consistent with the Scala APIs defined in `UDFRegistration`. We do not need to provide `returnType`. ---

[GitHub] spark issue #20240: [SPARK-23049][SQL] `spark.sql.files.ignoreCorruptFiles` ...

2018-01-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20240 Hi, @gatorsmile and @cloud-fan . Could you review this, too? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpar...

2018-01-11 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20217#discussion_r161126125 --- Diff: python/pyspark/sql/context.py --- @@ -203,18 +203,48 @@ def registerFunction(self, name, f, returnType=StringType()): >>> _ =

[GitHub] spark issue #20183: [SPARK-22986][Core] Use a cache to avoid instantiating m...

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20183 **[Test build #86010 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86010/testReport)** for PR 20183 at commit

[GitHub] spark issue #20183: [SPARK-22986][Core] Use a cache to avoid instantiating m...

2018-01-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20183 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20183: [SPARK-22986][Core] Use a cache to avoid instanti...

2018-01-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20183#discussion_r161127798 --- Diff: core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala --- @@ -206,37 +206,51 @@ private[spark] class TorrentBroadcast[T:

[GitHub] spark pull request #20183: [SPARK-22986][Core] Use a cache to avoid instanti...

2018-01-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20183#discussion_r161127725 --- Diff: core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala --- @@ -206,37 +206,51 @@ private[spark] class TorrentBroadcast[T:

[GitHub] spark pull request #20183: [SPARK-22986][Core] Use a cache to avoid instanti...

2018-01-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20183#discussion_r161127616 --- Diff: core/src/main/scala/org/apache/spark/broadcast/BroadcastManager.scala --- @@ -52,6 +54,10 @@ private[spark] class BroadcastManager(

[GitHub] spark issue #19892: [SPARK-22797][PySpark] Bucketizer support multi-column

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19892 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19892: [SPARK-22797][PySpark] Bucketizer support multi-column

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19892 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86009/ Test PASSed. ---

[GitHub] spark pull request #20226: [SPARK-23034][SQL][UI] Display tablename for `Hiv...

2018-01-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20226#discussion_r161126763 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveTableScanExec.scala --- @@ -62,6 +62,8 @@ case class HiveTableScanExec(

[GitHub] spark issue #19892: [SPARK-22797][PySpark] Bucketizer support multi-column

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19892 **[Test build #86009 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86009/testReport)** for PR 19892 at commit

[GitHub] spark issue #20056: [SPARK-22878] [CORE] Count totalDroppedEvents for LiveLi...

2018-01-11 Thread Ngone51
Github user Ngone51 commented on the issue: https://github.com/apache/spark/pull/20056 @squito by now, the variable `droppedEventsCounter ` is unused. So, I want to reuse it. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20121: [SPARK-22882][ML][TESTS] ML test for structured streamin...

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20121 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20121: [SPARK-22882][ML][TESTS] ML test for structured streamin...

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20121 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86005/ Test PASSed. ---

[GitHub] spark pull request #20226: [SPARK-23034][SQL][UI] Display tablename for `Hiv...

2018-01-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20226#discussion_r161126468 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveTableScanExec.scala --- @@ -62,6 +62,8 @@ case class HiveTableScanExec(

[GitHub] spark issue #20121: [SPARK-22882][ML][TESTS] ML test for structured streamin...

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20121 **[Test build #86005 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86005/testReport)** for PR 20121 at commit

[GitHub] spark issue #20240: [SPARK-23049][SQL] `spark.sql.files.ignoreCorruptFiles` ...

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20240 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20240: [SPARK-23049][SQL] `spark.sql.files.ignoreCorruptFiles` ...

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20240 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85995/ Test PASSed. ---

[GitHub] spark issue #20240: [SPARK-23049][SQL] `spark.sql.files.ignoreCorruptFiles` ...

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20240 **[Test build #85995 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85995/testReport)** for PR 20240 at commit

[GitHub] spark issue #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpark

2018-01-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20217 Any more comment? @icexelloss @HyukjinKwon @ueshin @viirya --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20214: [SPARK-23023][SQL] Cast field data to strings in ...

2018-01-11 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20214#discussion_r161123911 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -237,13 +237,19 @@ class Dataset[T] private[sql]( private[sql] def

[GitHub] spark pull request #20214: [SPARK-23023][SQL] Cast field data to strings in ...

2018-01-11 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20214#discussion_r161123821 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -237,13 +237,19 @@ class Dataset[T] private[sql]( private[sql] def

[GitHub] spark pull request #20214: [SPARK-23023][SQL] Cast field data to strings in ...

2018-01-11 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20214#discussion_r161123864 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -237,13 +237,19 @@ class Dataset[T] private[sql]( private[sql] def

[GitHub] spark pull request #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpar...

2018-01-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20217#discussion_r161124529 --- Diff: python/pyspark/sql/context.py --- @@ -203,18 +203,46 @@ def registerFunction(self, name, f, returnType=StringType()): >>> _ =

[GitHub] spark issue #20216: [SPARK-23024][WEB-UI]Spark ui about the contents of the ...

2018-01-11 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/20216 ![3](https://user-images.githubusercontent.com/26266482/34856154-87b381b6-f77e-11e7-932e-bb14415dc56a.png) --- - To

[GitHub] spark issue #20222: [SPARK-23028] Bump master branch version to 2.4.0-SNAPSH...

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20222 **[Test build #86008 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86008/testReport)** for PR 20222 at commit

[GitHub] spark issue #19892: [SPARK-22797][PySpark] Bucketizer support multi-column

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19892 **[Test build #86009 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86009/testReport)** for PR 19892 at commit

[GitHub] spark issue #20231: [SPARK-23000][TEST-HADOOP2.6] Fix Flaky test suite DataS...

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20231 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20231: [SPARK-23000][TEST-HADOOP2.6] Fix Flaky test suite DataS...

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20231 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85992/ Test PASSed. ---

[GitHub] spark issue #20231: [SPARK-23000][TEST-HADOOP2.6] Fix Flaky test suite DataS...

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20231 **[Test build #85992 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85992/testReport)** for PR 20231 at commit

[GitHub] spark issue #20225: [SPARK-23033] Don't use task level retry for continuous ...

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20225 **[Test build #86007 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86007/testReport)** for PR 20225 at commit

[GitHub] spark issue #20225: [SPARK-23033] Don't use task level retry for continuous ...

2018-01-11 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20225 The problem in the test runs above was also a test issue. The new code changed the synchronization such that the row wasn't written when the test expected; I verified manually that the failing

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20203 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85991/ Test PASSed. ---

[GitHub] spark issue #20241: [SPARK-23008][ML][FOLLOW-UP] mark OneHotEncoder python A...

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20241 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20241: [SPARK-23008][ML][FOLLOW-UP] mark OneHotEncoder python A...

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20241 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86006/ Test PASSed. ---

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20203 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20241: [SPARK-23008][ML][FOLLOW-UP] mark OneHotEncoder python A...

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20241 **[Test build #86006 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86006/testReport)** for PR 20241 at commit

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20203 **[Test build #85991 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85991/testReport)** for PR 20203 at commit

[GitHub] spark pull request #20229: [SPARK-23045][ML][SparkR] Update RFormula to use ...

2018-01-11 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/20229#discussion_r161120354 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/RFormula.scala --- @@ -230,16 +231,17 @@ class RFormula @Since("1.5.0") (@Since("1.5.0")

[GitHub] spark pull request #20211: [SPARK-23011][PYTHON][SQL] Prepend missing groupi...

2018-01-11 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20211#discussion_r161119843 --- Diff: python/pyspark/sql/group.py --- @@ -233,6 +233,27 @@ def apply(self, udf): | 2| 1.1094003924504583|

[GitHub] spark issue #20241: [SPARK-23008][ML][FOLLOW-UP] mark OneHotEncoder python A...

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20241 **[Test build #86004 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86004/testReport)** for PR 20241 at commit

[GitHub] spark issue #20241: [SPARK-23008][ML][FOLLOW-UP] mark OneHotEncoder python A...

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20241 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86004/ Test PASSed. ---

[GitHub] spark issue #20241: [SPARK-23008][ML][FOLLOW-UP] mark OneHotEncoder python A...

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20241 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20224: [SPARK-23032][SQL] Add a per-query codegenStageId to Who...

2018-01-11 Thread rednaxelafx
Github user rednaxelafx commented on the issue: https://github.com/apache/spark/pull/20224 Thanks for your comments, @viirya ! I'd say only having (1) and (2) makes it much less useful than having all 3, but it's still useful in its own for helping people understand exactly

[GitHub] spark issue #20241: [SPARK-23008][ML][FOLLOW-UP] mark OneHotEncoder python A...

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20241 **[Test build #86006 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86006/testReport)** for PR 20241 at commit

[GitHub] spark issue #20225: [SPARK-23033] Don't use task level retry for continuous ...

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20225 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20225: [SPARK-23033] Don't use task level retry for continuous ...

2018-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20225 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85993/ Test FAILed. ---

[GitHub] spark issue #20225: [SPARK-23033] Don't use task level retry for continuous ...

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20225 **[Test build #85993 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85993/testReport)** for PR 20225 at commit

[GitHub] spark issue #20121: [SPARK-22882][ML][TESTS] ML test for structured streamin...

2018-01-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20121 **[Test build #86005 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86005/testReport)** for PR 20121 at commit

[GitHub] spark pull request #20241: [SPARK-23008][ML][FOLLOW-UP] mark OneHotEncoder p...

2018-01-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20241#discussion_r161116898 --- Diff: python/pyspark/ml/feature.py --- @@ -1577,6 +1577,8 @@ class OneHotEncoder(JavaTransformer, HasInputCol, HasOutputCol, JavaMLReadable,

<    1   2   3   4   5   6   >