[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117374353 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1229,25 +1228,56 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark issue #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18023 **[Test build #77068 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77068/testReport)** for PR 18023 at commit [`7699e87`](https://github.com/apache/spark/commit/76

[GitHub] spark issue #13959: [SPARK-14351] [MLlib] [ML] Optimize findBestSplits metho...

2017-05-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/13959 > The reason for closing PRs but _not_ their corresponding JIRA would be that the PR is either poorly implemented or the author is non-responsive. Yes, I tried to identify this case.

[GitHub] spark issue #13959: [SPARK-14351] [MLlib] [ML] Optimize findBestSplits metho...

2017-05-18 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/13959 Yes, this is a tough issue. Let's wait and see if @jkbradley has thoughts on this issue. If we don't hear anything, then I'd leave it up to @MechCoder on whether to reopen. Thanks, btw, for taking th

[GitHub] spark issue #13959: [SPARK-14351] [MLlib] [ML] Optimize findBestSplits metho...

2017-05-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/13959 @MechCoder, I apologize that, probably, it sounds the reason for my suggestion was not clear initially and if it looks without a respect. --- If your project is set up for it, you can reply to

[GitHub] spark issue #18031: Record accurate size of blocks in MapStatus when it's ab...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18031 **[Test build #77069 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77069/testReport)** for PR 18031 at commit [`f6670d8`](https://github.com/apache/spark/commit/f6

[GitHub] spark pull request #18019: [SPARK-20748][SQL] Add built-in SQL function CH[A...

2017-05-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18019#discussion_r117378749 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -2138,6 +2138,40 @@ object functions { def ascii(e: Column): Column =

[GitHub] spark pull request #18019: [SPARK-20748][SQL] Add built-in SQL function CH[A...

2017-05-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18019#discussion_r117250411 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -1268,6 +1268,59 @@ case class Ascii(chil

[GitHub] spark pull request #18019: [SPARK-20748][SQL] Add built-in SQL function CH[A...

2017-05-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18019#discussion_r117376928 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -1268,6 +1268,59 @@ case class Ascii(chil

[GitHub] spark pull request #18019: [SPARK-20748][SQL] Add built-in SQL function CH[A...

2017-05-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18019#discussion_r117375494 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -1268,6 +1268,59 @@ case class Ascii(chil

[GitHub] spark pull request #18019: [SPARK-20748][SQL] Add built-in SQL function CH[A...

2017-05-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18019#discussion_r117376367 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -1268,6 +1268,59 @@ case class Ascii(chil

[GitHub] spark pull request #18019: [SPARK-20748][SQL] Add built-in SQL function CH[A...

2017-05-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18019#discussion_r117375988 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -1268,6 +1268,59 @@ case class Ascii(chil

[GitHub] spark pull request #18019: [SPARK-20748][SQL] Add built-in SQL function CH[A...

2017-05-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18019#discussion_r117378356 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -1268,6 +1268,59 @@ case class Ascii(chil

[GitHub] spark pull request #18019: [SPARK-20748][SQL] Add built-in SQL function CH[A...

2017-05-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18019#discussion_r117378811 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/StringFunctionsSuite.scala --- @@ -117,6 +117,41 @@ class StringFunctionsSuite extends QueryTest w

[GitHub] spark pull request #18019: [SPARK-20748][SQL] Add built-in SQL function CH[A...

2017-05-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18019#discussion_r117289000 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -1268,6 +1268,59 @@ case class Ascii(chil

[GitHub] spark issue #18031: Record accurate size of blocks in MapStatus when it's ab...

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18031 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77069/ Test FAILed. ---

[GitHub] spark issue #18031: Record accurate size of blocks in MapStatus when it's ab...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18031 **[Test build #77069 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77069/testReport)** for PR 18031 at commit [`f6670d8`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #18031: Record accurate size of blocks in MapStatus when it's ab...

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18031 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18031: Record accurate size of blocks in MapStatus when it's ab...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18031 **[Test build #77070 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77070/testReport)** for PR 18031 at commit [`970421b`](https://github.com/apache/spark/commit/97

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117368022 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1230,24 +1230,49 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117380055 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1230,24 +1230,49 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117367722 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1230,24 +1230,49 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117366828 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala --- @@ -84,6 +84,33 @@ case class UnresolvedTableValuedFun

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117367155 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala --- @@ -84,6 +84,33 @@ case class UnresolvedTableValuedFun

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117380037 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1230,24 +1230,49 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117367232 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala --- @@ -84,6 +84,33 @@ case class UnresolvedTableValuedFun

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117379878 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1230,24 +1230,49 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark issue #14971: [SPARK-17410] [SPARK-17284] Move Hive-generated Stats In...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14971 **[Test build #77066 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77066/testReport)** for PR 14971 at commit [`aa9a36e`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #18031: Record accurate size of blocks in MapStatus when it's ab...

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18031 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14971: [SPARK-17410] [SPARK-17284] Move Hive-generated Stats In...

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14971 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18031: Record accurate size of blocks in MapStatus when it's ab...

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18031 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77070/ Test FAILed. ---

[GitHub] spark issue #14971: [SPARK-17410] [SPARK-17284] Move Hive-generated Stats In...

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14971 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77066/ Test FAILed. ---

[GitHub] spark issue #18031: Record accurate size of blocks in MapStatus when it's ab...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18031 **[Test build #77070 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77070/testReport)** for PR 18031 at commit [`970421b`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #14971: [SPARK-17410] [SPARK-17284] Move Hive-generated Stats In...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14971 **[Test build #77071 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77071/testReport)** for PR 14971 at commit [`1e4182d`](https://github.com/apache/spark/commit/1e

[GitHub] spark pull request #18031: Record accurate size of blocks in MapStatus when ...

2017-05-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18031#discussion_r117385321 --- Diff: core/src/main/scala/org/apache/spark/scheduler/MapStatus.scala --- @@ -121,48 +126,69 @@ private[spark] class CompressedMapStatus( }

[GitHub] spark issue #18031: Record accurate size of blocks in MapStatus when it's ab...

2017-05-18 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/18031 @HyukjinKwon Thank you so much ! Really helpful 👍 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #18031: Record accurate size of blocks in MapStatus when it's ab...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18031 **[Test build #77072 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77072/testReport)** for PR 18031 at commit [`bfea9f5`](https://github.com/apache/spark/commit/bf

[GitHub] spark issue #17992: [SPARK-20759] SCALA_VERSION in _config.yml should be con...

2017-05-18 Thread liu-zhaokun
Github user liu-zhaokun commented on the issue: https://github.com/apache/spark/pull/17992 @srowen Hello,do you know how to finish the test? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark issue #17985: Add "full_outer" name to join types

2017-05-18 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/17985 @BartekH Yes, I think we can add that to exception message. Please also add a test case for checking supported join types. --- If your project is set up for it, you can reply to this email and have y

[GitHub] spark issue #17999: [SPARK-20751][SQL] Add built-in SQL Function - COT

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17999 **[Test build #77067 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77067/testReport)** for PR 17999 at commit [`ea10dee`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #17999: [SPARK-20751][SQL] Add built-in SQL Function - COT

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17999 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77067/ Test PASSed. ---

[GitHub] spark pull request #17955: [SPARK-20715] Store MapStatuses only in MapOutput...

2017-05-18 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/17955#discussion_r117385673 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1233,17 +1223,6 @@ class DAGScheduler( logInfo("w

[GitHub] spark issue #17999: [SPARK-20751][SQL] Add built-in SQL Function - COT

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17999 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #17955: [SPARK-20715] Store MapStatuses only in MapOutput...

2017-05-18 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/17955#discussion_r117385925 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ShuffleMapStage.scala --- @@ -42,13 +41,12 @@ private[spark] class ShuffleMapStage( pa

[GitHub] spark issue #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18023 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77068/ Test PASSed. ---

[GitHub] spark issue #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18023 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18023 **[Test build #77068 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77068/testReport)** for PR 18023 at commit [`7699e87`](https://github.com/apache/spark/commit/7

[GitHub] spark pull request #17955: [SPARK-20715] Store MapStatuses only in MapOutput...

2017-05-18 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/17955#discussion_r117388447 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1233,17 +1223,6 @@ class DAGScheduler( logInfo("wai

[GitHub] spark pull request #17955: [SPARK-20715] Store MapStatuses only in MapOutput...

2017-05-18 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/17955#discussion_r117388593 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ShuffleMapStage.scala --- @@ -42,13 +41,12 @@ private[spark] class ShuffleMapStage( pare

[GitHub] spark issue #18016: [SPARK-20786][SQL]Improve ceil and floor handle the valu...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18016 **[Test build #77073 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77073/testReport)** for PR 18016 at commit [`68ecf5e`](https://github.com/apache/spark/commit/68

[GitHub] spark issue #18016: [SPARK-20786][SQL]Improve ceil and floor handle the valu...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18016 **[Test build #77074 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77074/testReport)** for PR 18016 at commit [`1f771bd`](https://github.com/apache/spark/commit/1f

[GitHub] spark issue #18029: [SPARK-20168][WIP][DStream] Add changes to use kinesis f...

2017-05-18 Thread yssharma
Github user yssharma commented on the issue: https://github.com/apache/spark/pull/18029 @budde @brkyvz would love to hear your thoughts if this is the best way to add this functionality --- If your project is set up for it, you can reply to this email and have your reply appear on G

[GitHub] spark issue #18029: [SPARK-20168][WIP][DStream] Add changes to use kinesis f...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18029 **[Test build #77075 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77075/testReport)** for PR 18029 at commit [`75d8523`](https://github.com/apache/spark/commit/75

[GitHub] spark issue #18029: [SPARK-20168][WIP][DStream] Add changes to use kinesis f...

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18029 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18029: [SPARK-20168][WIP][DStream] Add changes to use kinesis f...

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18029 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77075/ Test FAILed. ---

[GitHub] spark issue #18029: [SPARK-20168][WIP][DStream] Add changes to use kinesis f...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18029 **[Test build #77075 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77075/testReport)** for PR 18029 at commit [`75d8523`](https://github.com/apache/spark/commit/7

[GitHub] spark pull request #14971: [SPARK-17410] [SPARK-17284] Move Hive-generated S...

2017-05-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14971#discussion_r117392683 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala --- @@ -414,6 +415,50 @@ private[hive] class HiveClientImpl(

[GitHub] spark pull request #14971: [SPARK-17410] [SPARK-17284] Move Hive-generated S...

2017-05-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14971#discussion_r117393090 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala --- @@ -215,6 +218,215 @@ class StatisticsSuite extends StatisticsColle

[GitHub] spark pull request #17936: [SPARK-20638][Core]Optimize the CartesianRDD to r...

2017-05-18 Thread ConeyLiu
Github user ConeyLiu commented on a diff in the pull request: https://github.com/apache/spark/pull/17936#discussion_r117393634 --- Diff: core/src/test/scala/org/apache/spark/metrics/InputOutputMetricsSuite.scala --- @@ -198,8 +198,12 @@ class InputOutputMetricsSuite extends SparkFu

[GitHub] spark pull request #17936: [SPARK-20638][Core]Optimize the CartesianRDD to r...

2017-05-18 Thread ConeyLiu
Github user ConeyLiu commented on a diff in the pull request: https://github.com/apache/spark/pull/17936#discussion_r117393923 --- Diff: core/src/test/scala/org/apache/spark/metrics/InputOutputMetricsSuite.scala --- @@ -198,8 +198,12 @@ class InputOutputMetricsSuite extends SparkFu

[GitHub] spark issue #18011: [SPARK-19089][SQL] Add support for nested sequences

2017-05-18 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18011 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #14971: [SPARK-17410] [SPARK-17284] Move Hive-generated Stats In...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14971 **[Test build #77071 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77071/testReport)** for PR 14971 at commit [`1e4182d`](https://github.com/apache/spark/commit/1

[GitHub] spark pull request #16986: [SPARK-18891][SQL] Support for Map collection typ...

2017-05-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16986#discussion_r117394141 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -329,35 +329,19 @@ object ScalaReflection extends ScalaR

[GitHub] spark issue #14971: [SPARK-17410] [SPARK-17284] Move Hive-generated Stats In...

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14971 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14971: [SPARK-17410] [SPARK-17284] Move Hive-generated Stats In...

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14971 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77071/ Test PASSed. ---

[GitHub] spark issue #18011: [SPARK-19089][SQL] Add support for nested sequences

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18011 **[Test build #77076 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77076/testReport)** for PR 18011 at commit [`dd3bf01`](https://github.com/apache/spark/commit/dd

[GitHub] spark issue #17936: [SPARK-20638][Core]Optimize the CartesianRDD to reduce r...

2017-05-18 Thread ConeyLiu
Github user ConeyLiu commented on the issue: https://github.com/apache/spark/pull/17936 @srowen Sorry for the late reply. I updated the code. Because we should reduce times of the remotely fetch, the second partition should be cached in locally. There are two ways, first cached by th

[GitHub] spark issue #18015: [SAPRK-20785][WEB-UI][SQL]Spark should provide jump link...

2017-05-18 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/18015 Thank you, I will work better for Spark web ui. jenkins to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[GitHub] spark issue #18016: [SPARK-20786][SQL]Improve ceil and floor handle the valu...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18016 **[Test build #77073 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77073/testReport)** for PR 18016 at commit [`68ecf5e`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #18016: [SPARK-20786][SQL]Improve ceil and floor handle the valu...

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18016 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18016: [SPARK-20786][SQL]Improve ceil and floor handle the valu...

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18016 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77073/ Test FAILed. ---

[GitHub] spark pull request #18016: [SPARK-20786][SQL]Improve ceil and floor handle t...

2017-05-18 Thread heary-cao
Github user heary-cao commented on a diff in the pull request: https://github.com/apache/spark/pull/18016#discussion_r117397093 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/MathFunctionsSuite.scala --- @@ -173,6 +173,14 @@ class MathFunctionsSuite extends QueryTest with

[GitHub] spark issue #18016: [SPARK-20786][SQL]Improve ceil and floor handle the valu...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18016 **[Test build #77077 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77077/testReport)** for PR 18016 at commit [`8b346e6`](https://github.com/apache/spark/commit/8b

[GitHub] spark issue #18016: [SPARK-20786][SQL]Improve ceil and floor handle the valu...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18016 **[Test build #77074 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77074/testReport)** for PR 18016 at commit [`1f771bd`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #18016: [SPARK-20786][SQL]Improve ceil and floor handle the valu...

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18016 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18016: [SPARK-20786][SQL]Improve ceil and floor handle the valu...

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18016 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77074/ Test FAILed. ---

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117397712 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala --- @@ -84,6 +84,33 @@ case class UnresolvedTableValuedFun

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117398110 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala --- @@ -84,6 +84,33 @@ case class UnresolvedTableValuedFun

[GitHub] spark issue #18016: [SPARK-20786][SQL]Improve ceil and floor handle the valu...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18016 **[Test build #77078 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77078/testReport)** for PR 18016 at commit [`6d51c07`](https://github.com/apache/spark/commit/6d

[GitHub] spark issue #16677: [SPARK-19355][SQL] Use map output statistices to improve...

2017-05-18 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16677 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, o

[GitHub] spark issue #16677: [SPARK-19355][SQL] Use map output statistices to improve...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16677 **[Test build #77079 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77079/testReport)** for PR 16677 at commit [`55ee6b0`](https://github.com/apache/spark/commit/55

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117398452 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala --- @@ -84,6 +84,33 @@ case class UnresolvedTableValuedFun

[GitHub] spark issue #18031: [SPARK-20801] Record accurate size of blocks in MapStatu...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18031 **[Test build #77072 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77072/testReport)** for PR 18031 at commit [`bfea9f5`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #18031: [SPARK-20801] Record accurate size of blocks in MapStatu...

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18031 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77072/ Test PASSed. ---

[GitHub] spark issue #18031: [SPARK-20801] Record accurate size of blocks in MapStatu...

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18031 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117399399 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1230,24 +1230,49 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117399718 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1230,24 +1230,49 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117399811 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1230,24 +1230,49 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark issue #18025: [WIP][SparkR] Update doc and examples for sql functions

2017-05-18 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/18025 @felixcheung Thanks for your feedback. - This does not affect discoverability: the name of the method is still on the index list - No problem with help either, e.g., one can use `?avg`

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117399877 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1230,24 +1230,49 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117399885 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1230,24 +1230,49 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark issue #18029: [SPARK-20168][WIP][DStream] Add changes to use kinesis f...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18029 **[Test build #77080 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77080/testReport)** for PR 18029 at commit [`9944da8`](https://github.com/apache/spark/commit/99

[GitHub] spark issue #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18023 **[Test build #77081 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77081/testReport)** for PR 18023 at commit [`6e37517`](https://github.com/apache/spark/commit/6e

[GitHub] spark issue #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18023 Like what we did for `*` in `Column.scala`, we also need to handle the Dataset APIs. You can follow the way we handle star there. ```Scala df.select(df("(a|b)?+.+")) ``` --- If your

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117402025 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -795,6 +795,12 @@ object SQLConf { .intConf

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117402094 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -795,6 +795,12 @@ object SQLConf { .intConf

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117402461 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -795,6 +795,12 @@ object SQLConf { .intConf

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117402527 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala --- @@ -177,6 +177,18 @@ object ParserUtils { sb.t

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117403303 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -795,6 +795,12 @@ object SQLConf { .intConf

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117403331 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -795,6 +795,12 @@ object SQLConf { .intConf

<    1   2   3   4   5   >