[GitHub] spark issue #18652: [SPARK-21497][SQL] Pull non-deterministic equi join keys...

2017-08-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18652 The order is different from the original one that is evaluated in the join conditions. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request #18581: [SPARK-21289][SQL][ML] Supports custom line separ...

2017-08-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18581#discussion_r134932083 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFileLinesReader.scala --- @@ -32,7 +32,9 @@ import

[GitHub] spark pull request #18962: [SPARK-21714][CORE][YARN] Avoiding re-uploading r...

2017-08-23 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/18962#discussion_r134932043 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -330,19 +332,21 @@ object SparkSubmit extends CommandLineUtils {

[GitHub] spark pull request #19017: [SPARK-21804][SQL] json_tuple returns null values...

2017-08-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19017#discussion_r134931404 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -447,7 +448,18 @@ case class

[GitHub] spark pull request #19017: [SPARK-21804][SQL] json_tuple returns null values...

2017-08-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19017#discussion_r134931120 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -447,7 +448,18 @@ case class

[GitHub] spark issue #18962: [SPARK-21714][CORE][YARN] Avoiding re-uploading remote r...

2017-08-23 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/18962 Yes @tgravescs if we download everything to local and then upload to yarn, http/https/ftp should be unrelated here. But still in yarn cluster mode, if we specify remote http jars, then yarn

[GitHub] spark issue #19018: [SPARK-21801][SPARKR][TEST] unit test randomly fail with...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19018 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81070/ Test PASSed. ---

[GitHub] spark issue #19018: [SPARK-21801][SPARKR][TEST] unit test randomly fail with...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19018 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19018: [SPARK-21801][SPARKR][TEST] unit test randomly fail with...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19018 **[Test build #81070 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81070/testReport)** for PR 19018 at commit

[GitHub] spark issue #18730: [SPARK-21527][CORE] Use buffer limit in order to use JAV...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18730 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19028: [MINOR][SQL] The comment of Class ExchangeCoordinator ex...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19028 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19028: [MINOR][SQL] The comment of Class ExchangeCoordinator ex...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19028 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81063/ Test PASSed. ---

[GitHub] spark issue #18730: [SPARK-21527][CORE] Use buffer limit in order to use JAV...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18730 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81064/ Test FAILed. ---

[GitHub] spark issue #19028: [MINOR][SQL] The comment of Class ExchangeCoordinator ex...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19028 **[Test build #81063 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81063/testReport)** for PR 19028 at commit

[GitHub] spark issue #18730: [SPARK-21527][CORE] Use buffer limit in order to use JAV...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18730 **[Test build #81064 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81064/testReport)** for PR 18730 at commit

[GitHub] spark issue #18964: [SPARK-21701][CORE] Enable RPC client to use ` SO_RCVBUF...

2017-08-23 Thread neoremind
Github user neoremind commented on the issue: https://github.com/apache/spark/pull/18964 @zsxwing I did try to create a performance test against spark rpc, the test result can be found [here](https://github.com/neoremind/kraps-rpc#4-performance-test), note that I created the project

[GitHub] spark issue #18730: [SPARK-21527][CORE] Use buffer limit in order to use JAV...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18730 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18730: [SPARK-21527][CORE] Use buffer limit in order to use JAV...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18730 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81061/ Test PASSed. ---

[GitHub] spark issue #18730: [SPARK-21527][CORE] Use buffer limit in order to use JAV...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18730 **[Test build #81061 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81061/testReport)** for PR 18730 at commit

[GitHub] spark issue #19027: [SPARK-19165][PYTHON][SQL] PySpark APIs using columns as...

2017-08-23 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/19027 LGTM. Btw, I'm just curious why we need tests with `numpy` here. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #19027: [SPARK-19165][PYTHON][SQL] PySpark APIs using columns as...

2017-08-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19027 Will probably take a look through the problem in the near future including hard dependencies and etc. I took a quick look but I think I need more time but yes it looks appearently vaild point.

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18581 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81060/ Test PASSed. ---

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18581 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19031: [SPARK-21603][SQL][FOLLOW-UP] Use -1 to disable maxLines...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19031 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19031: [SPARK-21603][SQL][FOLLOW-UP] Use -1 to disable maxLines...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19031 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81062/ Test PASSed. ---

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18581 **[Test build #81060 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81060/testReport)** for PR 18581 at commit

[GitHub] spark issue #19031: [SPARK-21603][SQL][FOLLOW-UP] Use -1 to disable maxLines...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19031 **[Test build #81062 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81062/testReport)** for PR 19031 at commit

[GitHub] spark issue #19018: [SPARK-21801][SPARKR][TEST] unit test randomly fail with...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19018 **[Test build #81070 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81070/testReport)** for PR 19018 at commit

[GitHub] spark issue #19018: [SPARK-21801][SPARKR][TEST] unit test randomly fail with...

2017-08-23 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19018 jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18581 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81059/ Test PASSed. ---

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18581 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #19017: [SPARK-21804][SQL] json_tuple returns null values...

2017-08-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19017#discussion_r134925698 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -447,7 +448,18 @@ case class

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18581 **[Test build #81059 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81059/testReport)** for PR 18581 at commit

[GitHub] spark pull request #19017: [SPARK-21804][SQL] json_tuple returns null values...

2017-08-23 Thread jmchung
Github user jmchung commented on a diff in the pull request: https://github.com/apache/spark/pull/19017#discussion_r134925669 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -447,7 +448,18 @@ case class

[GitHub] spark issue #19018: [SPARK-21801][SPARKR][TEST] unit test randomly fail with...

2017-08-23 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19018 but i think in general it's better to make tests more predictable like this --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #19018: [SPARK-21801][SPARKR][TEST] unit test randomly fail with...

2017-08-23 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19018 the error specifically is: `The input column stridx_87ea3065aeb2 should have at least two distinct values.` I don't think this would be only happening in R - I suppose whenever the

[GitHub] spark pull request #19016: [SPARK-21805][SPARKR] Disable R vignettes code on...

2017-08-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19016 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #18945: Add option to convert nullable int columns to flo...

2017-08-23 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18945#discussion_r134925269 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1762,7 +1762,7 @@ def toPandas(self): else: --- End diff -- If we use this

[GitHub] spark issue #19016: [SPARK-21805][SPARKR] Disable R vignettes code on Window...

2017-08-23 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19016 thanks, merged to master/2.2. will check for nightly build from tonight. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request #19017: [SPARK-21804][SQL] json_tuple returns null values...

2017-08-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19017#discussion_r134923877 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -447,7 +448,18 @@ case class

[GitHub] spark pull request #19017: [SPARK-21804][SQL] json_tuple returns null values...

2017-08-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19017#discussion_r134923403 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -447,7 +448,18 @@ case class

[GitHub] spark pull request #19017: [SPARK-21804][SQL] json_tuple returns null values...

2017-08-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19017#discussion_r134923130 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -447,7 +448,18 @@ case class

[GitHub] spark issue #19032: [SPARK-17321][YARN] Avoid writing shuffle metadata to di...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19032 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19032: [SPARK-17321][YARN] Avoid writing shuffle metadata to di...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19032 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81067/ Test PASSed. ---

[GitHub] spark issue #19032: [SPARK-17321][YARN] Avoid writing shuffle metadata to di...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19032 **[Test build #81067 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81067/testReport)** for PR 19032 at commit

[GitHub] spark issue #18730: [SPARK-21527][CORE] Use buffer limit in order to use JAV...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18730 **[Test build #81069 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81069/testReport)** for PR 18730 at commit

[GitHub] spark issue #19027: [SPARK-19165][PYTHON][SQL] PySpark APIs using columns as...

2017-08-23 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19027 I'm ok without the test since this is unlikely to break in the future. We do have tests that depends on (optionally) numpy (and Arrow) - seems like we should be able to take on dependencies

[GitHub] spark issue #18730: [SPARK-21527][CORE] Use buffer limit in order to use JAV...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18730 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81068/ Test FAILed. ---

[GitHub] spark issue #18730: [SPARK-21527][CORE] Use buffer limit in order to use JAV...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18730 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18730: [SPARK-21527][CORE] Use buffer limit in order to use JAV...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18730 **[Test build #81068 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81068/testReport)** for PR 18730 at commit

[GitHub] spark issue #18730: [SPARK-21527][CORE] Use buffer limit in order to use JAV...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18730 **[Test build #81068 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81068/testReport)** for PR 18730 at commit

[GitHub] spark issue #19032: [SPARK-17321][YARN] Avoid writing shuffle metadata to di...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19032 **[Test build #81067 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81067/testReport)** for PR 19032 at commit

[GitHub] spark issue #19032: [SPARK-17321][YARN] Avoid writing shuffle metadata to di...

2017-08-23 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19032 CC @lishuming please take a look at another approach to fix the bad disk issue. Also ping @tgravescs to view the PR. Thanks a lot. --- If your project is set up for it, you can

[GitHub] spark issue #19031: [SPARK-21603][SQL][FOLLOW-UP] Use -1 to disable maxLines...

2017-08-23 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19031 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #18730: [SPARK-21527][CORE] Use buffer limit in order to use JAV...

2017-08-23 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/18730 I mock some local test for two different api. From the simple test result, we can see slice will not affect the performance of write bytes.Test result below: ``` 【Test 10 chunks each with

[GitHub] spark pull request #19032: [SPARK-17321][YARN] Avoid writing shuffle metadat...

2017-08-23 Thread jerryshao
GitHub user jerryshao opened a pull request: https://github.com/apache/spark/pull/19032 [SPARK-17321][YARN] Avoid writing shuffle metadata to disk if NM recovery is disabled ## What changes were proposed in this pull request? In the current code, if NM recovery is not

[GitHub] spark issue #19017: [SPARK-21804][SQL] json_tuple returns null values within...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19017 **[Test build #81066 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81066/testReport)** for PR 19017 at commit

[GitHub] spark issue #19031: [SPARK-21603][SQL][FOLLOW-UP] Use -1 to disable maxLines...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19031 **[Test build #81065 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81065/testReport)** for PR 19031 at commit

[GitHub] spark issue #19017: [SPARK-21804][SQL] json_tuple returns null values within...

2017-08-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19017 LGTM too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #19022: [Spark-21807][SQL]Override ++ operation in Expres...

2017-08-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19022 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #18968: [SPARK-21759][SQL] In.checkInputDataTypes should ...

2017-08-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18968#discussion_r134920136 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala --- @@ -274,17 +274,24 @@ object ScalarSubquery { case

[GitHub] spark issue #19022: [Spark-21807][SQL]Override ++ operation in ExpressionSet...

2017-08-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19022 Thanks! Merging to master. You can fix this in your future PRs. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #19031: [SPARK-21603][SQL][FOLLOW-UP] Use -1 to disable m...

2017-08-23 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19031#discussion_r134920085 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -577,9 +577,11 @@ object SQLConf { .doc("The maximum lines

[GitHub] spark pull request #19031: [SPARK-21603][SQL][FOLLOW-UP] Use -1 to disable m...

2017-08-23 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19031#discussion_r134920074 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -577,9 +577,11 @@ object SQLConf { .doc("The maximum lines

[GitHub] spark pull request #19022: [Spark-21807][SQL]Override ++ operation in Expres...

2017-08-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19022#discussion_r134919964 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionSetSuite.scala --- @@ -210,4 +210,13 @@ class

[GitHub] spark issue #19022: [Spark-21807][SQL]Override ++ operation in ExpressionSet...

2017-08-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19022 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #19017: [SPARK-21804][SQL] json_tuple returns null values within...

2017-08-23 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19017 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #18997: [SPARK-21788][SS]Handle more exceptions when stopping a ...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18997 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18997: [SPARK-21788][SS]Handle more exceptions when stopping a ...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18997 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81058/ Test PASSed. ---

[GitHub] spark issue #18997: [SPARK-21788][SS]Handle more exceptions when stopping a ...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18997 **[Test build #81058 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81058/testReport)** for PR 18997 at commit

[GitHub] spark issue #19017: [SPARK-21804][SQL] json_tuple returns null values within...

2017-08-23 Thread jmchung
Github user jmchung commented on the issue: https://github.com/apache/spark/pull/19017 @viirya PR title fixed, thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #19017: SPARK-21804: json_tuple returns null values within repea...

2017-08-23 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19017 Please edit the PR title as `[SPARK-21804][SQL] json_tuple returns ...`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #19022: [Spark-21807][SQL]Override ++ operation in ExpressionSet...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19022 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19022: [Spark-21807][SQL]Override ++ operation in ExpressionSet...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19022 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81057/ Test PASSed. ---

[GitHub] spark issue #19022: [Spark-21807][SQL]Override ++ operation in ExpressionSet...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19022 **[Test build #81057 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81057/testReport)** for PR 19022 at commit

[GitHub] spark pull request #19031: [SPARK-21603][SQL][FOLLOW-UP] Use -1 to disable m...

2017-08-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19031#discussion_r134918924 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -577,9 +577,11 @@ object SQLConf { .doc("The maximum lines

[GitHub] spark pull request #19031: [SPARK-21603][SQL][FOLLOW-UP] Use -1 to disable m...

2017-08-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19031#discussion_r13491 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -577,9 +577,11 @@ object SQLConf { .doc("The maximum lines

[GitHub] spark issue #19031: [SPARK-21603][SQL][FOLLOW-UP] Use -1 to disable maxLines...

2017-08-23 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19031 I'd prefer using `-1` to disable `maxLinesPerFunction` check like this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #18730: [SPARK-21527][CORE] Use buffer limit in order to use JAV...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18730 **[Test build #81064 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81064/testReport)** for PR 18730 at commit

[GitHub] spark issue #19028: [MINOR][SQL] The comment of Class ExchangeCoordinator ex...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19028 **[Test build #81063 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81063/testReport)** for PR 19028 at commit

[GitHub] spark issue #19028: [MINOR][SQL] The comment of Class ExchangeCoordinator ex...

2017-08-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19028 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #19008: [SPARK-21756][SQL]Add JSON option to allow unquoted cont...

2017-08-23 Thread vinodkc
Github user vinodkc commented on the issue: https://github.com/apache/spark/pull/19008 @rxin , Sure, I'll update it --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #18652: [SPARK-21497][SQL] Pull non-deterministic equi join keys...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18652 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81054/ Test PASSed. ---

[GitHub] spark issue #18652: [SPARK-21497][SQL] Pull non-deterministic equi join keys...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18652 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19013: [SPARK-21728][core] Allow SparkSubmit to use Logging.

2017-08-23 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19013 LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #18652: [SPARK-21497][SQL] Pull non-deterministic equi join keys...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18652 **[Test build #81054 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81054/testReport)** for PR 18652 at commit

[GitHub] spark pull request #18969: [SPARK-21520][SQL][FOLLOW-UP]fix a special case f...

2017-08-23 Thread heary-cao
Github user heary-cao commented on a diff in the pull request: https://github.com/apache/spark/pull/18969#discussion_r134915918 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala --- @@ -24,6 +24,24 @@ import

[GitHub] spark issue #19021: [SPARK-21603][SQL][FOLLOW-UP] Change the default value o...

2017-08-23 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19021 @maropu Should be a good idea. Especially the number of lines of code may not be intuitive to set for this purpose. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #19031: [SPARK-21603][SQL][FOLLOW-UP] Use -1 to disable maxLines...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19031 **[Test build #81062 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81062/testReport)** for PR 19031 at commit

[GitHub] spark issue #19031: [SPARK-21603][SQL][FOLLOW-UP] Use -1 to disable maxLines...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19031 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81055/ Test FAILed. ---

[GitHub] spark issue #19031: [SPARK-21603][SQL][FOLLOW-UP] Use -1 to disable maxLines...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19031 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19031: [SPARK-21603][SQL][FOLLOW-UP] Use -1 to disable maxLines...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19031 **[Test build #81055 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81055/testReport)** for PR 19031 at commit

[GitHub] spark issue #18730: [SPARK-21527][CORE] Use buffer limit in order to use JAV...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18730 **[Test build #81061 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81061/testReport)** for PR 18730 at commit

[GitHub] spark pull request #18730: [SPARK-21527][CORE] Use buffer limit in order to ...

2017-08-23 Thread caneGuy
Github user caneGuy commented on a diff in the pull request: https://github.com/apache/spark/pull/18730#discussion_r134912125 --- Diff: core/src/main/scala/org/apache/spark/util/io/ChunkedByteBuffer.scala --- @@ -63,6 +65,19 @@ private[spark] class ChunkedByteBuffer(var chunks:

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18581 **[Test build #81060 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81060/testReport)** for PR 18581 at commit

[GitHub] spark issue #18730: [SPARK-21527][CORE] Use buffer limit in order to use JAV...

2017-08-23 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/18730 @jiangxb1987 Ok,i will try to do some benchmark tesing. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18581 **[Test build #81059 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81059/testReport)** for PR 18581 at commit

[GitHub] spark issue #19021: [SPARK-21603][SQL][FOLLOW-UP] Change the default value o...

2017-08-23 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19021 Just for your info, again, I looked into this issue in TPC-DS quries; I added [some code](https://github.com/apache/spark/compare/master...maropu:SPARK-21603-FOLLOWUP-3) to check the actual

[GitHub] spark issue #18652: [SPARK-21497][SQL] Pull non-deterministic equi join keys...

2017-08-23 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18652 Join [t1.a = rand(t2.b), t1.c = rand(t2.d)] Sort Project [t1.a, t1.c] TableScan t1 Sort Project [rand(t2.b) as rand(t2.b),

[GitHub] spark issue #19027: [SPARK-19165][PYTHON][SQL] PySpark APIs using columns as...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19027 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

  1   2   3   4   5   >