[GitHub] spark pull request #14864: [SPARK-15453] [SQL] FileSourceScanExec to extract...

2016-09-08 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/14864#discussion_r78129784 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -156,24 +155,57 @@ case class FileSourceScanExec(

[GitHub] spark issue #14980: [SPARK-17317][SparkR] Add SparkR vignette

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14980 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65134/ Test PASSed. ---

[GitHub] spark issue #15020: Spark 2.0 error in Intellij

2016-09-08 Thread yintengfei
Github user yintengfei commented on the issue: https://github.com/apache/spark/pull/15020 You can find : File -> Project Structure -> Modules -> Dependcies , change the provided to compile. As HyukjinKwon say , you'd better ask this to user-mailing list . --- If your projec

[GitHub] spark issue #14980: [SPARK-17317][SparkR] Add SparkR vignette

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14980 **[Test build #65134 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65134/consoleFull)** for PR 14980 at commit [`7b55255`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14980: [SPARK-17317][SparkR] Add SparkR vignette

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14980 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14864: [SPARK-15453] [SQL] FileSourceScanExec to extract `outpu...

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14864 **[Test build #65136 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65136/consoleFull)** for PR 14864 at commit [`445549b`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #15015: [SPARK-16445][MLlib][SparkR] Fix @return description for...

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15015 **[Test build #65135 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65135/consoleFull)** for PR 15015 at commit [`91ae8d6`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #14912: [SPARK-17357][SQL] Fix current predicate pushdown

2016-09-08 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14912 Could you define the conditions in which the predicates are unable to be pushed down? Then, we can easily justify the significance. --- If your project is set up for it, you can reply to this e

[GitHub] spark issue #15015: [SPARK-16445][MLlib][SparkR] Fix @return description for...

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15015 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65135/ Test PASSed. ---

[GitHub] spark issue #15015: [SPARK-16445][MLlib][SparkR] Fix @return description for...

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15015 **[Test build #65135 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65135/consoleFull)** for PR 15015 at commit [`91ae8d6`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15015: [SPARK-16445][MLlib][SparkR] Fix @return description for...

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15015 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15004: [SPARK-17440] [SPARK-17441] Fixed Multiple Bugs in ALTER...

2016-09-08 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15004 Accepted all the comments. Will address comments now. : ) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #14963: [SPARK-16992][PYSPARK] Reenable Pylint

2016-09-08 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/14963#discussion_r78133102 --- Diff: dev/lint-python --- @@ -26,30 +67,26 @@ PYLINT_REPORT_PATH="$SPARK_ROOT_DIR/dev/pylint-report.txt" PYLINT_INSTALL_INFO="$SPARK_ROOT_DIR/dev/py

[GitHub] spark pull request #14963: [SPARK-16992][PYSPARK] Reenable Pylint

2016-09-08 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/14963#discussion_r78133178 --- Diff: dev/lint-python --- @@ -17,6 +17,47 @@ # limitations under the License. # +VIRTUAL_ENV_DIR="build/venv" +if [[ ! -z $VIRTUAL

[GitHub] spark issue #14963: [SPARK-16992][PYSPARK] Reenable Pylint

2016-09-08 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14963 Thanks for working to reenable pylint (super important IMHO). Perhaps @davies could take a look at this approach? I like the building up of venv seems to help set up for your other work improvin

[GitHub] spark issue #15004: [SPARK-17440] [SPARK-17441] Fixed Multiple Bugs in ALTER...

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15004 **[Test build #65137 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65137/consoleFull)** for PR 15004 at commit [`5d0cffe`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #15014: [SPARK-17429][SQL] use ImplicitCastInputTypes with funct...

2016-09-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15014 The only one error looks just about the changed behavior in `StringFunctionsSuite`. Could you fix that? --- If your project is set up for it, you can reply to this email and have your reply a

[GitHub] spark issue #15021: [SPARK-17464][SparkR][ML] SparkR spark.als argument reg ...

2016-09-08 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15021 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the f

[GitHub] spark issue #15015: [SPARK-16445][MLlib][SparkR] Fix @return description for...

2016-09-08 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15015 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the f

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14638 **[Test build #65138 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65138/consoleFull)** for PR 14638 at commit [`63bcc08`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #14527: [SPARK-16938][SQL] `drop/dropDuplicate` should handle th...

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14527 **[Test build #65140 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65140/consoleFull)** for PR 14527 at commit [`c18781e`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #14116: [SPARK-16452][SQL] Support basic INFORMATION_SCHEMA

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14116 **[Test build #65141 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65141/consoleFull)** for PR 14116 at commit [`a8c30c6`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #14623: [SPARK-17044][SQL] Make test files for window functions ...

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14623 **[Test build #65139 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65139/consoleFull)** for PR 14623 at commit [`679efb2`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #14426: [SPARK-16475][SQL] Broadcast Hint for SQL Queries

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14426 **[Test build #65142 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65142/consoleFull)** for PR 14426 at commit [`01105f5`](https://github.com/apache/spark/commit/0

[GitHub] spark pull request #15004: [SPARK-17440] [SPARK-17441] Fixed Multiple Bugs i...

2016-09-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15004#discussion_r78135593 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -383,7 +385,14 @@ case class AlterTableRenamePartitionCommand(

[GitHub] spark pull request #14963: [SPARK-16992][PYSPARK] Reenable Pylint

2016-09-08 Thread Stibbons
Github user Stibbons commented on a diff in the pull request: https://github.com/apache/spark/pull/14963#discussion_r78135524 --- Diff: dev/lint-python --- @@ -26,30 +67,26 @@ PYLINT_REPORT_PATH="$SPARK_ROOT_DIR/dev/pylint-report.txt" PYLINT_INSTALL_INFO="$SPARK_ROOT_DIR/dev/p

[GitHub] spark pull request #14963: [SPARK-16992][PYSPARK] Reenable Pylint

2016-09-08 Thread Stibbons
Github user Stibbons commented on a diff in the pull request: https://github.com/apache/spark/pull/14963#discussion_r78135582 --- Diff: dev/requirements.txt --- @@ -1,3 +1,5 @@ jira==1.0.3 PyGithub==1.26.0 Unidecode==0.04.19 +pep8==1.7.0 --- End diff --

[GitHub] spark issue #14963: [SPARK-16992][PYSPARK] Reenable Pylint

2016-09-08 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14963 Oh wait I've been a little confused by the PR title - pylint is already reenabled and was only temporary disabled awhile back - maybe less important but would make sense in the context of the other

[GitHub] spark pull request #14963: [SPARK-16992][PYSPARK] Reenable Pylint

2016-09-08 Thread Stibbons
Github user Stibbons commented on a diff in the pull request: https://github.com/apache/spark/pull/14963#discussion_r78135769 --- Diff: dev/lint-python --- @@ -17,6 +17,47 @@ # limitations under the License. # +VIRTUAL_ENV_DIR="build/venv" +if [[ ! -z $VIRTUA

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution by dedupli...

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14452 **[Test build #65143 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65143/consoleFull)** for PR 14452 at commit [`23e2dc8`](https://github.com/apache/spark/commit/2

[GitHub] spark pull request #14963: [SPARK-16992][PYSPARK] Reenable Pylint

2016-09-08 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/14963#discussion_r78135944 --- Diff: dev/lint-python --- @@ -26,30 +67,26 @@ PYLINT_REPORT_PATH="$SPARK_ROOT_DIR/dev/pylint-report.txt" PYLINT_INSTALL_INFO="$SPARK_ROOT_DIR/dev/py

[GitHub] spark issue #14963: [SPARK-16992][PYSPARK] Reenable Pylint

2016-09-08 Thread Stibbons
Github user Stibbons commented on the issue: https://github.com/apache/spark/pull/14963 Hum I don't see how it was reenabled... Where is it called? And I had many errors to ignore once I have reenabled it on the execution of lint-python. I'll update the title. At least, being

[GitHub] spark issue #14912: [SPARK-17357][SQL] Fix current predicate pushdown

2016-09-08 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14912 @gatorsmile I've described it in the pr description. Simply said, now a Filter will be stopped to pushdown once it encounters another Filter. `BooleanSimplification` rule will simplify the pr

[GitHub] spark pull request #14963: [SPARK-16992][PYSPARK] Virtualenv for Pylint and ...

2016-09-08 Thread Stibbons
Github user Stibbons commented on a diff in the pull request: https://github.com/apache/spark/pull/14963#discussion_r78136301 --- Diff: dev/lint-python --- @@ -26,30 +67,26 @@ PYLINT_REPORT_PATH="$SPARK_ROOT_DIR/dev/pylint-report.txt" PYLINT_INSTALL_INFO="$SPARK_ROOT_DIR/dev/p

[GitHub] spark issue #14830: [SPARK-16992][PYSPARK][DOCS] PEP8 on Pyspark documentati...

2016-09-08 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14830 Could you maybe give this a clearer PR title as well? Right now it seems to imply the current examples aren't passing the pep8 tool but they currently are. Maybe something like "import sort and auto

<    1   2   3   4   5