[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r136509156 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveDirCommand.scala --- @@ -0,0 +1,134 @@ +/* + * Licensed to t

[GitHub] spark issue #19090: [SPARK-21877][DEPLOY, WINDOWS] Handle quotes in Windows ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19090 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81305/ Test PASSed. ---

[GitHub] spark issue #19090: [SPARK-21877][DEPLOY, WINDOWS] Handle quotes in Windows ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19090 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #19090: [SPARK-21877][DEPLOY, WINDOWS] Handle quotes in Windows ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19090 **[Test build #81305 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81305/testReport)** for PR 19090 at commit [`26fc756`](https://github.com/apache/spark/commit/2

[GitHub] spark pull request #19082: [SPARK-21870][SQL] Split aggregation code into sm...

2017-08-31 Thread rednaxelafx
Github user rednaxelafx commented on a diff in the pull request: https://github.com/apache/spark/pull/19082#discussion_r136506452 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -244,6 +246,92 @@ case class HashAggregateExe

[GitHub] spark pull request #19082: [SPARK-21870][SQL] Split aggregation code into sm...

2017-08-31 Thread rednaxelafx
Github user rednaxelafx commented on a diff in the pull request: https://github.com/apache/spark/pull/19082#discussion_r136506046 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -244,6 +246,92 @@ case class HashAggregateExe

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r136508091 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveDirCommand.scala --- @@ -0,0 +1,134 @@ +/* + * Licensed to t

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r136508055 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveDirCommand.scala --- @@ -0,0 +1,134 @@ +/* + * Licensed to t

[GitHub] spark issue #18697: [SPARK-16683][SQL] Repeated joins to same table can leak...

2017-08-31 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18697 shouldn't we fix `ProjectExec.outputPartitioning`? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18869 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81306/ Test PASSed. ---

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18869 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18869 **[Test build #81306 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81306/testReport)** for PR 18869 at commit [`b64c9e6`](https://github.com/apache/spark/commit/b

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r136506540 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveDirCommand.scala --- @@ -0,0 +1,134 @@ +/* + * Licensed to t

[GitHub] spark issue #19102: [SPARK-21859][CORE] Fix SparkFiles.get failed on driver ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19102 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #19079: [SPARK-21859][CORE] Fix SparkFiles.get failed on driver ...

2017-08-31 Thread lgrcyanny
Github user lgrcyanny commented on the issue: https://github.com/apache/spark/pull/19079 Hi @vanzin I have submit a PR based on master branch, please review it, thank you https://github.com/apache/spark/pull/19102 --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request #19102: [SPARK-21859][CORE] Fix SparkFiles.get failed on ...

2017-08-31 Thread lgrcyanny
GitHub user lgrcyanny opened a pull request: https://github.com/apache/spark/pull/19102 [SPARK-21859][CORE] Fix SparkFiles.get failed on driver in yarn-cluster and yarn-client mode ## What changes were proposed in this pull request? when use SparkFiles.get a file on driver in ya

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-31 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18869 @gatorsmile Right. Isn't too verbose if we describe map is not supported in each description? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub a

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18869 `map` is not supported, right? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r136502487 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala --- @@ -534,4 +534,115 @@ class InsertIntoHiveTableSuite extends QueryTest

[GitHub] spark pull request #19082: [SPARK-21870][SQL] Split aggregation code into sm...

2017-08-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19082#discussion_r136500779 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -244,6 +246,92 @@ case class HashAggregateExec(

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18931 **[Test build #81307 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81307/testReport)** for PR 18931 at commit [`1101b2c`](https://github.com/apache/spark/commit/11

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r136499419 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala --- @@ -534,4 +534,115 @@ class InsertIntoHiveTableSuite extends QueryTest

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r136497740 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -140,6 +141,10 @@ case class DataSourceAnal

[GitHub] spark issue #18647: [SPARK-21789][PYTHON] Remove obsolete codes for parsing ...

2017-08-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18647 Thank you @felixcheung and @holdenk. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature en

[GitHub] spark issue #18647: [SPARK-21789][PYTHON] Remove obsolete codes for parsing ...

2017-08-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18647 Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #18647: [SPARK-21789][PYTHON] Remove obsolete codes for p...

2017-08-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18647 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #18647: [SPARK-21789][PYTHON] Remove obsolete codes for parsing ...

2017-08-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18647 I double checked these **`_split_schema_abstract`**, **`_parse_field_abstract`**, **`_parse_schema_abstract`** and **`_infer_schema_type`** are not used in a public API. Under `./python

[GitHub] spark pull request #18999: [SPARK-21779][PYTHON] Simpler DataFrame.sample AP...

2017-08-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18999 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #18999: [SPARK-21779][PYTHON] Simpler DataFrame.sample API in Py...

2017-08-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18999 Thank you @viirya, @felixcheung, @rxin and @ueshin. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #18999: [SPARK-21779][PYTHON] Simpler DataFrame.sample API in Py...

2017-08-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18999 Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #19090: [SPARK-21877][DEPLOY, WINDOWS] Handle quotes in Windows ...

2017-08-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19090 Looks ok given the examples & syntax - https://ss64.com/nt/cmd.html and https://technet.microsoft.com/en-us/library/cc771320(v=ws.11).aspx and my manual tests. I think here is the very

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18869 **[Test build #81306 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81306/testReport)** for PR 18869 at commit [`b64c9e6`](https://github.com/apache/spark/commit/b6

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-31 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18869 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, o

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-31 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18869 @gatorsmile @HyukjinKwon #18818 is merged now. Can this PR go ahead? As what #18818 did is to allow structs, arrays to be input expression for predicates, currently looks like we don't have e

[GitHub] spark issue #19090: [SPARK-21877][DEPLOY, WINDOWS] Handle quotes in Windows ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19090 **[Test build #81305 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81305/testReport)** for PR 19090 at commit [`26fc756`](https://github.com/apache/spark/commit/26

[GitHub] spark issue #19090: [SPARK-21877][DEPLOY, WINDOWS] Handle quotes in Windows ...

2017-08-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19090 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #18931: [SPARK-21717][SQL] Decouple consume functions of ...

2017-08-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18931#discussion_r136492055 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -149,14 +149,144 @@ trait CodegenSupport extends SparkPl

[GitHub] spark pull request #18931: [SPARK-21717][SQL] Decouple consume functions of ...

2017-08-31 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18931#discussion_r136491920 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -149,14 +149,144 @@ trait CodegenSupport extends SparkPla

[GitHub] spark pull request #18966: [SPARK-21751][SQL] CodeGeneraor.splitExpressions ...

2017-08-31 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18966#discussion_r136491409 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -769,16 +769,27 @@ class CodegenContext {

[GitHub] spark issue #19077: [SPARK-21860][core]Improve memory reuse for heap memory ...

2017-08-31 Thread 10110346
Github user 10110346 commented on the issue: https://github.com/apache/spark/pull/19077 @jerryshao @JoshRosen yes, it would not generally be arbitrary sized allocations. Basically, we allocate memory in multiples of 4 or 8 bytes,even so, I think this change is also beneficial .

[GitHub] spark issue #18999: [SPARK-21779][PYTHON] Simpler DataFrame.sample API in Py...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18999 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18999: [SPARK-21779][PYTHON] Simpler DataFrame.sample API in Py...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18999 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81303/ Test PASSed. ---

[GitHub] spark issue #18999: [SPARK-21779][PYTHON] Simpler DataFrame.sample API in Py...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18999 **[Test build #81303 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81303/testReport)** for PR 18999 at commit [`f2608ab`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #18704: [SPARK-20783][SQL] Create ColumnVector to abstract exist...

2017-08-31 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18704 @cloud-fan Resolved conflict, could you please review? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request #19082: [SPARK-21870][SQL] Split aggregation code into sm...

2017-08-31 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19082#discussion_r136490017 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -244,6 +246,92 @@ case class HashAggregateExec(

[GitHub] spark pull request #19077: [SPARK-21860][core]Improve memory reuse for heap ...

2017-08-31 Thread 10110346
Github user 10110346 commented on a diff in the pull request: https://github.com/apache/spark/pull/19077#discussion_r136489896 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/HeapMemoryAllocator.java --- @@ -47,23 +47,29 @@ private boolean shouldPool(long size

[GitHub] spark pull request #19077: [SPARK-21860][core]Improve memory reuse for heap ...

2017-08-31 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/19077#discussion_r136487281 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/HeapMemoryAllocator.java --- @@ -47,23 +47,29 @@ private boolean shouldPool(long siz

[GitHub] spark issue #19077: [SPARK-21860][core]Improve memory reuse for heap memory ...

2017-08-31 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/19077 Just curious: do you know where are we allocating these close-in-size chunks of memory? I understand the motivation, but just curious to know what's causing this pattern. I think the original idea

[GitHub] spark issue #19100: [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL statement: ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19100 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81300/ Test PASSed. ---

[GitHub] spark issue #19100: [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL statement: ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19100 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #19100: [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL statement: ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19100 **[Test build #81300 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81300/testReport)** for PR 19100 at commit [`7dbd810`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #19101: [SPARK-21884] [BACKPORT-2.2] [SPARK-21477] [SQL] Mark Lo...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19101 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #19101: [SPARK-21884] [BACKPORT-2.2] [SPARK-21477] [SQL] Mark Lo...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19101 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81302/ Test PASSed. ---

[GitHub] spark issue #19101: [SPARK-21884] [BACKPORT-2.2] [SPARK-21477] [SQL] Mark Lo...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19101 **[Test build #81302 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81302/testReport)** for PR 19101 at commit [`3ee18ce`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #17014: [SPARK-18608][ML] Fix double-caching in ML algorithms

2017-08-31 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/17014 @WeichenXu123 Sounds good. And since adding `handlePersistence` as a `ml.Param` may influences many algs (more than that in this PR), I think we may need more discussion @MLnick @yanboliang

[GitHub] spark pull request #16774: [SPARK-19357][ML] Adding parallel model evaluatio...

2017-08-31 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/16774#discussion_r136482755 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tuning/CrossValidatorSuite.scala --- @@ -120,6 +120,33 @@ class CrossValidatorSuite }

[GitHub] spark pull request #19060: [WIP][SQL] Add DataSourceSuite validating data so...

2017-08-31 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19060#discussion_r136475026 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/sources/DataSourceSuite.scala --- @@ -0,0 +1,151 @@ +/* + * Licensed to the Apache Soft

[GitHub] spark issue #17014: [SPARK-18608][ML] Fix double-caching in ML algorithms

2017-08-31 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/17014 @smurching Yes this should be added as a `ml.Param`, we should not add as an argument. @zhengruifeng Would you mind update the PR according to our discussion result above ? Make `handle

[GitHub] spark issue #15334: [SPARK-10367][SQL] Support Parquet logical type INTERVAL

2017-08-31 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/15334 @gatorsmile Hi Sean, i tried apache-drill after looking through their documentation. And they are able to encode interval data into parquet. ``` 0: jdbc:drill:zk=local> CREATE TABLE

[GitHub] spark issue #18647: [SPARK-21789][PYTHON] Remove obsolete codes for parsing ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18647 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18647: [SPARK-21789][PYTHON] Remove obsolete codes for parsing ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18647 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81304/ Test PASSed. ---

[GitHub] spark issue #18647: [SPARK-21789][PYTHON] Remove obsolete codes for parsing ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18647 **[Test build #81304 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81304/testReport)** for PR 18647 at commit [`83228cb`](https://github.com/apache/spark/commit/8

[GitHub] spark pull request #19099: [SPARK-21652][SQL] Fix rule confliction between I...

2017-08-31 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19099#discussion_r136479931 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -2663,4 +2664,31 @@ class SQLQuerySuite extends QueryTest with SharedSQLCo

[GitHub] spark pull request #19099: [SPARK-21652][SQL] Fix rule confliction between I...

2017-08-31 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19099#discussion_r136479686 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -79,11 +79,11 @@ abstract class Optimizer(sessionCatalo

[GitHub] spark issue #18647: [SPARK-21789][PYTHON] Remove obsolete codes for parsing ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18647 **[Test build #81304 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81304/testReport)** for PR 18647 at commit [`83228cb`](https://github.com/apache/spark/commit/83

[GitHub] spark issue #18999: [SPARK-21779][PYTHON] Simpler DataFrame.sample API in Py...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18999 **[Test build #81303 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81303/testReport)** for PR 18999 at commit [`f2608ab`](https://github.com/apache/spark/commit/f2

[GitHub] spark issue #18999: [SPARK-21779][PYTHON] Simpler DataFrame.sample API in Py...

2017-08-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18999 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes s

[GitHub] spark issue #18647: [SPARK-21789][PYTHON] Remove obsolete codes for parsing ...

2017-08-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18647 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes s

[GitHub] spark pull request #19080: [SPARK-21865][SQL] simplify the distribution sema...

2017-08-31 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19080#discussion_r136477108 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/physical/partitioning.scala --- @@ -284,24 +241,17 @@ case class RangePartition

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-08-31 Thread janewangfb
Github user janewangfb commented on the issue: https://github.com/apache/spark/pull/18975 Jenkin test please! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes s

[GitHub] spark pull request #19080: [SPARK-21865][SQL] simplify the distribution sema...

2017-08-31 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19080#discussion_r136476025 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/physical/partitioning.scala --- @@ -284,24 +241,17 @@ case class RangePartition

[GitHub] spark issue #19100: [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL statement: ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19100 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81301/ Test FAILed. ---

[GitHub] spark issue #19060: [WIP][SQL] Add DataSourceSuite validating data sources l...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19060 **[Test build #81298 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81298/testReport)** for PR 19060 at commit [`104f24c`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #19060: [WIP][SQL] Add DataSourceSuite validating data sources l...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19060 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #19060: [WIP][SQL] Add DataSourceSuite validating data sources l...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19060 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81298/ Test PASSed. ---

[GitHub] spark issue #19100: [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL statement: ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19100 **[Test build #81301 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81301/testReport)** for PR 19100 at commit [`7954c0b`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #19101: [SPARK-21884] [BACKPORT-2.2] [SPARK-21477] [SQL] Mark Lo...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19101 **[Test build #81302 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81302/testReport)** for PR 19101 at commit [`3ee18ce`](https://github.com/apache/spark/commit/3e

[GitHub] spark issue #19100: [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL statement: ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19100 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #19101: [SPARK-21884] [BACKPORT-2.2] [SPARK-21477] [SQL] ...

2017-08-31 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/19101 [SPARK-21884] [BACKPORT-2.2] [SPARK-21477] [SQL] Mark LocalTableScanExec's input data transient This PR is to backport https://github.com/apache/spark/pull/18686 for resolving the issue in http

[GitHub] spark issue #18317: [SPARK-21113][CORE] Read ahead input stream to amortize ...

2017-08-31 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/18317 ping @zsxwing ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, o

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18975 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18975 **[Test build #81297 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81297/testReport)** for PR 18975 at commit [`e2db5e1`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18975 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81297/ Test FAILed. ---

[GitHub] spark issue #19050: [SPARK-21835][SQL] RewritePredicateSubquery should not p...

2017-08-31 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19050 ping @cloud-fan @hvanhovell Can you have time to review this? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[GitHub] spark issue #19100: [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL statement: ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19100 **[Test build #81301 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81301/testReport)** for PR 19100 at commit [`7954c0b`](https://github.com/apache/spark/commit/79

[GitHub] spark pull request #19100: [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL stat...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19100#discussion_r136473016 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/OptimizeMetadataOnlyQuerySuite.scala --- @@ -117,4 +117,12 @@ class OptimizeMetadataOnl

[GitHub] spark issue #19100: [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL statement: ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19100 **[Test build #81300 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81300/testReport)** for PR 19100 at commit [`7dbd810`](https://github.com/apache/spark/commit/7d

[GitHub] spark pull request #19100: [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL stat...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19100#discussion_r136472066 --- Diff: sql/core/src/test/resources/sql-tests/results/cross-join.sql.out --- @@ -128,6 +128,7 @@ two 2 two 2 one 1 two

[GitHub] spark issue #19100: [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL statement: ...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19100 @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #19100: [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL stat...

2017-08-31 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/19100 [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL statement: CREATE TABLE USING ## What changes were proposed in this pull request? Add `TBLPROPERTIES` to the DDL statement `CREATE TABLE USING`.

[GitHub] spark pull request #19078: [SPARK-21862][ML] Add overflow check in PCA

2017-08-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19078 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #19072: [SPARK-17139][ML][FOLLOW-UP] Add convenient metho...

2017-08-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19072 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #19078: [SPARK-21862][ML] Add overflow check in PCA

2017-08-31 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/19078 LGTM Merging with master Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18686: [SPARK-21477] [SQL] [MINOR] Mark LocalTableScanExec's in...

2017-08-31 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18686 Thank you so much! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #19072: [SPARK-17139][ML][FOLLOW-UP] Add convenient metho...

2017-08-31 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19072#discussion_r136470877 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -1473,6 +1473,17 @@ sealed trait LogisticRegressionSumm

[GitHub] spark issue #18686: [SPARK-21477] [SQL] [MINOR] Mark LocalTableScanExec's in...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18686 Sure. Will do it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #18686: [SPARK-21477] [SQL] [MINOR] Mark LocalTableScanExec's in...

2017-08-31 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18686 Yes. This is the fix. @gatorsmile and @cloud-fan . Can we have this in branch-2.2, too? --- If your project is set up for it, you can reply to this email and have your reply appear on GitH

[GitHub] spark issue #19094: [SPARK-21884][SQL][BRANCH-2.2] Fix StackOverflowError on...

2017-08-31 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19094 I close this issue. Thank you again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #19094: [SPARK-21884][SQL][BRANCH-2.2] Fix StackOverflowE...

2017-08-31 Thread dongjoon-hyun
Github user dongjoon-hyun closed the pull request at: https://github.com/apache/spark/pull/19094 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #19099: [SPARK-21652][SQL] Fix rule confliction between InferFil...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19099 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

  1   2   3   >