[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18869 `map` is not supported, right? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r136502487 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala --- @@ -534,4 +534,115 @@ class InsertIntoHiveTableSuite extends QueryTest

[GitHub] spark pull request #19082: [SPARK-21870][SQL] Split aggregation code into sm...

2017-08-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19082#discussion_r136500779 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -244,6 +246,92 @@ case class HashAggregateExec(

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18931 **[Test build #81307 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81307/testReport)** for PR 18931 at commit

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r136499419 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala --- @@ -534,4 +534,115 @@ class InsertIntoHiveTableSuite extends QueryTest

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r136497740 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -140,6 +141,10 @@ case class

[GitHub] spark issue #18647: [SPARK-21789][PYTHON] Remove obsolete codes for parsing ...

2017-08-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18647 Thank you @felixcheung and @holdenk. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18647: [SPARK-21789][PYTHON] Remove obsolete codes for parsing ...

2017-08-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18647 Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #18647: [SPARK-21789][PYTHON] Remove obsolete codes for p...

2017-08-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18647 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18647: [SPARK-21789][PYTHON] Remove obsolete codes for parsing ...

2017-08-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18647 I double checked these **`_split_schema_abstract`**, **`_parse_field_abstract`**, **`_parse_schema_abstract`** and **`_infer_schema_type`** are not used in a public API. Under

[GitHub] spark pull request #18999: [SPARK-21779][PYTHON] Simpler DataFrame.sample AP...

2017-08-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18999 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18999: [SPARK-21779][PYTHON] Simpler DataFrame.sample API in Py...

2017-08-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18999 Thank you @viirya, @felixcheung, @rxin and @ueshin. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #18999: [SPARK-21779][PYTHON] Simpler DataFrame.sample API in Py...

2017-08-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18999 Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #19090: [SPARK-21877][DEPLOY, WINDOWS] Handle quotes in Windows ...

2017-08-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19090 Looks ok given the examples & syntax - https://ss64.com/nt/cmd.html and https://technet.microsoft.com/en-us/library/cc771320(v=ws.11).aspx and my manual tests. I think here is the

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18869 **[Test build #81306 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81306/testReport)** for PR 18869 at commit

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-31 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18869 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-31 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18869 @gatorsmile @HyukjinKwon #18818 is merged now. Can this PR go ahead? As what #18818 did is to allow structs, arrays to be input expression for predicates, currently looks like we don't have

[GitHub] spark issue #19090: [SPARK-21877][DEPLOY, WINDOWS] Handle quotes in Windows ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19090 **[Test build #81305 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81305/testReport)** for PR 19090 at commit

[GitHub] spark issue #19090: [SPARK-21877][DEPLOY, WINDOWS] Handle quotes in Windows ...

2017-08-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19090 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #18931: [SPARK-21717][SQL] Decouple consume functions of ...

2017-08-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18931#discussion_r136492055 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -149,14 +149,144 @@ trait CodegenSupport extends

[GitHub] spark pull request #18931: [SPARK-21717][SQL] Decouple consume functions of ...

2017-08-31 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18931#discussion_r136491920 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -149,14 +149,144 @@ trait CodegenSupport extends

[GitHub] spark pull request #18966: [SPARK-21751][SQL] CodeGeneraor.splitExpressions ...

2017-08-31 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18966#discussion_r136491409 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -769,16 +769,27 @@ class CodegenContext {

[GitHub] spark issue #19077: [SPARK-21860][core]Improve memory reuse for heap memory ...

2017-08-31 Thread 10110346
Github user 10110346 commented on the issue: https://github.com/apache/spark/pull/19077 @jerryshao @JoshRosen yes, it would not generally be arbitrary sized allocations. Basically, we allocate memory in multiples of 4 or 8 bytes,even so, I think this change is also beneficial .

[GitHub] spark issue #18999: [SPARK-21779][PYTHON] Simpler DataFrame.sample API in Py...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18999 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18999: [SPARK-21779][PYTHON] Simpler DataFrame.sample API in Py...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18999 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81303/ Test PASSed. ---

[GitHub] spark issue #18999: [SPARK-21779][PYTHON] Simpler DataFrame.sample API in Py...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18999 **[Test build #81303 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81303/testReport)** for PR 18999 at commit

[GitHub] spark issue #18704: [SPARK-20783][SQL] Create ColumnVector to abstract exist...

2017-08-31 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18704 @cloud-fan Resolved conflict, could you please review? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #19082: [SPARK-21870][SQL] Split aggregation code into sm...

2017-08-31 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19082#discussion_r136490017 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -244,6 +246,92 @@ case class HashAggregateExec(

[GitHub] spark pull request #19077: [SPARK-21860][core]Improve memory reuse for heap ...

2017-08-31 Thread 10110346
Github user 10110346 commented on a diff in the pull request: https://github.com/apache/spark/pull/19077#discussion_r136489896 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/HeapMemoryAllocator.java --- @@ -47,23 +47,29 @@ private boolean shouldPool(long

[GitHub] spark pull request #19077: [SPARK-21860][core]Improve memory reuse for heap ...

2017-08-31 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/19077#discussion_r136487281 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/HeapMemoryAllocator.java --- @@ -47,23 +47,29 @@ private boolean shouldPool(long

[GitHub] spark issue #19077: [SPARK-21860][core]Improve memory reuse for heap memory ...

2017-08-31 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/19077 Just curious: do you know where are we allocating these close-in-size chunks of memory? I understand the motivation, but just curious to know what's causing this pattern. I think the original

[GitHub] spark issue #19100: [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL statement: ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19100 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81300/ Test PASSed. ---

[GitHub] spark issue #19100: [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL statement: ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19100 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19100: [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL statement: ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19100 **[Test build #81300 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81300/testReport)** for PR 19100 at commit

[GitHub] spark issue #19101: [SPARK-21884] [BACKPORT-2.2] [SPARK-21477] [SQL] Mark Lo...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19101 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19101: [SPARK-21884] [BACKPORT-2.2] [SPARK-21477] [SQL] Mark Lo...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19101 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81302/ Test PASSed. ---

[GitHub] spark issue #19101: [SPARK-21884] [BACKPORT-2.2] [SPARK-21477] [SQL] Mark Lo...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19101 **[Test build #81302 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81302/testReport)** for PR 19101 at commit

[GitHub] spark issue #17014: [SPARK-18608][ML] Fix double-caching in ML algorithms

2017-08-31 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/17014 @WeichenXu123 Sounds good. And since adding `handlePersistence` as a `ml.Param` may influences many algs (more than that in this PR), I think we may need more discussion @MLnick @yanboliang

[GitHub] spark pull request #16774: [SPARK-19357][ML] Adding parallel model evaluatio...

2017-08-31 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/16774#discussion_r136482755 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tuning/CrossValidatorSuite.scala --- @@ -120,6 +120,33 @@ class CrossValidatorSuite }

[GitHub] spark pull request #19060: [WIP][SQL] Add DataSourceSuite validating data so...

2017-08-31 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19060#discussion_r136475026 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/sources/DataSourceSuite.scala --- @@ -0,0 +1,151 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #17014: [SPARK-18608][ML] Fix double-caching in ML algorithms

2017-08-31 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/17014 @smurching Yes this should be added as a `ml.Param`, we should not add as an argument. @zhengruifeng Would you mind update the PR according to our discussion result above ? Make

[GitHub] spark issue #15334: [SPARK-10367][SQL] Support Parquet logical type INTERVAL

2017-08-31 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/15334 @gatorsmile Hi Sean, i tried apache-drill after looking through their documentation. And they are able to encode interval data into parquet. ``` 0: jdbc:drill:zk=local> CREATE TABLE

[GitHub] spark issue #18647: [SPARK-21789][PYTHON] Remove obsolete codes for parsing ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18647 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18647: [SPARK-21789][PYTHON] Remove obsolete codes for parsing ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18647 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81304/ Test PASSed. ---

[GitHub] spark issue #18647: [SPARK-21789][PYTHON] Remove obsolete codes for parsing ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18647 **[Test build #81304 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81304/testReport)** for PR 18647 at commit

[GitHub] spark pull request #19099: [SPARK-21652][SQL] Fix rule confliction between I...

2017-08-31 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19099#discussion_r136479931 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -2663,4 +2664,31 @@ class SQLQuerySuite extends QueryTest with

[GitHub] spark pull request #19099: [SPARK-21652][SQL] Fix rule confliction between I...

2017-08-31 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19099#discussion_r136479686 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -79,11 +79,11 @@ abstract class

[GitHub] spark issue #18647: [SPARK-21789][PYTHON] Remove obsolete codes for parsing ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18647 **[Test build #81304 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81304/testReport)** for PR 18647 at commit

[GitHub] spark issue #18999: [SPARK-21779][PYTHON] Simpler DataFrame.sample API in Py...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18999 **[Test build #81303 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81303/testReport)** for PR 18999 at commit

[GitHub] spark issue #18999: [SPARK-21779][PYTHON] Simpler DataFrame.sample API in Py...

2017-08-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18999 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #18647: [SPARK-21789][PYTHON] Remove obsolete codes for parsing ...

2017-08-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18647 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #19080: [SPARK-21865][SQL] simplify the distribution sema...

2017-08-31 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19080#discussion_r136477108 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/physical/partitioning.scala --- @@ -284,24 +241,17 @@ case class

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-08-31 Thread janewangfb
Github user janewangfb commented on the issue: https://github.com/apache/spark/pull/18975 Jenkin test please! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #19080: [SPARK-21865][SQL] simplify the distribution sema...

2017-08-31 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19080#discussion_r136476025 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/physical/partitioning.scala --- @@ -284,24 +241,17 @@ case class

[GitHub] spark issue #19100: [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL statement: ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19100 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81301/ Test FAILed. ---

[GitHub] spark issue #19060: [WIP][SQL] Add DataSourceSuite validating data sources l...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19060 **[Test build #81298 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81298/testReport)** for PR 19060 at commit

[GitHub] spark issue #19060: [WIP][SQL] Add DataSourceSuite validating data sources l...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19060 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19060: [WIP][SQL] Add DataSourceSuite validating data sources l...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19060 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81298/ Test PASSed. ---

[GitHub] spark issue #19100: [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL statement: ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19100 **[Test build #81301 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81301/testReport)** for PR 19100 at commit

[GitHub] spark issue #19101: [SPARK-21884] [BACKPORT-2.2] [SPARK-21477] [SQL] Mark Lo...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19101 **[Test build #81302 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81302/testReport)** for PR 19101 at commit

[GitHub] spark issue #19100: [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL statement: ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19100 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #19101: [SPARK-21884] [BACKPORT-2.2] [SPARK-21477] [SQL] ...

2017-08-31 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/19101 [SPARK-21884] [BACKPORT-2.2] [SPARK-21477] [SQL] Mark LocalTableScanExec's input data transient This PR is to backport https://github.com/apache/spark/pull/18686 for resolving the issue in

[GitHub] spark issue #18317: [SPARK-21113][CORE] Read ahead input stream to amortize ...

2017-08-31 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/18317 ping @zsxwing ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18975 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18975 **[Test build #81297 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81297/testReport)** for PR 18975 at commit

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18975 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81297/ Test FAILed. ---

[GitHub] spark issue #19050: [SPARK-21835][SQL] RewritePredicateSubquery should not p...

2017-08-31 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19050 ping @cloud-fan @hvanhovell Can you have time to review this? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #19100: [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL statement: ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19100 **[Test build #81301 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81301/testReport)** for PR 19100 at commit

[GitHub] spark pull request #19100: [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL stat...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19100#discussion_r136473016 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/OptimizeMetadataOnlyQuerySuite.scala --- @@ -117,4 +117,12 @@ class

[GitHub] spark issue #19100: [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL statement: ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19100 **[Test build #81300 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81300/testReport)** for PR 19100 at commit

[GitHub] spark pull request #19100: [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL stat...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19100#discussion_r136472066 --- Diff: sql/core/src/test/resources/sql-tests/results/cross-join.sql.out --- @@ -128,6 +128,7 @@ two 2 two 2 one 1 two

[GitHub] spark issue #19100: [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL statement: ...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19100 @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #19100: [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL stat...

2017-08-31 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/19100 [SPARK-21891] [SQL] Add TBLPROPERTIES to DDL statement: CREATE TABLE USING ## What changes were proposed in this pull request? Add `TBLPROPERTIES` to the DDL statement `CREATE TABLE USING`.

[GitHub] spark pull request #19078: [SPARK-21862][ML] Add overflow check in PCA

2017-08-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19078 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #19072: [SPARK-17139][ML][FOLLOW-UP] Add convenient metho...

2017-08-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19072 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #19078: [SPARK-21862][ML] Add overflow check in PCA

2017-08-31 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/19078 LGTM Merging with master Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18686: [SPARK-21477] [SQL] [MINOR] Mark LocalTableScanExec's in...

2017-08-31 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18686 Thank you so much! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #19072: [SPARK-17139][ML][FOLLOW-UP] Add convenient metho...

2017-08-31 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19072#discussion_r136470877 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -1473,6 +1473,17 @@ sealed trait

[GitHub] spark issue #18686: [SPARK-21477] [SQL] [MINOR] Mark LocalTableScanExec's in...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18686 Sure. Will do it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #18686: [SPARK-21477] [SQL] [MINOR] Mark LocalTableScanExec's in...

2017-08-31 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18686 Yes. This is the fix. @gatorsmile and @cloud-fan . Can we have this in branch-2.2, too? --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #19094: [SPARK-21884][SQL][BRANCH-2.2] Fix StackOverflowError on...

2017-08-31 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19094 I close this issue. Thank you again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #19094: [SPARK-21884][SQL][BRANCH-2.2] Fix StackOverflowE...

2017-08-31 Thread dongjoon-hyun
Github user dongjoon-hyun closed the pull request at: https://github.com/apache/spark/pull/19094 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #19099: [SPARK-21652][SQL] Fix rule confliction between InferFil...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19099 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18686: [SPARK-21477] [SQL] [MINOR] Mark LocalTableScanExec's in...

2017-08-31 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18686 I'm testing now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #19099: [SPARK-21652][SQL] Fix rule confliction between InferFil...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19099 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81299/ Test FAILed. ---

[GitHub] spark issue #19099: [SPARK-21652][SQL] Fix rule confliction between InferFil...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19099 **[Test build #81299 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81299/testReport)** for PR 19099 at commit

[GitHub] spark issue #19094: [SPARK-21884][SQL][BRANCH-2.2] Fix StackOverflowError on...

2017-08-31 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19094 Thank you! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #18686: [SPARK-21477] [SQL] [MINOR] Mark LocalTableScanExec's in...

2017-08-31 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18686 Oh, @gatorsmile . I didn't find the right fix. Could you backport this to branch-2.2, too? --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #19094: [SPARK-21884][SQL][BRANCH-2.2] Fix StackOverflowError on...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19094 https://github.com/apache/spark/pull/18686 My fix resolves your issue, right? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #19093: [SPARK-21880][web UI]In the SQL table page, modify jobs ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19093 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81296/ Test PASSed. ---

[GitHub] spark issue #19093: [SPARK-21880][web UI]In the SQL table page, modify jobs ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19093 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19093: [SPARK-21880][web UI]In the SQL table page, modify jobs ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19093 **[Test build #81296 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81296/testReport)** for PR 19093 at commit

[GitHub] spark pull request #18818: [SPARK-21110][SQL] Structs, arrays, and other ord...

2017-08-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18818 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18704: [SPARK-20783][SQL] Create ColumnVector to abstract exist...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18704 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18704: [SPARK-20783][SQL] Create ColumnVector to abstract exist...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18704 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81295/ Test PASSed. ---

[GitHub] spark issue #18818: [SPARK-21110][SQL] Structs, arrays, and other orderable ...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18818 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18704: [SPARK-20783][SQL] Create ColumnVector to abstract exist...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18704 **[Test build #81295 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81295/testReport)** for PR 18704 at commit

[GitHub] spark issue #18818: [SPARK-21110][SQL] Structs, arrays, and other orderable ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18818 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18818: [SPARK-21110][SQL] Structs, arrays, and other orderable ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18818 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81294/ Test PASSed. ---

[GitHub] spark issue #18818: [SPARK-21110][SQL] Structs, arrays, and other orderable ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18818 **[Test build #81294 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81294/testReport)** for PR 18818 at commit

  1   2   3   >