[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137416501 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -201,13 +201,14 @@ case class

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137416371 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -201,13 +201,14 @@ case class

[GitHub] spark issue #19149: [SPARK-21652][SQL][FOLLOW-UP] Fix rule conflict between ...

2017-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19149 **[Test build #81483 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81483/testReport)** for PR 19149 at commit

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137415411 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -201,13 +201,14 @@ case class AlterTableAddColumnsCommand(

[GitHub] spark pull request #19148: [SPARK-21936][SQL][WIP] backward compatibility te...

2017-09-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19148#discussion_r137413347 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala --- @@ -0,0 +1,199 @@ +/* + * Licensed to

[GitHub] spark pull request #19148: [SPARK-21936][SQL][WIP] backward compatibility te...

2017-09-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19148#discussion_r137412267 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala --- @@ -0,0 +1,199 @@ +/* + * Licensed to

[GitHub] spark pull request #18659: [SPARK-21190][PYSPARK][WIP] Simple Python Vectori...

2017-09-06 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/18659#discussion_r137412019 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowEvalPythonExec.scala --- @@ -0,0 +1,127 @@ +/* + * Licensed to

[GitHub] spark issue #18659: [SPARK-21190][PYSPARK][WIP] Simple Python Vectorized UDF...

2017-09-06 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/18659 Cool! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19112: [SPARK-21901][SS] Define toString for StateOperat...

2017-09-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19112 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19112: [SPARK-21901][SS] Define toString for StateOperatorProgr...

2017-09-06 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/19112 LGTM. Thanks! Merging to master and 2.2. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19096: [SPARK-21869][SS] A cached Kafka producer should ...

2017-09-06 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19096#discussion_r137409030 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriteTask.scala --- @@ -43,8 +43,10 @@ private[kafka010] class

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18975 **[Test build #81482 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81482/testReport)** for PR 18975 at commit

[GitHub] spark issue #12646: [SPARK-14878][SQL] Trim characters string function suppo...

2017-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12646 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81473/ Test PASSed. ---

[GitHub] spark issue #12646: [SPARK-14878][SQL] Trim characters string function suppo...

2017-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12646 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #12646: [SPARK-14878][SQL] Trim characters string function suppo...

2017-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12646 **[Test build #81473 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81473/testReport)** for PR 12646 at commit

[GitHub] spark issue #19046: [SPARK-18769][yarn] Limit resource requests based on RM'...

2017-09-06 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/19046 Yeah if we are releasing and they reacquiring right away over and over again that would be bad, but I don't know when we would do that so more details would be great if possible. ---

[GitHub] spark issue #18253: [SPARK-18838][CORE] Introduce multiple queues in LiveLis...

2017-09-06 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18253 @bOOm-X I pushed some code to my repo: https://github.com/vanzin/spark/tree/SPARK-18838 Which is an attempt to do things the way I've been trying to explain. It tries to keep

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-06 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137406552 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -2346,6 +2347,45 @@ abstract class DDLSuite extends

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-06 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137406502 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/SaveAsHiveFile.scala --- @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18975 **[Test build #81481 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81481/testReport)** for PR 18975 at commit

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-09-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18975 Looks pretty good! Will do the final pass after addressing the above comments. Thanks for your work! --- - To

[GitHub] spark pull request #18659: [SPARK-21190][PYSPARK][WIP] Simple Python Vectori...

2017-09-06 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/18659#discussion_r137402913 --- Diff: python/pyspark/sql/functions.py --- @@ -2112,7 +2113,7 @@ def wrapper(*args): @since(1.3) -def udf(f=None,

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137403025 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -2346,6 +2347,45 @@ abstract class DDLSuite extends

[GitHub] spark issue #19046: [SPARK-18769][yarn] Limit resource requests based on RM'...

2017-09-06 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19046 I was told that the preemption issue was fixed in YARN (that's YARN-6210); so I don't think there's a need for this code currently (just use a fixed YARN if you want reliable preemption).

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137401845 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/SaveAsHiveFile.scala --- @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #18659: [SPARK-21190][PYSPARK][WIP] Simple Python Vectorized UDF...

2017-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18659 **[Test build #81480 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81480/testReport)** for PR 18659 at commit

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-06 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137401720 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/SaveAsHiveFile.scala --- @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-06 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137401187 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -234,12 +82,8 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137401217 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/SaveAsHiveFile.scala --- @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #18659: [SPARK-21190][PYSPARK][WIP] Simple Python Vectorized UDF...

2017-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18659 **[Test build #81479 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81479/testReport)** for PR 18659 at commit

[GitHub] spark issue #19149: [SPARK-21652][SQL][FOLLOW-UP] Fix rule confliction betwe...

2017-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19149 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81476/ Test FAILed. ---

[GitHub] spark issue #19149: [SPARK-21652][SQL][FOLLOW-UP] Fix rule confliction betwe...

2017-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19149 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19149: [SPARK-21652][SQL][FOLLOW-UP] Fix rule confliction betwe...

2017-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19149 **[Test build #81476 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81476/testReport)** for PR 19149 at commit

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137399745 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -234,12 +82,8 @@ case class InsertIntoHiveTable(

[GitHub] spark issue #19046: [SPARK-18769][yarn] Limit resource requests based on RM'...

2017-09-06 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/19046 It might help if you can give an exact scenario where you see the issue and perhaps configs if those matter. meaning do you have like some set of minimum containers, a small idle timeout, etc..

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-06 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137399408 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveTmpPath.scala --- @@ -0,0 +1,204 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-06 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19140 LGTM pending tests. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137398964 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveTmpPath.scala --- @@ -0,0 +1,204 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-06 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137398917 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -360,6 +360,27 @@ case class

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-06 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137398869 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -360,6 +360,27 @@ case class

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-06 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137398776 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveTmpPath.scala --- @@ -0,0 +1,204 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-06 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137398299 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -178,11 +179,60 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark issue #19149: [SPARK-21652][SQL][FOLLOW-UP] Fix rule confliction betwe...

2017-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19149 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19149: [SPARK-21652][SQL][FOLLOW-UP] Fix rule confliction betwe...

2017-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19149 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81475/ Test FAILed. ---

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-06 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137397811 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -178,11 +179,60 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark issue #19149: [SPARK-21652][SQL][FOLLOW-UP] Fix rule confliction betwe...

2017-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19149 **[Test build #81475 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81475/testReport)** for PR 19149 at commit

[GitHub] spark issue #19046: [SPARK-18769][yarn] Limit resource requests based on RM'...

2017-09-06 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/19046 Unfortunately that isn't clear to me as to the cause or what the issue is with the yarn side. I'm not sure what he means by "AM is releasing and then acquiring the reservations again and

[GitHub] spark issue #18659: [SPARK-21190][PYSPARK][WIP] Simple Python Vectorized UDF...

2017-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18659 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81478/ Test FAILed. ---

[GitHub] spark issue #18659: [SPARK-21190][PYSPARK][WIP] Simple Python Vectorized UDF...

2017-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18659 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18659: [SPARK-21190][PYSPARK][WIP] Simple Python Vectorized UDF...

2017-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18659 **[Test build #81478 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81478/testReport)** for PR 18659 at commit

[GitHub] spark issue #19135: [SPARK-21923][CORE]Avoid call reserveUnrollMemoryForThis...

2017-09-06 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/19135 It would be great to test the perf on executors with various corsPerExecutor settings to ensure we don't bring in regressions by the code change. ---

[GitHub] spark issue #18659: [SPARK-21404][PYSPARK][WIP] Simple Python Vectorized UDF...

2017-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18659 **[Test build #81478 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81478/testReport)** for PR 18659 at commit

[GitHub] spark issue #19046: [SPARK-18769][yarn] Limit resource requests based on RM'...

2017-09-06 Thread Tagar
Github user Tagar commented on the issue: https://github.com/apache/spark/pull/19046 @tgravescs, here's quote from Wilfred Spiegelenburg - hope it answers both of your questions. > The behaviour I discussed earlier around the Spark AM reservations is not optimal. It turns

[GitHub] spark issue #19046: [SPARK-18769][yarn] Limit resource requests based on RM'...

2017-09-06 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/19046 @Tagar can you be more specific about the problems you are seeing? how does this affect preemption? Why don't you see the same issues on MapReduce/Tez? ---

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137390245 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -201,13 +201,14 @@ case class

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137388410 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -360,6 +360,27 @@ case class

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137388263 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -360,6 +360,27 @@ case class

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137387725 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveTmpPath.scala --- @@ -0,0 +1,204 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137387842 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveTmpPath.scala --- @@ -0,0 +1,204 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137386317 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -178,11 +179,60 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137385849 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -178,11 +179,60 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18975 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18975 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81468/ Test PASSed. ---

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18975 **[Test build #81468 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81468/testReport)** for PR 18975 at commit

[GitHub] spark issue #19086: [SPARK-21874][SQL] Support changing database when rename...

2017-09-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19086 After more discussion with others, this behavior change really bothers us. Both changes do not sound good to us. Where is the requirement from? ---

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137383267 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -201,13 +201,14 @@ case class AlterTableAddColumnsCommand(

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19140 **[Test build #81477 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81477/testReport)** for PR 19140 at commit

[GitHub] spark pull request #19083: [SPARK-21871][SQL] Check actual bytecode size whe...

2017-09-06 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19083#discussion_r137378351 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CodeGenerationSuite.scala --- @@ -98,19 +99,23 @@ class CodeGenerationSuite

[GitHub] spark issue #17435: [SPARK-20098][PYSPARK] dataType's typeName fix

2017-09-06 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/17435 Gentle ping (going through old PySpark PRs) :) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19149: [SPARK-21652][SQL][FOLLOW-UP] Fix rule confliction betwe...

2017-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19149 **[Test build #81476 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81476/testReport)** for PR 19149 at commit

[GitHub] spark issue #19149: [SPARK-21652][SQL][FOLLOW-UP] Fix rule confliction betwe...

2017-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19149 **[Test build #81475 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81475/testReport)** for PR 19149 at commit

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19140 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19140 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81474/ Test FAILed. ---

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19140 **[Test build #81474 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81474/testReport)** for PR 19140 at commit

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19140 **[Test build #81474 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81474/testReport)** for PR 19140 at commit

[GitHub] spark issue #19149: [SPARK-21652][SQL][FOLLOW-UP] Fix rule confliction betwe...

2017-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19149 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19149: [SPARK-21652][SQL][FOLLOW-UP] Fix rule confliction betwe...

2017-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19149 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81470/ Test FAILed. ---

[GitHub] spark issue #19149: [SPARK-21652][SQL][FOLLOW-UP] Fix rule confliction betwe...

2017-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19149 **[Test build #81470 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81470/testReport)** for PR 19149 at commit

[GitHub] spark issue #17096: [SPARK-15243][ML][SQL][PYTHON] Add missing support for u...

2017-09-06 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/17096 Sorry for the delay. Lets get Jenkins to retest this and make sure everything is ok but it looks like a good change :) LGTM pending jenkins/merge issues (if they show up during jenkins).

[GitHub] spark issue #19149: [SPARK-21652][SQL][FOLLOW-UP] Fix rule confliction betwe...

2017-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19149 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19149: [SPARK-21652][SQL][FOLLOW-UP] Fix rule confliction betwe...

2017-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19149 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81469/ Test FAILed. ---

[GitHub] spark issue #19149: [SPARK-21652][SQL][FOLLOW-UP] Fix rule confliction betwe...

2017-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19149 **[Test build #81469 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81469/testReport)** for PR 19149 at commit

[GitHub] spark pull request #17096: [SPARK-15243][ML][SQL][PYTHON] Add missing suppor...

2017-09-06 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/17096#discussion_r137369131 --- Diff: python/pyspark/ml/tests.py --- @@ -319,6 +320,20 @@ def test_hasparam(self): testParams = TestParams()

[GitHub] spark issue #18826: LogisticRegressionModel.toString should summarize model

2017-09-06 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/18826 On the Python side using __repr__ looks reasonable (although I would like to see a doctest for this). But we really should get @jkbradley or maybe @dbtsai to take a look on the ML side.

[GitHub] spark issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2017-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18339 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2017-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18339 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81472/ Test PASSed. ---

[GitHub] spark issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2017-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18339 **[Test build #81472 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81472/testReport)** for PR 18339 at commit

[GitHub] spark issue #17451: [SPARK-19866][ML][PySpark] Add local version of Word2Vec...

2017-09-06 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/17451 Sorry for the slow reply, I left a note with how to get at the tuples contents (the _call_java is behaving weirdly I'm not sure why its shoving the tuple into a string). ---

[GitHub] spark pull request #17451: [SPARK-19866][ML][PySpark] Add local version of W...

2017-09-06 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/17451#discussion_r137368179 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Word2Vec.scala --- @@ -274,6 +274,29 @@ class Word2VecModel private[ml] (

[GitHub] spark issue #19046: [SPARK-18769][yarn] Limit resource requests based on RM'...

2017-09-06 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19046 I'm in general not a fan of adding more and more config options; the end result is that most people won't enable it until they run into problems, and to me that's too late. I'm still

[GitHub] spark issue #18515: [SPARK-21287][SQL] Ability to use Integer.MIN_VALUE as a...

2017-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18515 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81467/ Test PASSed. ---

[GitHub] spark issue #18515: [SPARK-21287][SQL] Ability to use Integer.MIN_VALUE as a...

2017-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18515 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #12646: [SPARK-14878][SQL] Trim characters string function suppo...

2017-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12646 **[Test build #81473 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81473/testReport)** for PR 12646 at commit

[GitHub] spark issue #18515: [SPARK-21287][SQL] Ability to use Integer.MIN_VALUE as a...

2017-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18515 **[Test build #81467 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81467/testReport)** for PR 18515 at commit

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-09-06 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r137366326 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -503,69 +504,319 @@ case class

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-09-06 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r137366302 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -503,69 +504,319 @@ case class

[GitHub] spark issue #19136: [DO NOT MERGE][SPARK-15689][SQL] data source v2

2017-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19136 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81466/ Test PASSed. ---

[GitHub] spark issue #19136: [DO NOT MERGE][SPARK-15689][SQL] data source v2

2017-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19136 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19046: [SPARK-18769][yarn] Limit resource requests based on RM'...

2017-09-06 Thread Tagar
Github user Tagar commented on the issue: https://github.com/apache/spark/pull/19046 > This could just be an adhoc queue but the spark users would lose out to tez/mapreduce users. I'm pretty positive this will hurt spark users on some of our cluster so would want performance numbers

[GitHub] spark issue #19136: [DO NOT MERGE][SPARK-15689][SQL] data source v2

2017-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19136 **[Test build #81466 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81466/testReport)** for PR 19136 at commit

<    1   2   3   4   5   >