[GitHub] spark pull request #19155: [MINOR][TEST] Tables created in unit tests should...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19155#discussion_r137650844 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala --- @@ -659,62 +659,67 @@ class HiveDDLSuite }

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19129 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19129 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81523/ Test PASSed. ---

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19129 **[Test build #81523 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81523/testReport)** for PR 19129 at commit

[GitHub] spark issue #19130: [SPARK-21917][CORE][YARN] Supporting adding http(s) reso...

2017-09-07 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/19130 sorry probably wont' get to this today, will look tomorrow. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #19155: [MINOR][TEST] Tables created in unit tests should be dro...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19155 Since it's a big patch, `+408 −359`, we had better have a JIRA issue with 'TRIVIAL` priority. --- - To unsubscribe,

[GitHub] spark pull request #19155: [MINOR][TEST] Tables created in unit tests should...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19155#discussion_r137638893 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala --- @@ -1603,7 +1617,7 @@ class SQLQuerySuite extends

[GitHub] spark pull request #19153: [SPARK-21941] Stop storing unused attemptId in SQ...

2017-09-07 Thread ajbozarth
Github user ajbozarth commented on a diff in the pull request: https://github.com/apache/spark/pull/19153#discussion_r137637831 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLListener.scala --- @@ -478,10 +476,11 @@ private[ui] class SQLStageMetrics(

[GitHub] spark pull request #19155: [MINOR][TEST] Tables created in unit tests should...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19155#discussion_r137637931 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveQuerySuite.scala --- @@ -370,21 +370,24 @@ class HiveQuerySuite extends

[GitHub] spark pull request #19155: [MINOR][TEST] Tables created in unit tests should...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19155#discussion_r137637228 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala --- @@ -349,21 +351,23 @@ class SQLQuerySuite extends

[GitHub] spark pull request #19155: [MINOR][TEST] Tables created in unit tests should...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19155#discussion_r137637074 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala --- @@ -116,69 +116,71 @@ class SQLQuerySuite extends

[GitHub] spark pull request #19155: [MINOR][TEST] Tables created in unit tests should...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19155#discussion_r137636117 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertIntoHiveTableSuite.scala --- @@ -461,19 +471,23 @@ class InsertIntoHiveTableSuite

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-07 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/16578 @snir It's hard to pin down exactly where the problem is and how to fix it. We removed the `NullIntolerant` trait from the implementations of the `ExtractValue` trait in

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-07 Thread snir
Github user snir commented on the issue: https://github.com/apache/spark/pull/16578 @mallman Yes, the issue is only reproducible with codegen on. However the performance improvement by codegen is too significant to disable. I managed to fix it with some changes to the codegen (Added

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18975 **[Test build #81528 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81528/testReport)** for PR 18975 at commit

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137629284 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala --- @@ -534,4 +534,150 @@ class InsertIntoHiveTableSuite extends QueryTest

[GitHub] spark pull request #18941: [SPARK-21715][WebUI] History Server should not re...

2017-09-07 Thread zhouyejoe
Github user zhouyejoe closed the pull request at: https://github.com/apache/spark/pull/18941 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #17589: [SPARK-16544][SQL] Support for conversion from numeric c...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17589 **[Test build #81527 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81527/testReport)** for PR 17589 at commit

[GitHub] spark issue #19157: [SPARK-20589][Core][Scheduler] Allow limiting task concu...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19157 **[Test build #81526 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81526/testReport)** for PR 19157 at commit

[GitHub] spark issue #19157: [SPARK-20589][Core][Scheduler] Allow limiting task concu...

2017-09-07 Thread dhruve
Github user dhruve commented on the issue: https://github.com/apache/spark/pull/19157 Reopened this because CI was having issues with the previous PR. [18950](https://github.com/apache/spark/pull/18950) --- - To

[GitHub] spark issue #19157: [SPARK-20589][Core][Scheduler] Allow limiting task concu...

2017-09-07 Thread dhruve
Github user dhruve commented on the issue: https://github.com/apache/spark/pull/19157 @squito @markhamstra @tgravescs Can you review this PR. Thanks. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19157: [SPARK-20589][Core][Scheduler] Allow limiting tas...

2017-09-07 Thread dhruve
GitHub user dhruve opened a pull request: https://github.com/apache/spark/pull/19157 [SPARK-20589][Core][Scheduler] Allow limiting task concurrency per job group ## What changes were proposed in this pull request? This change allows the user to specify the maximum no. of tasks

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19129 **[Test build #81525 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81525/testReport)** for PR 19129 at commit

[GitHub] spark issue #18950: [SPARK-20589][Core][Scheduler] Allow limiting task concu...

2017-09-07 Thread dhruve
Github user dhruve commented on the issue: https://github.com/apache/spark/pull/18950 CI is having issues downloading my repo. Closing this PR and opening a new one. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #18950: [SPARK-20589][Core][Scheduler] Allow limiting tas...

2017-09-07 Thread dhruve
Github user dhruve closed the pull request at: https://github.com/apache/spark/pull/18950 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cache...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19129#discussion_r137622298 --- Diff: docs/sql-programming-guide.md --- @@ -1587,6 +1580,10 @@ options. Note that this is different from the Hive behavior. -

[GitHub] spark pull request #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cache...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19129#discussion_r137621847 --- Diff: docs/sql-programming-guide.md --- @@ -1587,6 +1580,10 @@ options. Note that this is different from the Hive behavior. -

[GitHub] spark pull request #19148: [SPARK-21936][SQL][WIP] backward compatibility te...

2017-09-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19148#discussion_r137621549 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala --- @@ -0,0 +1,193 @@ +/* + * Licensed to the

[GitHub] spark pull request #19148: [SPARK-21936][SQL][WIP] backward compatibility te...

2017-09-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19148#discussion_r137621065 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogBackwardCompatibilitySuite.scala --- @@ -1,260 +0,0 @@ -/* - *

[GitHub] spark pull request #19148: [SPARK-21936][SQL][WIP] backward compatibility te...

2017-09-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19148#discussion_r137621130 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/MetastoreDataSourcesSuite.scala --- @@ -1354,31 +1354,4 @@ class MetastoreDataSourcesSuite

[GitHub] spark pull request #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cache...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19129#discussion_r137621009 --- Diff: docs/sql-programming-guide.md --- @@ -1587,6 +1580,10 @@ options. Note that this is different from the Hive behavior. -

[GitHub] spark pull request #19148: [SPARK-21936][SQL][WIP] backward compatibility te...

2017-09-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19148#discussion_r137620814 --- Diff: sql/hive/pom.xml --- @@ -177,6 +177,10 @@ libfb303 + org.apache.derby + derby --- End diff

[GitHub] spark pull request #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cache...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19129#discussion_r137620763 --- Diff: docs/sql-programming-guide.md --- @@ -1587,6 +1580,10 @@ options. Note that this is different from the Hive behavior. -

[GitHub] spark pull request #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cache...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19129#discussion_r137620489 --- Diff: docs/sql-programming-guide.md --- @@ -1587,6 +1580,10 @@ options. Note that this is different from the Hive behavior. -

[GitHub] spark issue #19148: [SPARK-21936][SQL][WIP] backward compatibility test fram...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19148 **[Test build #81524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81524/testReport)** for PR 19148 at commit

[GitHub] spark pull request #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cache...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19129#discussion_r137619510 --- Diff: docs/sql-programming-guide.md --- @@ -1587,6 +1580,10 @@ options. Note that this is different from the Hive behavior. - As

[GitHub] spark pull request #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cache...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19129#discussion_r137619452 --- Diff: docs/sql-programming-guide.md --- @@ -1587,6 +1580,10 @@ options. Note that this is different from the Hive behavior. -

[GitHub] spark pull request #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cache...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19129#discussion_r137619209 --- Diff: docs/sql-programming-guide.md --- @@ -1587,6 +1580,10 @@ options. Note that this is different from the Hive behavior. - As

[GitHub] spark pull request #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cache...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19129#discussion_r137617487 --- Diff: docs/sql-programming-guide.md --- @@ -1587,6 +1580,10 @@ options. Note that this is different from the Hive behavior. -

[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data

2017-09-07 Thread erikerlandson
Github user erikerlandson commented on the issue: https://github.com/apache/spark/pull/19041 I have been thinking about a different but related [use case](https://github.com/apache-spark-on-k8s/spark/issues/261); supporting the ability to operate in dynamic allocation mode without

[GitHub] spark pull request #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cache...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19129#discussion_r137616632 --- Diff: docs/sql-programming-guide.md --- @@ -1587,6 +1580,10 @@ options. Note that this is different from the Hive behavior. - As

[GitHub] spark pull request #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_...

2017-09-07 Thread parente
Github user parente commented on a diff in the pull request: https://github.com/apache/spark/pull/18339#discussion_r137616450 --- Diff: python/pyspark/java_gateway.py --- @@ -33,14 +33,18 @@ from pyspark.serializers import read_int -def

[GitHub] spark pull request #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_...

2017-09-07 Thread parente
Github user parente commented on a diff in the pull request: https://github.com/apache/spark/pull/18339#discussion_r137616353 --- Diff: python/pyspark/java_gateway.py --- @@ -33,14 +33,18 @@ from pyspark.serializers import read_int -def

[GitHub] spark issue #19148: [SPARK-21936][SQL][WIP] backward compatibility test fram...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19148 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19148: [SPARK-21936][SQL][WIP] backward compatibility test fram...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19148 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81520/ Test FAILed. ---

[GitHub] spark issue #19148: [SPARK-21936][SQL][WIP] backward compatibility test fram...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19148 **[Test build #81520 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81520/testReport)** for PR 19148 at commit

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19129 **[Test build #81523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81523/testReport)** for PR 19129 at commit

[GitHub] spark pull request #18956: [SPARK-21726][SQL] Check for structural integrity...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18956#discussion_r137613115 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/rules/RuleExecutor.scala --- @@ -64,6 +64,14 @@ abstract class

[GitHub] spark pull request #18956: [SPARK-21726][SQL] Check for structural integrity...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18956#discussion_r137611075 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/rules/RuleExecutor.scala --- @@ -93,6 +101,13 @@ abstract class

[GitHub] spark pull request #18956: [SPARK-21726][SQL] Check for structural integrity...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18956#discussion_r137610235 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/OptimizerSICheckerSuite.scala --- @@ -0,0 +1,60 @@ +/* + *

[GitHub] spark pull request #18956: [SPARK-21726][SQL] Check for structural integrity...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18956#discussion_r137609405 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/rules/RuleExecutor.scala --- @@ -93,6 +101,13 @@ abstract class

[GitHub] spark issue #19147: [WIP][SPARK-21190][SQL][PYTHON] Vectorized UDFs in Pytho...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19147 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81519/ Test FAILed. ---

[GitHub] spark issue #19147: [WIP][SPARK-21190][SQL][PYTHON] Vectorized UDFs in Pytho...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19147 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137607929 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/InsertIntoDataSourceDirCommand.scala --- @@ -0,0 +1,81 @@ +/* + *

[GitHub] spark issue #19147: [WIP][SPARK-21190][SQL][PYTHON] Vectorized UDFs in Pytho...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19147 **[Test build #81519 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81519/testReport)** for PR 19147 at commit

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137607749 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/InsertIntoDataSourceDirCommand.scala --- @@ -0,0 +1,81 @@ +/* + *

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137607545 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveDirCommand.scala --- @@ -0,0 +1,138 @@ +/* + * Licensed to

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137607475 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveDirCommand.scala --- @@ -0,0 +1,138 @@ +/* + * Licensed to

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19129 The PR title resolved two issues under title `[SPARK-15639][SPARK-16321][SQL] Push down filter at RowGroups level for parquet reader` I'll add like the following. Is it enough? ``` -

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137606401 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveDirCommand.scala --- @@ -0,0 +1,138 @@ +/* + * Licensed to

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137605940 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveTmpPath.scala --- @@ -0,0 +1,202 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137605602 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/sources/InsertSuite.scala --- @@ -366,4 +367,43 @@ class InsertSuite extends DataSourceTest with

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137605509 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -140,6 +143,14 @@ case class

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137605311 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLParserSuite.scala --- @@ -524,6 +525,50 @@ class DDLParserSuite extends

[GitHub] spark issue #18956: [SPARK-21726][SQL] Check for structural integrity of the...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18956 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18956: [SPARK-21726][SQL] Check for structural integrity of the...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18956 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81518/ Test PASSed. ---

[GitHub] spark issue #18956: [SPARK-21726][SQL] Check for structural integrity of the...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18956 **[Test build #81518 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81518/testReport)** for PR 18956 at commit

[GitHub] spark pull request #19156: [SPARK-19634][FOLLOW-UP][ML] Improve interface of...

2017-09-07 Thread thunterdb
Github user thunterdb commented on a diff in the pull request: https://github.com/apache/spark/pull/19156#discussion_r137603986 --- Diff: mllib/src/main/scala/org/apache/spark/ml/stat/Summarizer.scala --- @@ -109,31 +108,47 @@ object Summarizer extends Logging { }

[GitHub] spark issue #18266: [SPARK-20427][SQL] Read JDBC table use custom schema

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18266 will review this today. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19129 Thank you! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19103: [SPARK-21890] Credentials not being passed to add the to...

2017-09-07 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19103 Looks good (since the master PR didn't merge). Merging to 2.2. @redsanket please close the PR manually. --- - To unsubscribe,

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19129 ` SQL 1.6 to 2.0` sounds good to me. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #14151: [SPARK-16496][SQL] Add wholetext as option for reading t...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14151 cc @viirya --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137601757 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLParserSuite.scala --- @@ -524,6 +525,50 @@ class DDLParserSuite extends

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137601572 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLParserSuite.scala --- @@ -524,6 +525,50 @@ class DDLParserSuite extends

[GitHub] spark pull request #14151: [SPARK-16496][SQL] Add wholetext as option for re...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14151#discussion_r137601363 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/text/TextFileFormat.scala --- @@ -101,10 +111,24 @@ class TextFileFormat

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137601380 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLParserSuite.scala --- @@ -32,7 +32,8 @@ import

[GitHub] spark pull request #14151: [SPARK-16496][SQL] Add wholetext as option for re...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14151#discussion_r137601163 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/text/TextFileFormat.scala --- @@ -101,10 +111,24 @@ class TextFileFormat

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19129 Or, should I made `Upgrading From Spark SQL 2.2 to 2.3`? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #14151: [SPARK-16496][SQL] Add wholetext as option for re...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14151#discussion_r137601004 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/text/TextFileFormat.scala --- @@ -99,8 +100,22 @@ class TextFileFormat

[GitHub] spark pull request #14151: [SPARK-16496][SQL] Add wholetext as option for re...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14151#discussion_r137600781 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/text/TextFileFormat.scala --- @@ -53,6 +57,12 @@ class TextFileFormat

[GitHub] spark issue #17451: [SPARK-19866][ML][PySpark] Add local version of Word2Vec...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17451 **[Test build #81521 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81521/testReport)** for PR 17451 at commit

[GitHub] spark issue #17451: [SPARK-19866][ML][PySpark] Add local version of Word2Vec...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17451 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #17451: [SPARK-19866][ML][PySpark] Add local version of Word2Vec...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17451 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81521/ Test PASSed. ---

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19129 It's marked as 2.0.1 and 2.1.0 with the following commit logs. ``` ~/s/spark-2.2:branch-2.0$ git log --oneline | grep SPARK-15639 977fbbfcae [SPARK-15639] [SPARK-16321] [SQL] Push

[GitHub] spark pull request #19136: [DO NOT MERGE][SPARK-15689][SQL] data source v2

2017-09-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/19136#discussion_r137597910 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -0,0 +1,39 @@ +/* + * Licensed to

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19129 Sure, @gatorsmile . BTW, I searched more and updated the PR description. It's [SPARK-15639](https://github.com/apache/spark/pull/13701) ---

[GitHub] spark pull request #14151: [SPARK-16496][SQL] Add wholetext as option for re...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14151#discussion_r137595908 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/text/TextFileFormat.scala --- @@ -101,10 +111,24 @@ class TextFileFormat

[GitHub] spark pull request #14151: [SPARK-16496][SQL] Add wholetext as option for re...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14151#discussion_r137595716 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/text/TextSuite.scala --- @@ -185,8 +233,7 @@ class TextSuite extends

[GitHub] spark pull request #19136: [DO NOT MERGE][SPARK-15689][SQL] data source v2

2017-09-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19136#discussion_r137595695 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -0,0 +1,39 @@ +/* + * Licensed

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19129 Please document it in the migration guides. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19103: [SPARK-21890] Credentials not being passed to add the to...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19103 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19103: [SPARK-21890] Credentials not being passed to add the to...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19103 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81522/ Test PASSed. ---

[GitHub] spark issue #19103: [SPARK-21890] Credentials not being passed to add the to...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19103 **[Test build #81522 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81522/testReport)** for PR 19103 at commit

[GitHub] spark pull request #19136: [DO NOT MERGE][SPARK-15689][SQL] data source v2

2017-09-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19136#discussion_r137593873 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/ReadTask.java --- @@ -0,0 +1,43 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #19136: [DO NOT MERGE][SPARK-15689][SQL] data source v2

2017-09-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/19136#discussion_r137591425 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -0,0 +1,39 @@ +/* + * Licensed to

[GitHub] spark issue #19103: [SPARK-21890] Credentials not being passed to add the to...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19103 **[Test build #81522 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81522/testReport)** for PR 19103 at commit

[GitHub] spark pull request #19140: [SPARK-21890] Credentials not being passed to add...

2017-09-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19140 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19103: [SPARK-21890] Credentials not being passed to add the to...

2017-09-07 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/19103 Jenkins,test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-07 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/19140 didn't merge to branch 2.2, will handle under https://github.com/apache/spark/pull/19103 --- - To unsubscribe, e-mail:

<    1   2   3   4   >