[GitHub] spark issue #15708: [SPARK-18167] [SQL] Retry when the SQLQuerySuite test fl...

2016-10-31 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15708 Merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #15707: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15707 **[Test build #3384 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3384/consoleFull)** for PR 15707 at commit

[GitHub] spark issue #15702: [SPARK-18124] Observed-delay based Event Time Watermarks

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15702 **[Test build #67866 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67866/consoleFull)** for PR 15702 at commit

[GitHub] spark pull request #15709: [SPARK-18190][Build][SparkR] Fix R version to not...

2016-10-31 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/15709 [SPARK-18190][Build][SparkR] Fix R version to not the latest in AppVeyor ## What changes were proposed in this pull request? Currently, Spark supports the test on Windows via AppVeyor

[GitHub] spark issue #15707: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15707 **[Test build #3386 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3386/consoleFull)** for PR 15707 at commit

[GitHub] spark issue #15707: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15707 **[Test build #67865 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67865/consoleFull)** for PR 15707 at commit

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15626 **[Test build #67845 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67845/consoleFull)** for PR 15626 at commit

[GitHub] spark issue #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15704 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67857/ Test PASSed. ---

[GitHub] spark issue #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15704 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15709: [SPARK-18190][Build][SparkR] Fix R version to not the la...

2016-10-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15709 Maybe, I might create a JIRA to describe the known issues with AppVeyor. (does it make sense?) --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION sho...

2016-10-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15704#discussion_r85873381 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -631,6 +631,42 @@ private[spark] class HiveExternalCatalog(conf:

[GitHub] spark issue #15688: [SPARK-18173][SQL] data source tables should support tru...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15688 **[Test build #67871 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67871/consoleFull)** for PR 15688 at commit

[GitHub] spark pull request #15705: [SPARK-18183] [SPARK-18184] Fix INSERT [INTO|OVER...

2016-10-31 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/15705#discussion_r85875944 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/CatalogFileIndex.scala --- @@ -67,7 +67,10 @@ class CatalogFileIndex(

[GitHub] spark issue #15710: [SPARK-18025] Use commit protocol API in structured stre...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15710 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15710: [SPARK-18025] Use commit protocol API in structured stre...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15710 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67872/ Test FAILed. ---

[GitHub] spark pull request #15710: [SPARK-18025] Use commit protocol API in structur...

2016-10-31 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15710#discussion_r85878038 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala --- @@ -84,17 +84,22 @@ case class

[GitHub] spark issue #15705: [SPARK-18183] [SPARK-18184] Fix INSERT [INTO|OVERWRITE] ...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15705 **[Test build #67846 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67846/consoleFull)** for PR 15705 at commit

[GitHub] spark issue #15705: [SPARK-18183] [SPARK-18184] Fix INSERT [INTO|OVERWRITE] ...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15705 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67846/ Test FAILed. ---

[GitHub] spark issue #15705: [SPARK-18183] [SPARK-18184] Fix INSERT [INTO|OVERWRITE] ...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15705 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15626 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15706: [SPARK-18189] [Core] Fix serialization issue in KeyValue...

2016-10-31 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15706 This guy @transient must get a lot of emails ... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #15706: [SPARK-18189] [Core] Fix serialization issue in KeyValue...

2016-10-31 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15706 Can you change the title to "[SQL]" rather than "[core]"? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #15707: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15707 **[Test build #67855 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67855/consoleFull)** for PR 15707 at commit

[GitHub] spark issue #15708: [SPARK-18167] [SQL] Retry when the SQLQuerySuite test fl...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15708 **[Test build #67854 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67854/consoleFull)** for PR 15708 at commit

[GitHub] spark pull request #15708: [SPARK-18167] [SQL] Retry when the SQLQuerySuite ...

2016-10-31 Thread ericl
GitHub user ericl opened a pull request: https://github.com/apache/spark/pull/15708 [SPARK-18167] [SQL] Retry when the SQLQuerySuite test flakes ## What changes were proposed in this pull request? This will re-run the flaky test a few times after it fails. This will help

[GitHub] spark issue #14547: [SPARK-16718][MLlib] gbm-style treeboost

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14547 **[Test build #67858 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67858/consoleFull)** for PR 14547 at commit

[GitHub] spark issue #14547: [SPARK-16718][MLlib] gbm-style treeboost

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14547 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67858/ Test FAILed. ---

[GitHub] spark issue #14547: [SPARK-16718][MLlib] gbm-style treeboost

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14547 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15633: [SPARK-18087] [SQL] Optimize insert to not requir...

2016-10-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15633 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15693: [SPARK-18125][SQL] Fix a compilation error in codegen du...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15693 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15693: [SPARK-18125][SQL] Fix a compilation error in codegen du...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15693 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67850/ Test PASSed. ---

[GitHub] spark issue #15693: [SPARK-18125][SQL] Fix a compilation error in codegen du...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15693 **[Test build #67850 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67850/consoleFull)** for PR 15693 at commit

[GitHub] spark issue #15708: [SPARK-18167] [SQL] Retry when the SQLQuerySuite test fl...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15708 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67854/ Test PASSed. ---

[GitHub] spark issue #15694: [SPARK-18179][SQL] Throws analysis exception with a prop...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15694 **[Test build #67856 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67856/consoleFull)** for PR 15694 at commit

[GitHub] spark issue #15706: [SPARK-18189] [SQL] Fix serialization issue in KeyValueG...

2016-10-31 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15706 Actually is there a test case you can create for this? Make sure the test case would fail when the patch is not applied. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #15706: [SPARK-18189] [SQL] Fix serialization issue in KeyValueG...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15706 **[Test build #3385 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3385/consoleFull)** for PR 15706 at commit

[GitHub] spark issue #15697: [SparkR][Test]:remove unnecessary suppressWarnings

2016-10-31 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/15697 @HyukjinKwon Thanks for your time! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15709: [SPARK-18190][Build][SparkR] Fix R version to not the la...

2016-10-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15709 (cc @nchammas too who I also guess met this issue first. I would like to let him know.) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #15666: [SPARK-11421] [Core][Python][R] Added ability for addJar...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15666 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15666: [SPARK-11421] [Core][Python][R] Added ability for addJar...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15666 **[Test build #67860 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67860/consoleFull)** for PR 15666 at commit

[GitHub] spark pull request #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pys...

2016-10-31 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/13599#discussion_r85859191 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonWorkerFactory.scala --- @@ -55,6 +65,11 @@ private[spark] class

[GitHub] spark pull request #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pys...

2016-10-31 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/13599#discussion_r85860115 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonWorkerFactory.scala --- @@ -69,6 +84,66 @@ private[spark] class

[GitHub] spark pull request #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pys...

2016-10-31 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/13599#discussion_r85859768 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonWorkerFactory.scala --- @@ -69,6 +84,67 @@ private[spark] class

[GitHub] spark issue #15673: [SPARK-17992][SQL] Return all partitions from HiveShim w...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15673 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67859/ Test PASSed. ---

[GitHub] spark issue #15673: [SPARK-17992][SQL] Return all partitions from HiveShim w...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15673 Build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15707: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15707 Looks like the test failed due to a flaky test, but other than that everything else was fine. I'm going to merge this optimistically. --- If your project is set up for it, you can reply to this

[GitHub] spark issue #15710: [SPARK-18025] Use commit protocol API in structured stre...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15710 **[Test build #67874 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67874/consoleFull)** for PR 15710 at commit

[GitHub] spark pull request #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pys...

2016-10-31 Thread zjffdu
Github user zjffdu commented on a diff in the pull request: https://github.com/apache/spark/pull/13599#discussion_r85877935 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonWorkerFactory.scala --- @@ -69,6 +84,66 @@ private[spark] class

[GitHub] spark issue #15710: [SPARK-18025] Use commit protocol API in structured stre...

2016-10-31 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15710 cc @ericl, @marmbrus, @zsxwing and @lw-lin (I guess this would supersede your old PR). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #14444: [SPARK-16839] [SQL] redundant aliases after cleanupAlias...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/1 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15708: [SPARK-18167] [SQL] Retry when the SQLQuerySuite test fl...

2016-10-31 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/15708 Ok interesting, it seems the test repeatedly fails. This probably means it is a suite ordering issue. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15626 **[Test build #67829 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67829/consoleFull)** for PR 15626 at commit

[GitHub] spark issue #15699: [SPARK-18030][Tests]Fix flaky FileStreamSourceSuite by n...

2016-10-31 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15699 cc @marmbrus --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #15699: [SPARK-18030][Tests]Fix flaky FileStreamSourceSui...

2016-10-31 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/15699 [SPARK-18030][Tests]Fix flaky FileStreamSourceSuite by not deleting the files ## What changes were proposed in this pull request? The test `when schema inference is turned on, should read

[GitHub] spark issue #15695: [SPARK-18143][SQL]Ignore Structured Streaming event logs...

2016-10-31 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15695 Fixing the flaky test in #15699. Retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #15699: [SPARK-18030][Tests]Fix flaky FileStreamSourceSuite by n...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15699 **[Test build #67830 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67830/consoleFull)** for PR 15699 at commit

[GitHub] spark pull request #15670: [SPARK-18161] [Python] Allow pickle to serialize ...

2016-10-31 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/15670#discussion_r85826175 --- Diff: python/pyspark/serializers.py --- @@ -64,7 +64,7 @@ from itertools import izip as zip else: import pickle -protocol =

[GitHub] spark issue #15651: [SPARK-17972][SQL] Add Dataset.checkpoint() to truncate ...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15651 **[Test build #67802 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67802/consoleFull)** for PR 15651 at commit

[GitHub] spark issue #15691: [SPARK-18175] [SQL] Improve the test case coverage of im...

2016-10-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15691 @rxin @cloud-fan @hvanhovell @tejasapatil @srowen @HyukjinKwon This PR is now ready to review. This matrix can help us understand whether the implicit casting has any bug. If any change is made

[GitHub] spark issue #14649: [SPARK-17059][SQL] Allow FileFormat to specify partition...

2016-10-31 Thread andreweduffy
Github user andreweduffy commented on the issue: https://github.com/apache/spark/pull/14649 Hey @paulata, sounds like you want to make the pruner pluggable at runtime. That sounds like it would be good, though IMO is a bit orthogonal to this change. This is more of an optimization

[GitHub] spark issue #15651: [SPARK-17972][SQL] Add Dataset.checkpoint() to truncate ...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15651 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15651: [SPARK-17972][SQL] Add Dataset.checkpoint() to truncate ...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15651 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67802/ Test PASSed. ---

[GitHub] spark issue #15651: [SPARK-17972][SQL] Add Dataset.checkpoint() to truncate ...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15651 **[Test build #67802 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67802/consoleFull)** for PR 15651 at commit

[GitHub] spark issue #15669: [SPARK-18160][CORE][YARN] SparkContext.addFile doesn't w...

2016-10-31 Thread zjffdu
Github user zjffdu commented on the issue: https://github.com/apache/spark/pull/15669 @jerryshao spark.files is always passed to driver so SparkContext.addFile is called in yarn-cluster.

[GitHub] spark issue #15692: [SPARK-18177][ML][PYSPARK] Add missing 'subsamplingRate'...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15692 **[Test build #67805 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67805/consoleFull)** for PR 15692 at commit

[GitHub] spark issue #14640: [SPARK-17055] [MLLIB] add groupKFold to CrossValidator

2016-10-31 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14640 OK let's close this one for now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15672: [SPARK-18148][SQL] Misleading Error Message for Aggregat...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15672 **[Test build #67803 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67803/consoleFull)** for PR 15672 at commit

[GitHub] spark issue #15616: [SPARK-16827][Shuffle] add disk spill bytes to UnsafeExt...

2016-10-31 Thread dreamworks007
Github user dreamworks007 commented on the issue: https://github.com/apache/spark/pull/15616 @joshrosen - could you take a look ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15672: [SPARK-18148][SQL] Misleading Error Message for Aggregat...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15672 **[Test build #67803 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67803/consoleFull)** for PR 15672 at commit

[GitHub] spark pull request #15692: [SPARK-18177][ML][PYSPARK] Add missing 'subsampli...

2016-10-31 Thread zhengruifeng
GitHub user zhengruifeng opened a pull request: https://github.com/apache/spark/pull/15692 [SPARK-18177][ML][PYSPARK] Add missing 'subsamplingRate' of pyspark GBTClassifier ## What changes were proposed in this pull request? Add missing 'subsamplingRate' of pyspark

[GitHub] spark issue #15690: [SPARK-18174] [SQL] [DO-NOT-MERGE] Avoid Implicit Type C...

2016-10-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15690 @rxin A good idea! Then, users can turn it off! Will submit a separate PR for it. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request #15663: [SPARK-18143][SQL]Ignore Structured Streaming eve...

2016-10-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15663 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #15651: [SPARK-17972][SQL] Add Dataset.checkpoint() to tr...

2016-10-31 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/15651#discussion_r8567 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ExistingRDD.scala --- @@ -130,17 +130,40 @@ case class ExternalRDDScanExec[T]( /**

[GitHub] spark issue #15651: [SPARK-17972][SQL] Add Dataset.checkpoint() to truncate ...

2016-10-31 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15651 LGTM except for one minor comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15691: [SPARK-18175] [SQL] Improve the test case coverage of im...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15691 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67801/ Test PASSed. ---

[GitHub] spark issue #15691: [SPARK-18175] [SQL] Improve the test case coverage of im...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15691 **[Test build #67801 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67801/consoleFull)** for PR 15691 at commit

[GitHub] spark issue #15691: [SPARK-18175] [SQL] Improve the test case coverage of im...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15691 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15667: [SPARK-18107][SQL] Insert overwrite statement runs much ...

2016-10-31 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15667 @snodawn Thanks! I will address dynamic partition in next commit. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #15672: [SPARK-18148][SQL] Misleading Error Message for Aggregat...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15672 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15672: [SPARK-18148][SQL] Misleading Error Message for Aggregat...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15672 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67803/ Test PASSed. ---

[GitHub] spark issue #15667: [SPARK-18107][SQL] Insert overwrite statement runs much ...

2016-10-31 Thread snodawn
Github user snodawn commented on the issue: https://github.com/apache/spark/pull/15667 @viirya I have tested the newest patch. It performs good in running the same sql as I ran before. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #15011: [SPARK-17122][SQL]support drop current database

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15011 **[Test build #67804 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67804/consoleFull)** for PR 15011 at commit

[GitHub] spark issue #15692: [SPARK-18177][ML][PYSPARK] Add missing 'subsamplingRate'...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15692 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15692: [SPARK-18177][ML][PYSPARK] Add missing 'subsamplingRate'...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15692 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67805/ Test PASSed. ---

[GitHub] spark issue #15686: [MINOR][DOC] Remove spaces following slashs

2016-10-31 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15686 This change seems fine. If Appveyor is encountering errors regularly, we might consider disabling it, just because it wouldn't be giving useful information if most failures were spurious.

[GitHub] spark issue #15603: [WEBUI][MINOR] Remove unused appUIAddress in SparkUI + c...

2016-10-31 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15603 Since this turned into a bit more of a change than where it started, and because it removes a method (internal, defunct as it may be), and since this isn't a hurry, would you open a JIRA just to

[GitHub] spark pull request #15417: [SPARK-17851][SQL][TESTS] Make sure all test sqls...

2016-10-31 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/15417#discussion_r85692528 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/SimplifyCastsSuite.scala --- @@ -44,7 +44,9 @@ class SimplifyCastsSuite extends

[GitHub] spark pull request #15417: [SPARK-17851][SQL][TESTS] Make sure all test sqls...

2016-10-31 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/15417#discussion_r85694427 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/FilterPushdownSuite.scala --- @@ -1016,6 +1016,8 @@ class

[GitHub] spark issue #15669: [SPARK-18160][CORE][YARN] SparkContext.addFile doesn't w...

2016-10-31 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/15669 From the code what I can is that `spark.files` will not be set in yarn mode. ```scala OptionAssigner(args.files, LOCAL | STANDALONE | MESOS, ALL_DEPLOY_MODES,

[GitHub] spark issue #15538: [SPARK-17993][SQL] Fix Parquet log output redirection

2016-10-31 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15538 If this is necessary, then isn't it simpler to leave the log configuration call where it was, so that it doesn't depend on the constructor? that wasn't the actual problem was it, just the logging?

[GitHub] spark pull request #15651: [SPARK-17972][SQL] Add Dataset.checkpoint() to tr...

2016-10-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15651#discussion_r85690320 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -483,6 +484,60 @@ class Dataset[T] private[sql]( def isStreaming: Boolean =

[GitHub] spark issue #15672: [SPARK-18148][SQL] Misleading Error Message for Aggregat...

2016-10-31 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/15672 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15663: [SPARK-18143][SQL]Ignore Structured Streaming event logs...

2016-10-31 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/15663 Okay, I merged only to master, this does not merge cleanly to 2.0. I am on vacation so i cant test out the 2.0 fix quickly, better make a separate PR for 2.0 --- If your project is set up for it,

[GitHub] spark issue #15637: [SPARK-18000] [SQL] Aggregation function for computing e...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15637 **[Test build #67806 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67806/consoleFull)** for PR 15637 at commit

[GitHub] spark issue #15692: [SPARK-18177][ML][PYSPARK] Add missing 'subsamplingRate'...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15692 **[Test build #67805 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67805/consoleFull)** for PR 15692 at commit

[GitHub] spark pull request #15693: [SPARK-18125][SQL] Fix a compilation error in cod...

2016-10-31 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/15693 [SPARK-18125][SQL] Fix a compilation error in codegen due to splitExpression ## What changes were proposed in this pull request? As reported in the jira, sometimes the generated java code

[GitHub] spark issue #15011: [SPARK-17122][SQL]support drop current database

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15011 **[Test build #67804 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67804/consoleFull)** for PR 15011 at commit

[GitHub] spark issue #15011: [SPARK-17122][SQL]support drop current database

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15011 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67804/ Test PASSed. ---

[GitHub] spark issue #15011: [SPARK-17122][SQL]support drop current database

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15011 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15693: [SPARK-18125][SQL] Fix a compilation error in codegen du...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15693 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

<    1   2   3   4   5   6   >