[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15626 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67822/ Test FAILed. ---

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15626 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15696 **[Test build #67818 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67818/consoleFull)** for PR 15696 at commit [`2a61351`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15696 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15696 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67818/ Test PASSed. ---

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15626 **[Test build #67829 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67829/consoleFull)** for PR 15626 at commit [`7beedcc`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #15699: [SPARK-18030][Tests]Fix flaky FileStreamSourceSuite by n...

2016-10-31 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15699 cc @marmbrus --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #15699: [SPARK-18030][Tests]Fix flaky FileStreamSourceSui...

2016-10-31 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/15699 [SPARK-18030][Tests]Fix flaky FileStreamSourceSuite by not deleting the files ## What changes were proposed in this pull request? The test `when schema inference is turned on, should read

[GitHub] spark issue #15695: [SPARK-18143][SQL]Ignore Structured Streaming event logs...

2016-10-31 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15695 Fixing the flaky test in #15699. Retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #15699: [SPARK-18030][Tests]Fix flaky FileStreamSourceSuite by n...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15699 **[Test build #67830 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67830/consoleFull)** for PR 15699 at commit [`64040ee`](https://github.com/apache/spark/commit/6

[GitHub] spark pull request #15670: [SPARK-18161] [Python] Allow pickle to serialize ...

2016-10-31 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/15670#discussion_r85826175 --- Diff: python/pyspark/serializers.py --- @@ -64,7 +64,7 @@ from itertools import izip as zip else: import pickle -protocol =

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15626 **[Test build #67831 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67831/consoleFull)** for PR 15626 at commit [`d6fec94`](https://github.com/apache/spark/commit/d

[GitHub] spark pull request #15675: [SPARK-18144][SQL] logging StreamingQueryListener...

2016-10-31 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15675#discussion_r85827662 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingQueryListenerBus.scala --- @@ -41,7 +41,9 @@ class StreamingQueryListene

[GitHub] spark issue #15695: [SPARK-18143][SQL]Ignore Structured Streaming event logs...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15695 **[Test build #3380 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3380/consoleFull)** for PR 15695 at commit [`0d4461a`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15651: [SPARK-17972][SQL] Add Dataset.checkpoint() to tr...

2016-10-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15651 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #15697: [SparkR][Test]:remove unnecessary suppressWarnings

2016-10-31 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/15697 retest it please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #15673: [SPARK-17992][SQL] Return all partitions from HiveShim w...

2016-10-31 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/15673 @ericl I've pushed a commit with the changes you recommended. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #15673: [SPARK-17992][SQL] Return all partitions from HiveShim w...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15673 **[Test build #67834 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67834/consoleFull)** for PR 15673 at commit [`4c438c8`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15696 **[Test build #67832 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67832/consoleFull)** for PR 15696 at commit [`6af14b5`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #15675: [SPARK-18144][SQL] logging StreamingQueryListener$QueryS...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15675 **[Test build #67833 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67833/consoleFull)** for PR 15675 at commit [`8d2b3a6`](https://github.com/apache/spark/commit/8

[GitHub] spark pull request #15666: [SPARK-11421] [Core][Python][R] Added ability for...

2016-10-31 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15666#discussion_r85829640 --- Diff: R/pkg/R/context.R --- @@ -290,6 +290,33 @@ spark.addFile <- function(path, recursive = FALSE) { invisible(callJMethod(sc, "addFile", sup

[GitHub] spark pull request #15666: [SPARK-11421] [Core][Python][R] Added ability for...

2016-10-31 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15666#discussion_r85829905 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1700,19 +1700,34 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request #15666: [SPARK-11421] [Core][Python][R] Added ability for...

2016-10-31 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15666#discussion_r85830311 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1700,19 +1700,34 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request #15675: [SPARK-18144][SQL] logging StreamingQueryListener...

2016-10-31 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15675#discussion_r85830450 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingQueryListenerBus.scala --- @@ -41,6 +41,8 @@ class StreamingQueryListene

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15696 **[Test build #67835 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67835/consoleFull)** for PR 15696 at commit [`6166093`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #14653: [SPARK-10931][PYSPARK][ML] PySpark ML Models should cont...

2016-10-31 Thread evanyc15
Github user evanyc15 commented on the issue: https://github.com/apache/spark/pull/14653 Hey @jkbradley the checkParams method already exists in the Python side. It's defined in the tests.py DefaultValuesTests class and is being called by test_java_params. I'm removing the param testin

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11105 **[Test build #67825 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67825/consoleFull)** for PR 11105 at commit [`8c560ca`](https://github.com/apache/spark/commit/

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11105 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67825/ Test PASSed. ---

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11105 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15538: [SPARK-17993][SQL] Fix Parquet log output redirection

2016-10-31 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/15538 In fact, if no one else is working on the Parquet upgrade it probably makes more sense for me to contribute that then continue working on this PR. I'll check with the dev mailing list. --- If your

[GitHub] spark issue #15608: [SPARK-17838][SparkR] Check named arguments for options ...

2016-10-31 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15608 LGTM. I ran through a few other cases and I think the omitted names are handled properly with this. This should go to master then (handledCallJMethod is not in branch-2.0) --- If your proj

[GitHub] spark pull request #15700: [SPARK-17964][SparkR] Enable SparkR with Mesos cl...

2016-10-31 Thread susanxhuynh
GitHub user susanxhuynh opened a pull request: https://github.com/apache/spark/pull/15700 [SPARK-17964][SparkR] Enable SparkR with Mesos client mode and cluster mode ## What changes were proposed in this pull request? Enabled SparkR with Mesos client mode and cluster mode. J

[GitHub] spark pull request #15666: [SPARK-11421] [Core][Python][R] Added ability for...

2016-10-31 Thread mariusvniekerk
Github user mariusvniekerk commented on a diff in the pull request: https://github.com/apache/spark/pull/15666#discussion_r85833766 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1700,19 +1700,34 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark issue #15700: [SPARK-17964][SparkR] Enable SparkR with Mesos client mo...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15700 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #15688: [SPARK-18173][SQL] data source tables should supp...

2016-10-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15688#discussion_r85834452 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -1636,15 +1635,23 @@ class DDLSuite extends QueryTest with

[GitHub] spark issue #15688: [SPARK-18173][SQL] data source tables should support tru...

2016-10-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15688 If we try to truncate a partition that does not exist, we just do nothing? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15659 **[Test build #67824 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67824/consoleFull)** for PR 15659 at commit [`3bf961e`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15659 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15659 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67824/ Test PASSed. ---

[GitHub] spark issue #15698: [SPARK-18182] Expose ReplayListenerBus.read() overload w...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15698 **[Test build #67828 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67828/consoleFull)** for PR 15698 at commit [`b777ee5`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15698: [SPARK-18182] Expose ReplayListenerBus.read() overload w...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15698 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15698: [SPARK-18182] Expose ReplayListenerBus.read() overload w...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15698 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67828/ Test FAILed. ---

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

2016-10-31 Thread nchammas
Github user nchammas commented on the issue: https://github.com/apache/spark/pull/15659 We have an AppVeyor build now? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #15701: [SPARK-18167] [SQL] Also log all partitions when ...

2016-10-31 Thread ericl
GitHub user ericl opened a pull request: https://github.com/apache/spark/pull/15701 [SPARK-18167] [SQL] Also log all partitions when the SQLQuerySuite test flakes ## What changes were proposed in this pull request? One possibility for this test flaking is that we have corru

[GitHub] spark issue #15701: [SPARK-18167] [SQL] Also log all partitions when the SQL...

2016-10-31 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/15701 @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feat

[GitHub] spark issue #15701: [SPARK-18167] [SQL] Also log all partitions when the SQL...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15701 **[Test build #67836 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67836/consoleFull)** for PR 15701 at commit [`767fef8`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15696 **[Test build #67832 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67832/consoleFull)** for PR 15696 at commit [`6af14b5`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15696 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67832/ Test FAILed. ---

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15696 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #14444: [SPARK-16839] [SQL] redundant aliases after clean...

2016-10-31 Thread eyalfa
Github user eyalfa commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r85839282 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeColumnCommand.scala --- @@ -137,7 +137,7 @@ object ColumnStatStruct {

[GitHub] spark issue #14444: [SPARK-16839] [SQL] redundant aliases after cleanupAlias...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/1 **[Test build #67837 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67837/consoleFull)** for PR 1 at commit [`b397d04`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #14444: [SPARK-16839] [SQL] redundant aliases after cleanupAlias...

2016-10-31 Thread eyalfa
Github user eyalfa commented on the issue: https://github.com/apache/spark/pull/1 @hvanhovell, can you please relaunch the build? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15626 **[Test build #67829 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67829/consoleFull)** for PR 15626 at commit [`7beedcc`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15696 **[Test build #67835 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67835/consoleFull)** for PR 15696 at commit [`6166093`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15626 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15696 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67835/ Test FAILed. ---

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15626 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67829/ Test FAILed. ---

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15696 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15669: [SPARK-18160][CORE][YARN] SparkContext.addFile doesn't w...

2016-10-31 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/15669 If I understand the issue here, the problem is not `--files` but setting `spark.files` through `--conf` (or the config file). The former is translated into `spark.yarn.dist.files` by `SparkSubmit` bu

[GitHub] spark issue #15398: [SPARK-17647][SQL] Fix backslash escaping in 'LIKE' patt...

2016-10-31 Thread jodersky
Github user jodersky commented on the issue: https://github.com/apache/spark/pull/15398 @hvanhovell, I just wanted to add that option 2 from my last comment won't work: if we replace `\\` with `` in the unescape function (as is done for `\%` and `\_`), it will be impossible for a

[GitHub] spark issue #15701: [SPARK-18167] [SQL] Also log all partitions when the SQL...

2016-10-31 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/15701 lgtm --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15696 **[Test build #67838 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67838/consoleFull)** for PR 15696 at commit [`040bbba`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #15699: [SPARK-18030][Tests]Fix flaky FileStreamSourceSuite by n...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15699 **[Test build #67830 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67830/consoleFull)** for PR 15699 at commit [`64040ee`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15702: [SPARK-18124] Observed-delay based Even Time Wate...

2016-10-31 Thread marmbrus
GitHub user marmbrus opened a pull request: https://github.com/apache/spark/pull/15702 [SPARK-18124] Observed-delay based Even Time Watermarks This PR adds a new method `withWatermark` to the `Dataset` API, which can be used specify an _event time watermark_. An event time watermar

[GitHub] spark issue #15702: [SPARK-18124] Observed-delay based Even Time Watermarks

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15702 **[Test build #67839 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67839/consoleFull)** for PR 15702 at commit [`5b92132`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #15699: [SPARK-18030][Tests]Fix flaky FileStreamSourceSuite by n...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15699 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67830/ Test PASSed. ---

[GitHub] spark issue #15699: [SPARK-18030][Tests]Fix flaky FileStreamSourceSuite by n...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15699 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #15702: [SPARK-18124] Observed-delay based Even Time Wate...

2016-10-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/15702#discussion_r85845880 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/EventTimeWatermarkExec.scala --- @@ -0,0 +1,93 @@ +/* + * Licensed to th

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15696 **[Test build #67840 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67840/consoleFull)** for PR 15696 at commit [`51d0919`](https://github.com/apache/spark/commit/5

[GitHub] spark pull request #15696: [SPARK-18024][SQL] Introduce an internal commit p...

2016-10-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/15696#discussion_r85847969 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/sources/SimpleTextRelation.scala --- @@ -141,15 +139,14 @@ class SimpleTextOutputWriter( }

[GitHub] spark pull request #15696: [SPARK-18024][SQL] Introduce an internal commit p...

2016-10-31 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15696#discussion_r85848150 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/sources/SimpleTextRelation.scala --- @@ -141,15 +139,14 @@ class SimpleTextOutputWriter( } }

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

2016-10-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15659 @nchammas Ah, yes it is. It is a known problem. AppVeyor triggers the build when it has the changes in `R` directory but when it's merged (instead of rebased) for example, https://github.com/ap

[GitHub] spark pull request #15696: [SPARK-18024][SQL] Introduce an internal commit p...

2016-10-31 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15696#discussion_r85848357 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/sources/SimpleTextRelation.scala --- @@ -141,15 +139,14 @@ class SimpleTextOutputWriter( } }

[GitHub] spark pull request #15694: [SPARK-18179][SQL] Throws analysis exception with...

2016-10-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15694#discussion_r85848749 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/MiscFunctionsSuite.scala --- @@ -31,6 +34,47 @@ class MiscFunctionsSuite extends QueryTest with S

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15696 **[Test build #67841 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67841/consoleFull)** for PR 15696 at commit [`2d7d373`](https://github.com/apache/spark/commit/2

[GitHub] spark pull request #15703: [SPARK-18186] Migrate HiveUDAFFunction to TypedIm...

2016-10-31 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/15703 [SPARK-18186] Migrate HiveUDAFFunction to TypedImperativeAggregate for partial aggregation support ## What changes were proposed in this pull request? This PR migrates `HiveUDAFFunction`

[GitHub] spark issue #15703: [SPARK-18186] Migrate HiveUDAFFunction to TypedImperativ...

2016-10-31 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/15703 Will add more details in the PR description soon. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #15673: [SPARK-17992][SQL] Return all partitions from HiveShim w...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15673 **[Test build #67834 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67834/consoleFull)** for PR 15673 at commit [`4c438c8`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15673: [SPARK-17992][SQL] Return all partitions from HiveShim w...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15673 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67834/ Test FAILed. ---

[GitHub] spark issue #15673: [SPARK-17992][SQL] Return all partitions from HiveShim w...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15673 Build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15703: [SPARK-18186] Migrate HiveUDAFFunction to TypedImperativ...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15703 **[Test build #67842 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67842/consoleFull)** for PR 15703 at commit [`c0029f1`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #15677: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

2016-10-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15677 Okay, @gatorsmile could you maybe submit a small one first please? Will refer yours and then I can work together (we can divide the files or packages to deal with I guess). --- If your project

[GitHub] spark issue #15695: [SPARK-18143][SQL]Ignore Structured Streaming event logs...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15695 **[Test build #3380 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3380/consoleFull)** for PR 15695 at commit [`0d4461a`](https://github.com/apache/spark/commit

[GitHub] spark issue #15673: [SPARK-17992][SQL] Return all partitions from HiveShim w...

2016-10-31 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/15673 It looks like all the unit tests passed, however one of the forked test java processes exited with nonzero status for some unknown reason. --- If your project is set up for it, you can reply to thi

[GitHub] spark pull request #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION sho...

2016-10-31 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/15704 [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should support comparators ## What changes were proposed in this pull request? This PR aims to support `comparators`, e.g. '<', '<=', '>'

[GitHub] spark issue #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15704 **[Test build #67843 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67843/consoleFull)** for PR 15704 at commit [`84f2315`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #15675: [SPARK-18144][SQL] logging StreamingQueryListener$QueryS...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15675 **[Test build #67833 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67833/consoleFull)** for PR 15675 at commit [`8d2b3a6`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15695: [SPARK-18143][SQL]Ignore Structured Streaming event logs...

2016-10-31 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15695 Merging in branch-2.0. Can you close this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature ena

[GitHub] spark pull request #15172: [SPARK-13331] AES support for over-the-wire encry...

2016-10-31 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15172#discussion_r85851023 --- Diff: docs/configuration.md --- @@ -1435,6 +1435,21 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark pull request #15172: [SPARK-13331] AES support for over-the-wire encry...

2016-10-31 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15172#discussion_r85845790 --- Diff: common/network-common/src/main/java/org/apache/spark/network/sasl/SaslClientBootstrap.java --- @@ -88,9 +90,25 @@ public void doBootstrap(Transport

[GitHub] spark pull request #15172: [SPARK-13331] AES support for over-the-wire encry...

2016-10-31 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15172#discussion_r85850499 --- Diff: common/network-common/src/main/java/org/apache/spark/network/sasl/aes/AesCipher.java --- @@ -0,0 +1,311 @@ +/* + * Licensed to the Apache S

[GitHub] spark pull request #15172: [SPARK-13331] AES support for over-the-wire encry...

2016-10-31 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15172#discussion_r85848147 --- Diff: common/network-common/src/main/java/org/apache/spark/network/sasl/aes/AesCipher.java --- @@ -0,0 +1,311 @@ +/* + * Licensed to the Apache S

[GitHub] spark pull request #15172: [SPARK-13331] AES support for over-the-wire encry...

2016-10-31 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15172#discussion_r85849868 --- Diff: common/network-common/src/main/java/org/apache/spark/network/util/ByteArrayReadableChannel.java --- @@ -0,0 +1,71 @@ +/* + * Licensed to th

[GitHub] spark pull request #15172: [SPARK-13331] AES support for over-the-wire encry...

2016-10-31 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15172#discussion_r85848236 --- Diff: common/network-common/src/main/java/org/apache/spark/network/sasl/aes/AesCipher.java --- @@ -0,0 +1,311 @@ +/* + * Licensed to the Apache S

[GitHub] spark pull request #15172: [SPARK-13331] AES support for over-the-wire encry...

2016-10-31 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15172#discussion_r85847230 --- Diff: common/network-common/src/main/java/org/apache/spark/network/sasl/aes/AesCipher.java --- @@ -0,0 +1,311 @@ +/* + * Licensed to the Apache S

[GitHub] spark issue #15695: [SPARK-18143][SQL]Ignore Structured Streaming event logs...

2016-10-31 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15695 Sure. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feat

[GitHub] spark pull request #15172: [SPARK-13331] AES support for over-the-wire encry...

2016-10-31 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15172#discussion_r85849098 --- Diff: common/network-common/src/main/java/org/apache/spark/network/sasl/aes/AesCipher.java --- @@ -0,0 +1,311 @@ +/* + * Licensed to the Apache S

[GitHub] spark pull request #15172: [SPARK-13331] AES support for over-the-wire encry...

2016-10-31 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15172#discussion_r85847599 --- Diff: common/network-common/src/main/java/org/apache/spark/network/sasl/aes/AesCipher.java --- @@ -0,0 +1,311 @@ +/* + * Licensed to the Apache S

[GitHub] spark pull request #15172: [SPARK-13331] AES support for over-the-wire encry...

2016-10-31 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15172#discussion_r85849759 --- Diff: common/network-common/src/main/java/org/apache/spark/network/sasl/aes/AesConfigMessage.java --- @@ -0,0 +1,104 @@ +/* + * Licensed to the A

[GitHub] spark issue #15675: [SPARK-18144][SQL] logging StreamingQueryListener$QueryS...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15675 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67833/ Test PASSed. ---

<    1   2   3   4   5   6   7   >