[GitHub] spark issue #14803: [SPARK-17153][SQL] Should read partition data when readi...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14803 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14803: [SPARK-17153][SQL] Should read partition data when readi...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14803 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65812/ Test PASSed. ---

[GitHub] spark pull request #15205: [SPARK-16240][ML] ML persistence backward compati...

2016-09-22 Thread jkbradley
Github user jkbradley closed the pull request at: https://github.com/apache/spark/pull/15205 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14803: [SPARK-17153][SQL] Should read partition data when readi...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14803 **[Test build #65812 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65812/consoleFull)** for PR 14803 at commit

[GitHub] spark issue #10212: [SPARK-12221] add cpu time to metrics

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/10212 **[Test build #65815 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65815/consoleFull)** for PR 10212 at commit

[GitHub] spark issue #15041: [SPARK-17488][CORE] TakeAndOrder will OOM when the data ...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15041 **[Test build #65814 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65814/consoleFull)** for PR 15041 at commit

[GitHub] spark pull request #10212: [SPARK-12221] add cpu time to metrics

2016-09-22 Thread jisookim0513
Github user jisookim0513 commented on a diff in the pull request: https://github.com/apache/spark/pull/10212#discussion_r80184799 --- Diff: core/src/test/scala/org/apache/spark/util/JsonProtocolSuite.scala --- @@ -1097,7 +1100,9 @@ private[spark] object JsonProtocolSuite extends

[GitHub] spark pull request #10212: [SPARK-12221] add cpu time to metrics

2016-09-22 Thread jisookim0513
Github user jisookim0513 commented on a diff in the pull request: https://github.com/apache/spark/pull/10212#discussion_r80184744 --- Diff: core/src/test/resources/HistoryServerExpectations/complete_stage_list_json_expectation.json --- @@ -6,6 +6,7 @@ "numCompleteTasks" :

[GitHub] spark issue #15205: [SPARK-16240][ML] ML persistence backward compatibility ...

2016-09-22 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/15205 Local testing worked, so merging with branch-2.0 now --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #15102: [SPARK-17346][SQL] Add Kafka source for Structured Strea...

2016-09-22 Thread jwbear
Github user jwbear commented on the issue: https://github.com/apache/spark/pull/15102 Just curious looking at this, if you are comparing "sequential" offsets across partitions a rebalance would definitely affect this and, unless something has changed, it probably not a good idea to

[GitHub] spark issue #14897: [SPARK-17338][SQL] add global temp view

2016-09-22 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14897 We also need to update [the Analyzer rule

[GitHub] spark pull request #14359: [SPARK-16719][ML] Random Forests should communica...

2016-09-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14359 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14359: [SPARK-16719][ML] Random Forests should communicate fewe...

2016-09-22 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/14359 Merging with master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80183259 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -188,6 +199,10 @@ class SessionCatalog(

[GitHub] spark issue #15210: [SPARK-17604][SQL][Streaming] Supprt purging aged file e...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15210 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65811/ Test PASSed. ---

[GitHub] spark issue #15210: [SPARK-17604][SQL][Streaming] Supprt purging aged file e...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15210 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14698: [SPARK-17061][SPARK-17093][SQL] `MapObjects` shou...

2016-09-22 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/14698#discussion_r80182811 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -136,7 +136,7 @@ trait

[GitHub] spark issue #15210: [SPARK-17604][SQL][Streaming] Supprt purging aged file e...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15210 **[Test build #65811 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65811/consoleFull)** for PR 15210 at commit

[GitHub] spark issue #15172: [SPARK-13331] AES support for over-the-wire encryption

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15172 **[Test build #65813 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65813/consoleFull)** for PR 15172 at commit

[GitHub] spark issue #15090: [SPARK-17073] [SQL] generate column-level statistics

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15090 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15090: [SPARK-17073] [SQL] generate column-level statistics

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15090 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65810/ Test PASSed. ---

[GitHub] spark issue #15090: [SPARK-17073] [SQL] generate column-level statistics

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15090 **[Test build #65810 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65810/consoleFull)** for PR 15090 at commit

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80180856 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -36,6 +36,9 @@ import

[GitHub] spark pull request #15209: replace function type with function isinstance

2016-09-22 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15209#discussion_r80180841 --- Diff: python/pyspark/ml/linalg/__init__.py --- @@ -101,7 +101,7 @@ def _vector_size(v): return len(v) elif type(v) in

[GitHub] spark issue #15172: [SPARK-13331] AES support for over-the-wire encryption

2016-09-22 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/15172 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #15204: [SPARK-17639][build] Add jce.jar to buildclasspat...

2016-09-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15204 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15204: [SPARK-17639][build] Add jce.jar to buildclasspath when ...

2016-09-22 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/15204 Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #15209: replace function type with function isinstance

2016-09-22 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15209 I think we need a JIRA because `type` and `isinstance` are not exactly same. Also, maybe it'd better if the PR descriptions explains the bug and how this PR tries to resolve it. BTW,

[GitHub] spark issue #15204: [SPARK-17639][build] Add jce.jar to buildclasspath when ...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15204 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15204: [SPARK-17639][build] Add jce.jar to buildclasspath when ...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15204 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65808/ Test PASSed. ---

[GitHub] spark issue #15204: [SPARK-17639][build] Add jce.jar to buildclasspath when ...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15204 **[Test build #65808 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65808/consoleFull)** for PR 15204 at commit

[GitHub] spark issue #15200: Skip building R vignettes if Spark is not built

2016-09-22 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15200 we could hand write the result instead of running test/Spark to generate the vignettes? It seems that could be problematic if output are getting out of sync - and similar problem if we

[GitHub] spark issue #14818: [SPARK-17157][SPARKR][WIP]: Add multiclass logistic regr...

2016-09-22 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/14818 `glm` has a `link=logit` parameter? not sure if it maps to this http://www.statmethods.net/advstats/glm.html --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #15082: [SPARK-17528][SQL] MutableProjection should not cache co...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15082 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65807/ Test PASSed. ---

[GitHub] spark issue #15082: [SPARK-17528][SQL] MutableProjection should not cache co...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15082 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15082: [SPARK-17528][SQL] MutableProjection should not cache co...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15082 **[Test build #65807 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65807/consoleFull)** for PR 15082 at commit

[GitHub] spark issue #15207: [SPARK-17643] Remove comparable requirement from Offset

2016-09-22 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/15207 LGTM. You probably already checked this, but FWIW I verified the kafka topic deletion test does pass once this is merged:

[GitHub] spark issue #14803: [SPARK-17153][SQL] Should read partition data when readi...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14803 **[Test build #65812 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65812/consoleFull)** for PR 14803 at commit

[GitHub] spark issue #15208: [SPARK-17641][SQL] Collect_list/Collect_set should not c...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15208 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65806/ Test PASSed. ---

[GitHub] spark issue #15208: [SPARK-17641][SQL] Collect_list/Collect_set should not c...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15208 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14803: [SPARK-17153][SQL] Should read partition data when readi...

2016-09-22 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14803 > * What error is printed (if any) if an invalid partition directory is created midstream. The error is: [info] org.apache.spark.sql.streaming.StreamingQueryException:

[GitHub] spark issue #15208: [SPARK-17641][SQL] Collect_list/Collect_set should not c...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15208 **[Test build #65806 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65806/consoleFull)** for PR 15208 at commit

[GitHub] spark pull request #15195: [SPARK-17632][SQL]make console sink and other sin...

2016-09-22 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/15195#discussion_r80178129 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala --- @@ -290,8 +284,8 @@ final class DataStreamWriter[T]

[GitHub] spark issue #14644: [SPARK-14082][MESOS] Enable GPU support with Mesos

2016-09-22 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/14644 @klueska Just updated the patch and I think it's using the right semantics now, where it has a global gpus max just like cores. Can you try it out? --- If your project is set up for it, you can

[GitHub] spark issue #15207: [SPARK-17643] Remove comparable requirement from Offset

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15207 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65805/ Test PASSed. ---

[GitHub] spark issue #15207: [SPARK-17643] Remove comparable requirement from Offset

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15207 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15207: [SPARK-17643] Remove comparable requirement from Offset

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15207 **[Test build #65805 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65805/consoleFull)** for PR 15207 at commit

[GitHub] spark issue #15159: [SPARK-17605][SPARK_SUBMIT] Add option spark.usePython a...

2016-09-22 Thread zjffdu
Github user zjffdu commented on the issue: https://github.com/apache/spark/pull/15159 Add @rxin @davies @JoshRosen @shivaram for more feedback. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #15205: [SPARK-16240][ML] ML persistence backward compatibility ...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15205 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15205: [SPARK-16240][ML] ML persistence backward compatibility ...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15205 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65804/ Test PASSed. ---

[GitHub] spark issue #15205: [SPARK-16240][ML] ML persistence backward compatibility ...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15205 **[Test build #65804 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65804/consoleFull)** for PR 15205 at commit

[GitHub] spark issue #15210: [SPARK-17604][SQL][Streaming] Supprt purging aged file e...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15210 **[Test build #65811 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65811/consoleFull)** for PR 15210 at commit

[GitHub] spark pull request #15210: [SPARK-17604][SQL][Streaming] Supprt purging aged...

2016-09-22 Thread jerryshao
GitHub user jerryshao opened a pull request: https://github.com/apache/spark/pull/15210 [SPARK-17604][SQL][Streaming] Supprt purging aged file entries in FileStreamSourceLog ## What changes were proposed in this pull request? Currently with

[GitHub] spark issue #15209: replace function type with function isinstance

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15209 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #15209: replace function type with function isinstance

2016-09-22 Thread frankfqchen
GitHub user frankfqchen opened a pull request: https://github.com/apache/spark/pull/15209 replace function type with function isinstance ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch

[GitHub] spark issue #15204: [SPARK-17639][build] Add jce.jar to buildclasspath when ...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15204 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65802/ Test PASSed. ---

[GitHub] spark pull request #15089: [SPARK-15621] [SQL] Support spilling for Python U...

2016-09-22 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15089#discussion_r80174300 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/RowQueue.scala --- @@ -0,0 +1,278 @@ +/* +* Licensed to the Apache Software

[GitHub] spark issue #15204: [SPARK-17639][build] Add jce.jar to buildclasspath when ...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15204 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15204: [SPARK-17639][build] Add jce.jar to buildclasspath when ...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15204 **[Test build #65802 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65802/consoleFull)** for PR 15204 at commit

[GitHub] spark issue #15090: [SPARK-17073] [SQL] generate column-level statistics

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15090 **[Test build #65810 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65810/consoleFull)** for PR 15090 at commit

[GitHub] spark issue #15206: [SPARK-17640][SQL]Avoid using -1 as the default batchId ...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15206 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65803/ Test PASSed. ---

[GitHub] spark issue #15206: [SPARK-17640][SQL]Avoid using -1 as the default batchId ...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15206 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15206: [SPARK-17640][SQL]Avoid using -1 as the default batchId ...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15206 **[Test build #65803 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65803/consoleFull)** for PR 15206 at commit

[GitHub] spark issue #15102: [SPARK-17346][SQL] Add Kafka source for Structured Strea...

2016-09-22 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/15102 > I agree that if/when we add that ability to add existing partitions midstream we'd probably need to add two offsets in to the SQL offset for new partitions. It's not just existing

[GitHub] spark issue #15041: [SPARK-17488][CORE] TakeAndOrder will OOM when the data ...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15041 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15041: [SPARK-17488][CORE] TakeAndOrder will OOM when the data ...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15041 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65809/ Test FAILed. ---

[GitHub] spark issue #15041: [SPARK-17488][CORE] TakeAndOrder will OOM when the data ...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15041 **[Test build #65809 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65809/consoleFull)** for PR 15041 at commit

[GitHub] spark issue #15199: [SPARK-17635][SQL] Remove hardcode "agg_plan" in HashAgg...

2016-09-22 Thread yucai
Github user yucai commented on the issue: https://github.com/apache/spark/pull/15199 Thanks all, it should be fixed in master only, my mistake. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #15041: [SPARK-17488][CORE] TakeAndOrder will OOM when the data ...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15041 **[Test build #65809 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65809/consoleFull)** for PR 15041 at commit

[GitHub] spark issue #15204: [SPARK-17639][build] Add jce.jar to buildclasspath when ...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15204 **[Test build #65808 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65808/consoleFull)** for PR 15204 at commit

[GitHub] spark issue #15102: [SPARK-17346][SQL] Add Kafka source for Structured Strea...

2016-09-22 Thread marmbrus
Github user marmbrus commented on the issue: https://github.com/apache/spark/pull/15102 > "I want to be able to add a topicpartition mid stream, but I don't want to start it from the beginning." I see, I was thinking only of new topics that appear that match your pattern. I

[GitHub] spark issue #15082: [SPARK-17528][SQL] MutableProjection should not cache co...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15082 **[Test build #65807 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65807/consoleFull)** for PR 15082 at commit

[GitHub] spark issue #14659: [SPARK-16757] Set up Spark caller context to HDFS and YA...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14659 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14659: [SPARK-16757] Set up Spark caller context to HDFS and YA...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14659 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65799/ Test PASSed. ---

[GitHub] spark issue #14659: [SPARK-16757] Set up Spark caller context to HDFS and YA...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14659 **[Test build #65799 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65799/consoleFull)** for PR 14659 at commit

[GitHub] spark pull request #15174: [SPARK-17502] [17609] [SQL] [Backport] [2.0] Fix ...

2016-09-22 Thread gatorsmile
Github user gatorsmile closed the pull request at: https://github.com/apache/spark/pull/15174 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #15174: [SPARK-17502] [17609] [SQL] [Backport] [2.0] Fix Multipl...

2016-09-22 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15174 Let me close it. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15102: [SPARK-17346][SQL] Add Kafka source for Structured Strea...

2016-09-22 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/15102 @tdas I think as long as marmbrus' PR to remove comparable from the interface works for sane variations of subscription changes it's the best way to go. I'm honestly fine with someone getting

[GitHub] spark issue #15206: [SPARK-17640][SQL]Avoid using -1 as the default batchId ...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15206 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15206: [SPARK-17640][SQL]Avoid using -1 as the default batchId ...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15206 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65801/ Test PASSed. ---

[GitHub] spark issue #15174: [SPARK-17502] [17609] [SQL] [Backport] [2.0] Fix Multipl...

2016-09-22 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15174 thanks, merging to 2.0! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15206: [SPARK-17640][SQL]Avoid using -1 as the default batchId ...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15206 **[Test build #65801 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65801/consoleFull)** for PR 15206 at commit

[GitHub] spark issue #15208: [SPARK-17641][SQL] Collect_list/Collect_set should not c...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15208 **[Test build #65806 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65806/consoleFull)** for PR 15208 at commit

[GitHub] spark issue #15208: [SPARK-17641][SQL] Collect_list/Collect_set should not c...

2016-09-22 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15208 cc @mengxr --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #15208: [SPARK-17641][SQL] Collect_list/Collect_set shoul...

2016-09-22 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/15208 [SPARK-17641][SQL] Collect_list/Collect_set should not collect null values. ## What changes were proposed in this pull request? We added native versions of `collect_set` and `collect_list`

[GitHub] spark issue #15102: [SPARK-17346][SQL] Add Kafka source for Structured Strea...

2016-09-22 Thread marmbrus
Github user marmbrus commented on the issue: https://github.com/apache/spark/pull/15102 Comparable requirement removed in #15207. > I think in the absence of prior information about the position in a topicpartition, you start a new batch on topic B starting from wherever the

[GitHub] spark issue #15207: [SPARK-17643] Remove comparable requirement from Offset

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15207 **[Test build #65805 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65805/consoleFull)** for PR 15207 at commit

[GitHub] spark issue #14971: [SPARK-17410] [SPARK-17284] Move Hive-generated Stats In...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14971 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65796/ Test PASSed. ---

[GitHub] spark pull request #15207: [SPARK-17643] Remove comparable requirement from ...

2016-09-22 Thread marmbrus
GitHub user marmbrus opened a pull request: https://github.com/apache/spark/pull/15207 [SPARK-17643] Remove comparable requirement from Offset For some sources, it is difficult to provide a global ordering based only on the data in the offset. Since we don't use comparison for

[GitHub] spark issue #14971: [SPARK-17410] [SPARK-17284] Move Hive-generated Stats In...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14971 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14971: [SPARK-17410] [SPARK-17284] Move Hive-generated Stats In...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14971 **[Test build #65796 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65796/consoleFull)** for PR 14971 at commit

[GitHub] spark issue #15102: [SPARK-17346][SQL] Add Kafka source for Structured Strea...

2016-09-22 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/15102 @koeninger I did some independent brainstorming with @zsxwing on topic deletion, and yeah I agree with you that attempting to account for deleted topics in the offset in the KafkaSourceOffset such

[GitHub] spark issue #14971: [SPARK-17410] [SPARK-17284] Move Hive-generated Stats In...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14971 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65795/ Test PASSed. ---

[GitHub] spark issue #14971: [SPARK-17410] [SPARK-17284] Move Hive-generated Stats In...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14971 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14971: [SPARK-17410] [SPARK-17284] Move Hive-generated Stats In...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14971 **[Test build #65795 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65795/consoleFull)** for PR 14971 at commit

[GitHub] spark issue #15204: [SPARK-17639][build] Add jce.jar to buildclasspath when ...

2016-09-22 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/15204 If tests pass I'll merge this to unblock #15172. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #15205: [SPARK-16240][ML] ML persistence backward compatibility ...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15205 **[Test build #65804 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65804/consoleFull)** for PR 15205 at commit

[GitHub] spark issue #9766: [SPARK-11775][PYSPARK][SQL] Allow PySpark to register Jav...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/9766 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65797/ Test PASSed. ---

[GitHub] spark issue #9766: [SPARK-11775][PYSPARK][SQL] Allow PySpark to register Jav...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/9766 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #9766: [SPARK-11775][PYSPARK][SQL] Allow PySpark to register Jav...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/9766 **[Test build #65797 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65797/consoleFull)** for PR 9766 at commit

  1   2   3   4   5   >