[GitHub] spark issue #15263: [SPARK-14525][SQL][FOLLOWUP] Clean up JdbcRelationProvid...

2016-10-06 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15263 Hi @gatorsmile, would there other things more I should take care of? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #15354: [SPARK-17764][SQL] Add `to_json` supporting to co...

2016-10-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15354#discussion_r82314175 --- Diff: python/pyspark/sql/functions.py --- @@ -1729,6 +1729,29 @@ def from_json(col, schema, options={}): return Column(jc)

[GitHub] spark pull request #15354: [SPARK-17764][SQL] Add `to_json` supporting to co...

2016-10-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15354#discussion_r82313950 --- Diff: python/pyspark/sql/functions.py --- @@ -1729,6 +1729,29 @@ def from_json(col, schema, options={}): return Column(jc)

[GitHub] spark pull request #15367: [SPARK-17346][SQL][test-maven]Add Kafka source fo...

2016-10-06 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15367#discussion_r82313748 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSourceSuite.scala --- @@ -0,0 +1,425 @@ +/* + * Licensed to the

[GitHub] spark issue #15367: [SPARK-17346][SQL][test-maven]Add Kafka source for Struc...

2016-10-06 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15367 > @zsxwing Have you tested the maven build? > Test build #66433 has finished for PR 15367 at commit 0915826. > > This patch passes all tests. > This patch merges

[GitHub] spark issue #15292: [SPARK-17719][SPARK-17776][SQL] Unify and tie up options...

2016-10-06 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15292 @gatorsmile Sure, let me just wait and sweep it once! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #11601: [SPARK-13568] [ML] Create feature transformer to impute ...

2016-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11601 **[Test build #66476 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66476/consoleFull)** for PR 11601 at commit

[GitHub] spark issue #15371: [SPARK-17816] [Core] Fix ConcurrentModificationException...

2016-10-06 Thread seyfe
Github user seyfe commented on the issue: https://github.com/apache/spark/pull/15371 I also want to point out that below is the core part of fix. Rest of the code changes are side-effects of it. ``` + // `asScala` accesses the internal values using `java.util.Iterator`

[GitHub] spark issue #15367: [SPARK-17346][SQL][test-maven]Add Kafka source for Struc...

2016-10-06 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/15367 @zsxwing Have you test the maven build? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15367: [SPARK-17346][SQL][test-maven]Add Kafka source fo...

2016-10-06 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15367#discussion_r82313116 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSourceSuite.scala --- @@ -0,0 +1,425 @@ +/* + * Licensed to the

[GitHub] spark issue #15307: [SPARK-17731][SQL][STREAMING] Metrics for structured str...

2016-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15307 **[Test build #66475 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66475/consoleFull)** for PR 15307 at commit

[GitHub] spark issue #15384: [SPARK-17346][SQL][Tests]Fix the flaky topic deletion in...

2016-10-06 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/15384 Correction. Merging to master only. Can you fix the 2.0 PR with this issue since that is still open? --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #15384: [SPARK-17346][SQL][Tests]Fix the flaky topic deletion in...

2016-10-06 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/15384 LGTM. Merging it to master and 2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15386: [SPARK-17808][PYSPARK] Upgraded version of Pyrolite to 4...

2016-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15386 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15386: [SPARK-17808][PYSPARK] Upgraded version of Pyrolite to 4...

2016-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15386 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66474/ Test FAILed. ---

[GitHub] spark issue #15386: [SPARK-17808][PYSPARK] Upgraded version of Pyrolite to 4...

2016-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15386 **[Test build #66474 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66474/consoleFull)** for PR 15386 at commit

[GitHub] spark issue #14215: [SPARK-16544][SQL][WIP] Support for conversion from comp...

2016-10-06 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14215 @wgtmac BTW, as you might already know, my plan and though is, to implement each first and then unify them within a common parent at the end if possible and it makes sense.. --- If your

[GitHub] spark issue #15386: [SPARK-17808][PYSPARK] Upgraded version of Pyrolite to 4...

2016-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15386 **[Test build #66474 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66474/consoleFull)** for PR 15386 at commit

[GitHub] spark issue #15371: [SPARK-17816] [Core] Fix ConcurrentModificationException...

2016-10-06 Thread seyfe
Github user seyfe commented on the issue: https://github.com/apache/spark/pull/15371 @zsxwing . `JArray(v.asInstanceOf[java.util.List[(BlockId, BlockStatus)]].asScala.toList.map` this is the line of code which is causing ConcurrentModificationException. And you can see that from call

[GitHub] spark issue #14215: [SPARK-16544][SQL][WIP] Support for conversion from comp...

2016-10-06 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14215 @wgtmac Thanks for pinging. I think I can proceed this on this weekend. I haven't looked into vectorized one closely yet. If you have already looked into that, I think it'd also make sense not

[GitHub] spark pull request #15386: [SPARK-17808][PYSPARK] Upgraded version of Pyroli...

2016-10-06 Thread BryanCutler
GitHub user BryanCutler opened a pull request: https://github.com/apache/spark/pull/15386 [SPARK-17808][PYSPARK] Upgraded version of Pyrolite to 4.13 ## What changes were proposed in this pull request? Upgraded to a newer version of Pyrolite which supports serialization of a

[GitHub] spark issue #15371: [SPARK-17816] [Core] Fix ConcurrentModificationException...

2016-10-06 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15371 > @zsxwing. BlockStatusesAccumulator is using synchronizedList which is _seq for concurrency. But when the value method is executed and the output is used by asScala.toList it

[GitHub] spark issue #15249: [SPARK-17675] [CORE] Expand Blacklist for TaskSets

2016-10-06 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/15249 If I understood the change correctly, a node can get blacklisted for a taskset if sufficient (even different) tasks fail on executers on it. Which can potentially cause all nodes to be

[GitHub] spark issue #15371: [SPARK-17816] [Core] Fix ConcurrentModificationException...

2016-10-06 Thread seyfe
Github user seyfe commented on the issue: https://github.com/apache/spark/pull/15371 @zsxwing. `BlockStatusesAccumulator` is using synchronizedList which is `_seq` for concurrency. But when the `value` method is executed and the output is used by `asScala.toList` it caused the

[GitHub] spark issue #14215: [SPARK-16544][SQL][WIP] Support for conversion from comp...

2016-10-06 Thread wgtmac
Github user wgtmac commented on the issue: https://github.com/apache/spark/pull/14215 @HyukjinKwon Do you have a timeline for this patch? Also, what's your plan on vectorized parquet reader? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #15384: [SPARK-17346][SQL][Tests]Fix the flaky topic deletion in...

2016-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15384 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66471/ Test PASSed. ---

[GitHub] spark issue #15384: [SPARK-17346][SQL][Tests]Fix the flaky topic deletion in...

2016-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15384 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15384: [SPARK-17346][SQL][Tests]Fix the flaky topic deletion in...

2016-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15384 **[Test build #66471 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66471/consoleFull)** for PR 15384 at commit

[GitHub] spark issue #15218: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15218 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66465/ Test PASSed. ---

[GitHub] spark issue #15218: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15218 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15218: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15218 **[Test build #66465 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66465/consoleFull)** for PR 15218 at commit

[GitHub] spark issue #15371: [SPARK-17816] [Core] Fix ConcurrentModificationException...

2016-10-06 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15371 @seyfe The issue is `TaskInfo.accumulables` is accessed in multiple threads without synchronization. `TaskMetrics.updatedBlockStatuses` is fine. --- If your project is set up for it, you can reply

[GitHub] spark issue #15371: [SPARK-17816] [Core] Fix ConcurrentModificationException...

2016-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15371 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66473/ Test FAILed. ---

[GitHub] spark issue #15371: [SPARK-17816] [Core] Fix ConcurrentModificationException...

2016-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15371 **[Test build #66473 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66473/consoleFull)** for PR 15371 at commit

[GitHub] spark issue #15371: [SPARK-17816] [Core] Fix ConcurrentModificationException...

2016-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15371 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15371: [SPARK-17816] [Core] Fix ConcurrentModificationEx...

2016-10-06 Thread seyfe
Github user seyfe commented on a diff in the pull request: https://github.com/apache/spark/pull/15371#discussion_r82309886 --- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala --- @@ -281,7 +281,7 @@ private[spark] object JsonProtocol { ("Finish Time" ->

[GitHub] spark issue #15375: [SPARK-17790] Support for parallelizing R data.frame lar...

2016-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15375 **[Test build #66472 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66472/consoleFull)** for PR 15375 at commit

[GitHub] spark issue #15371: [SPARK-17816] [Core] Fix ConcurrentModificationException...

2016-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15371 **[Test build #66473 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66473/consoleFull)** for PR 15371 at commit

[GitHub] spark issue #15292: [SPARK-17719][SPARK-17776][SQL] Unify and tie up options...

2016-10-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15292 Will continue the review tonight. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #15365: [SPARK-17157][SPARKR]: Add multiclass logistic regressio...

2016-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15365 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15365: [SPARK-17157][SPARKR]: Add multiclass logistic regressio...

2016-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15365 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66468/ Test PASSed. ---

[GitHub] spark issue #15365: [SPARK-17157][SPARKR]: Add multiclass logistic regressio...

2016-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15365 **[Test build #66468 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66468/consoleFull)** for PR 15365 at commit

[GitHub] spark issue #15384: [SPARK-17346][SQL][Tests]Fix the flaky topic deletion in...

2016-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15384 **[Test build #66471 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66471/consoleFull)** for PR 15384 at commit

[GitHub] spark issue #15384: [SPARK-17346][SQL][Tests]Fix the flaky topic deletion in...

2016-10-06 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15384 > Are you sure we are ignoring all internal topics with that filter? > Because, it maybe better to not make assumption, and rather prefix all topics we create with something, and deleted only

[GitHub] spark issue #15385: [DO NOT MERGE]Try to reproduce DirectKafkaStreamSuite fa...

2016-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15385 **[Test build #66470 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66470/consoleFull)** for PR 15385 at commit

[GitHub] spark pull request #15385: [DO NOT MERGE]Try to reproduce DirectKafkaStreamS...

2016-10-06 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/15385 [DO NOT MERGE]Try to reproduce DirectKafkaStreamSuite failure ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was

[GitHub] spark pull request #15292: [SPARK-17719][SPARK-17776][SQL] Unify and tie up ...

2016-10-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15292#discussion_r82306624 --- Diff: docs/sql-programming-guide.md --- @@ -1014,16 +1014,31 @@ bin/spark-shell --driver-class-path postgresql-9.4.1207.jar --jars postgresql-9.

[GitHub] spark issue #15384: [SPARK-17346][SQL][Tests]Fix the flaky topic deletion in...

2016-10-06 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/15384 Are you sure we are ignoring all internal topics with that filter? Because, it maybe better to not make assumption, and rather prefix all topics we create with something, and deleted only those.

[GitHub] spark issue #15190: [SPARK-17620][SQL] Determine Serde by hive.default.filef...

2016-10-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15190 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15366: [SPARK-17793] [Web UI] Sorting on the description on the...

2016-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15366 **[Test build #66469 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66469/consoleFull)** for PR 15366 at commit

[GitHub] spark issue #15351: [SPARK-17612][SQL][branch-2.0] Support `DESCRIBE table P...

2016-10-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15351 Could you review this backport, @gatorsmile ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #15383: [SPARK-17750][SQL][BACKPORT-2.0] Fix CREATE VIEW ...

2016-10-06 Thread dongjoon-hyun
Github user dongjoon-hyun closed the pull request at: https://github.com/apache/spark/pull/15383 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15383: [SPARK-17750][SQL][BACKPORT-2.0] Fix CREATE VIEW with IN...

2016-10-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15383 Merged to 2.0. @dongjoon-hyun Could you please close it? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #15383: [SPARK-17750][SQL][BACKPORT-2.0] Fix CREATE VIEW with IN...

2016-10-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15383 Thank you so much, @gatorsmile ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15383: [SPARK-17750][SQL][BACKPORT-2.0] Fix CREATE VIEW with IN...

2016-10-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15383 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15355: [SPARK-17782][STREAMING] Disable Kafka 010 pattern based...

2016-10-06 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15355 @koeninger you can download the unit test logs from https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-sbt-hadoop-2.6/1756/artifact/ I saw the offsets of the first batch (pat1 0

[GitHub] spark issue #15351: [SPARK-17612][SQL][branch-2.0] Support `DESCRIBE table P...

2016-10-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15351 @hvanhovell If you are busy, I can take a look at this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #15249: [SPARK-17675] [CORE] Expand Blacklist for TaskSets

2016-10-06 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/15249 @mridulm re: job failures, can you elaborate on the job failure scenario you're concerned about? Jobs can only fail when some tasks are unschedulable, which can happen if a task is

[GitHub] spark issue #15355: [SPARK-17782][STREAMING] Disable Kafka 010 pattern based...

2016-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15355 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66463/ Test PASSed. ---

[GitHub] spark pull request #15369: [SPARK-17795] [Web UI] Sorting on stage or job ta...

2016-10-06 Thread ajbozarth
Github user ajbozarth commented on a diff in the pull request: https://github.com/apache/spark/pull/15369#discussion_r82301075 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StageTable.scala --- @@ -241,7 +246,8 @@ private[ui] class StagePagedTable( val

[GitHub] spark issue #15355: [SPARK-17782][STREAMING] Disable Kafka 010 pattern based...

2016-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15355 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15355: [SPARK-17782][STREAMING] Disable Kafka 010 pattern based...

2016-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15355 **[Test build #66463 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66463/consoleFull)** for PR 15355 at commit

[GitHub] spark issue #15365: [SPARK-17157][SPARKR]: Add multiclass logistic regressio...

2016-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15365 **[Test build #66468 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66468/consoleFull)** for PR 15365 at commit

[GitHub] spark issue #15300: [SPARK-17729] [SQL] Enable creating hive bucketed tables

2016-10-06 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/15300 @hvanhovell , @cloud-fan : Can you please review this PR ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #15384: [SPARK-17346][SQL][Tests]Fix the flaky topic deletion in...

2016-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15384 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66466/ Test PASSed. ---

[GitHub] spark issue #15384: [SPARK-17346][SQL][Tests]Fix the flaky topic deletion in...

2016-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15384 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15384: [SPARK-17346][SQL][Tests]Fix the flaky topic deletion in...

2016-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15384 **[Test build #66466 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66466/consoleFull)** for PR 15384 at commit

[GitHub] spark issue #15249: [SPARK-17675] [CORE] Expand Blacklist for TaskSets

2016-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15249 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15249: [SPARK-17675] [CORE] Expand Blacklist for TaskSets

2016-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15249 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66462/ Test PASSed. ---

[GitHub] spark issue #15249: [SPARK-17675] [CORE] Expand Blacklist for TaskSets

2016-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15249 **[Test build #66462 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66462/consoleFull)** for PR 15249 at commit

[GitHub] spark issue #15249: [SPARK-17675] [CORE] Expand Blacklist for TaskSets

2016-10-06 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/15249 @squito I am hoping we _can_ remove the old code/functionality actually (it is klunky very specific to single executor resource contention/shutdown usecase - unfortunately common enough to warrant

[GitHub] spark issue #15371: [SPARK-17463] [Core] Fix ConcurrentModificationException...

2016-10-06 Thread seyfe
Github user seyfe commented on the issue: https://github.com/apache/spark/pull/15371 Hi @zsxwing. I have a fix ready and testing it now. I will create a new ticket and send an updated PR today. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #15360: [SPARK-17073] [SQL] [FOLLOWUP] generate column-le...

2016-10-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15360#discussion_r82297698 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala --- @@ -405,6 +405,78 @@ class StatisticsSuite extends QueryTest with

[GitHub] spark pull request #15360: [SPARK-17073] [SQL] [FOLLOWUP] generate column-le...

2016-10-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15360#discussion_r82297526 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala --- @@ -405,6 +405,78 @@ class StatisticsSuite extends QueryTest with

[GitHub] spark pull request #15370: [SPARK-17417][Core] Fix # of partitions for Relia...

2016-10-06 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15370#discussion_r82296991 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -71,6 +72,11 @@ private[spark] object CallSite { private[spark] object Utils

[GitHub] spark pull request #15218: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-06 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/15218#discussion_r82296969 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskAssigner.scala --- @@ -0,0 +1,151 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #15378: [SPARK-17803][TESTS] Upgrade docker-client dependency

2016-10-06 Thread ckadner
Github user ckadner commented on the issue: https://github.com/apache/spark/pull/15378 Thanks @JoshRosen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #15375: [SPARK-17790] Support for parallelizing R data.fr...

2016-10-06 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/15375#discussion_r82296182 --- Diff: R/pkg/R/context.R --- @@ -123,19 +126,46 @@ parallelize <- function(sc, coll, numSlices = 1) { if (numSlices > length(coll))

[GitHub] spark issue #15365: [SPARK-17157][SPARKR]: Add multiclass logistic regressio...

2016-10-06 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/15365 re-test please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15375: [SPARK-17790] Support for parallelizing R data.frame lar...

2016-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15375 **[Test build #66467 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66467/consoleFull)** for PR 15375 at commit

[GitHub] spark issue #15384: [SPARK-17346][SQL][Tests]Fix the flaky topic deletion in...

2016-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15384 **[Test build #66466 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66466/consoleFull)** for PR 15384 at commit

[GitHub] spark pull request #15384: [SPARK-17346][SQL]Fix the flaky topic deletion in...

2016-10-06 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/15384 [SPARK-17346][SQL]Fix the flaky topic deletion in tests ## What changes were proposed in this pull request? A follow up Pr for SPARK-17346 to fix flaky

[GitHub] spark issue #15384: [SPARK-17346][SQL][Tests]Fix the flaky topic deletion in...

2016-10-06 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15384 /cc @tdas --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #15383: [SPARK-17750][SQL][BACKPORT-2.0] Fix CREATE VIEW with IN...

2016-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15383 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15383: [SPARK-17750][SQL][BACKPORT-2.0] Fix CREATE VIEW with IN...

2016-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15383 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66460/ Test PASSed. ---

[GitHub] spark issue #15383: [SPARK-17750][SQL][BACKPORT-2.0] Fix CREATE VIEW with IN...

2016-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15383 **[Test build #66460 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66460/consoleFull)** for PR 15383 at commit

[GitHub] spark pull request #14087: [SPARK-16411][SQL][STREAMING] Add textFile to Str...

2016-10-06 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/14087#discussion_r82293680 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamReader.scala --- @@ -311,6 +311,37 @@ final class DataStreamReader

[GitHub] spark pull request #15360: [SPARK-17073] [SQL] [FOLLOWUP] generate column-le...

2016-10-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15360#discussion_r82293578 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeColumnCommand.scala --- @@ -62,7 +62,7 @@ case class

[GitHub] spark issue #15249: [SPARK-17675] [CORE] Expand Blacklist for TaskSets

2016-10-06 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/15249 to be clear, when I proposed leaving the old feature in place, my intent was *not* to make them interact nicely at all. you wouldn't even be able to use the two features together. The idea was

[GitHub] spark pull request #14087: [SPARK-16411][SQL][STREAMING] Add textFile to Str...

2016-10-06 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/14087#discussion_r82293331 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/FileStreamSourceSuite.scala --- @@ -378,6 +378,24 @@ class FileStreamSourceSuite extends

[GitHub] spark issue #15367: [SPARK-17346][SQL][test-maven]Add Kafka source for Struc...

2016-10-06 Thread marmbrus
Github user marmbrus commented on the issue: https://github.com/apache/spark/pull/15367 We should definitly vet this PR carefully to make sure its safe. One thing that is missing from that guide, that I do believe is accepted practice, is more leeway when the feature is marked

[GitHub] spark issue #15218: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15218 **[Test build #66465 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66465/consoleFull)** for PR 15218 at commit

[GitHub] spark issue #15365: [SPARK-17157][SPARKR]: Add multiclass logistic regressio...

2016-10-06 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/15365 My local tests passed. Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #15218: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-06 Thread zhzhan
Github user zhzhan commented on a diff in the pull request: https://github.com/apache/spark/pull/15218#discussion_r82290564 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskAssigner.scala --- @@ -0,0 +1,151 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #15381: [SPARK-17707] [WEBUI] Web UI prevents spark-submit appli...

2016-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15381 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15381: [SPARK-17707] [WEBUI] Web UI prevents spark-submit appli...

2016-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15381 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66458/ Test PASSed. ---

[GitHub] spark issue #15381: [SPARK-17707] [WEBUI] Web UI prevents spark-submit appli...

2016-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15381 **[Test build #66458 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66458/consoleFull)** for PR 15381 at commit

[GitHub] spark pull request #15378: [SPARK-17803][TESTS] Upgrade docker-client depend...

2016-10-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15378 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15377: [SPARK-17802] Improved caller context logging.

2016-10-06 Thread weiqingy
Github user weiqingy commented on the issue: https://github.com/apache/spark/pull/15377 Thanks for proposing this PR. LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15249: [SPARK-17675] [CORE] Expand Blacklist for TaskSets

2016-10-06 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/15249 @tgravescs no decision here yet. @mridulm the main question for (2), though, is are the consequences a deal-breaker? It doesn't seem disastrous if a task needs to run on a non-local

<    1   2   3   4   5   >