[GitHub] spark issue #14818: [SPARK-17157][SPARKR][WIP]: Add multiclass logistic regr...

2016-08-25 Thread junyangq
Github user junyangq commented on the issue: https://github.com/apache/spark/pull/14818 We may want to use a different name? glmnet related name could be confusing if it is actually only multiclass logistic. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #13584: [SPARK-15509][ML][SparkR] R MLlib algorithms should supp...

2016-08-25 Thread junyangq
Github user junyangq commented on the issue: https://github.com/apache/spark/pull/13584 Sounds good. That's also what we meant. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13584: [SPARK-15509][ML][SparkR] R MLlib algorithms should supp...

2016-08-25 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13584 Yeah I was going to say that we need to handle cases where `labels_output` is also used. We can just add a numeric suffix maybe ? --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #14753: [SPARK-17187][SQL] Supports using arbitrary Java object ...

2016-08-25 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/14753 Thanks. Overall looks good. I am merging this to master. Let me tweak the interface later. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #14811: [SPARK-17231][CORE] Avoid building debug or trace log me...

2016-08-25 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/14811 @mallman Could you close this PR since GitHub won't close it automatically, please? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #14814: [SPARK-17242][Document]Update links of external dstream ...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14814 **[Test build #6 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/6/consoleFull)** for PR 14814 at commit

[GitHub] spark issue #14817: [SPARK-17247][SQL]: when calcualting size of a relation ...

2016-08-25 Thread Parth-Brahmbhatt
Github user Parth-Brahmbhatt commented on the issue: https://github.com/apache/spark/pull/14817 @hvanhovell The behavior in case this fallbackToHdfs is not enabled ( and by default it is not enabled for performance reason) is to return the value specified via

[GitHub] spark issue #13584: [SPARK-15509][ML][SparkR] R MLlib algorithms should supp...

2016-08-25 Thread junyangq
Github user junyangq commented on the issue: https://github.com/apache/spark/pull/13584 @shivaram Does it sound reasonable to you? Just discussed this with @jkbradley. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #14811: [SPARK-17231][CORE] Avoid building debug or trace log me...

2016-08-25 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/14811 Thanks! Merging into branch 2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #14811: [SPARK-17231][CORE] Avoid building debug or trace log me...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14811 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14816: [SPARK-17245] [SQL] [BRANCH-1.6] Do not rely on Hive's s...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14816 **[Test build #64443 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64443/consoleFull)** for PR 14816 at commit

[GitHub] spark issue #14815: [SPARK-17244] Catalyst should not pushdown non-determini...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14815 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64437/ Test PASSed. ---

[GitHub] spark issue #14811: [SPARK-17231][CORE] Avoid building debug or trace log me...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14811 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64438/ Test PASSed. ---

[GitHub] spark issue #14815: [SPARK-17244] Catalyst should not pushdown non-determini...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14815 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14811: [SPARK-17231][CORE] Avoid building debug or trace log me...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14811 **[Test build #64438 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64438/consoleFull)** for PR 14811 at commit

[GitHub] spark issue #14815: [SPARK-17244] Catalyst should not pushdown non-determini...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14815 **[Test build #64437 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64437/consoleFull)** for PR 14815 at commit

[GitHub] spark issue #14816: [SPARK-17245] [SQL] [BRANCH-1.6] Do not rely on Hive's s...

2016-08-25 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/14816 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #14816: [SPARK-17245] [SQL] [BRANCH-1.6] Do not rely on Hive's s...

2016-08-25 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/14816 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #14710: [SPARK-16533][CORE] resolve deadlocking in driver when e...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14710 **[Test build #64442 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64442/consoleFull)** for PR 14710 at commit

[GitHub] spark issue #14710: [SPARK-16533][CORE] resolve deadlocking in driver when e...

2016-08-25 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/14710 wow a core dump in the build. retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #13584: [SPARK-15509][ML][SparkR] R MLlib algorithms should supp...

2016-08-25 Thread keypointt
Github user keypointt commented on the issue: https://github.com/apache/spark/pull/13584 sure I'll try to scan through all the mllib algorithms --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #14813: [SPARK-17240][core] Make SparkConf serializable a...

2016-08-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14813 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14818: [SPARK-17157][SPARKR][WIP]: Add multiclass logistic regr...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14818 **[Test build #64441 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64441/consoleFull)** for PR 14818 at commit

[GitHub] spark issue #14818: [SPARK-17157][SPARKR][WIP]: Add multiclass logistic regr...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14818 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64441/ Test FAILed. ---

[GitHub] spark issue #14818: [SPARK-17157][SPARKR][WIP]: Add multiclass logistic regr...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14818 **[Test build #64441 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64441/consoleFull)** for PR 14818 at commit

[GitHub] spark issue #14818: [SPARK-17157][SPARKR][WIP]: Add multiclass logistic regr...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14818 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14813: [SPARK-17240][core] Make SparkConf serializable again.

2016-08-25 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/14813 Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #14176: [SPARK-16525][SQL] Enable Row Based HashMap in Ha...

2016-08-25 Thread clockfly
Github user clockfly commented on a diff in the pull request: https://github.com/apache/spark/pull/14176#discussion_r76341601 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -459,52 +475,91 @@ case class HashAggregateExec(

[GitHub] spark issue #14817: [SPARK-17247][SQL]: when calcualting size of a relation ...

2016-08-25 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14817 @Parth-Brahmbhatt we are currently working Cost Based Optimization in Spark. An important input will be the actual size of the table. Having partial statistics (what you are suggestion) will not

[GitHub] spark pull request #14818: [SPARK-17157][SPARKR][WIP]: Add multiclass logist...

2016-08-25 Thread wangmiao1981
GitHub user wangmiao1981 opened a pull request: https://github.com/apache/spark/pull/14818 [SPARK-17157][SPARKR][WIP]: Add multiclass logistic regression SparkR Wrapper ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix)

[GitHub] spark issue #14809: [SPARK-17238][SQL] simplify the logic for converting dat...

2016-08-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14809 https://github.com/apache/spark/pull/13060 This change is based on another PR. If users specify the location in `CREATE TABLE`, we always set the table type to `EXTERNAL`. --- If your project

[GitHub] spark issue #14793: [SPARK-17221] [SQL] Build File-based Test Cases for Usin...

2016-08-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14793 @rxin How about this one? This is porting our existing test cases, instead of HiveCompatbilitySuite --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #14782: [SPARK-17208] [SQL] Build Outer Join Test Cases in File-...

2016-08-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14782 I see. Thanks! Let me close this one. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14782: [SPARK-17208] [SQL] Build Outer Join Test Cases i...

2016-08-25 Thread gatorsmile
Github user gatorsmile closed the pull request at: https://github.com/apache/spark/pull/14782 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request #14176: [SPARK-16525][SQL] Enable Row Based HashMap in Ha...

2016-08-25 Thread clockfly
Github user clockfly commented on a diff in the pull request: https://github.com/apache/spark/pull/14176#discussion_r76340251 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -518,15 +573,30 @@ case class HashAggregateExec(

[GitHub] spark pull request #14176: [SPARK-16525][SQL] Enable Row Based HashMap in Ha...

2016-08-25 Thread clockfly
Github user clockfly commented on a diff in the pull request: https://github.com/apache/spark/pull/14176#discussion_r76340181 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -518,15 +573,30 @@ case class HashAggregateExec(

[GitHub] spark pull request #14176: [SPARK-16525][SQL] Enable Row Based HashMap in Ha...

2016-08-25 Thread clockfly
Github user clockfly commented on a diff in the pull request: https://github.com/apache/spark/pull/14176#discussion_r76339927 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -459,52 +475,91 @@ case class HashAggregateExec(

[GitHub] spark issue #13584: [SPARK-15509][ML][SparkR] R MLlib algorithms should supp...

2016-08-25 Thread junyangq
Github user junyangq commented on the issue: https://github.com/apache/spark/pull/13584 @keypointt Can we keep searching (in random or sequential way) until an unused column name has been found? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #14637: [SPARK-16967] move mesos to module

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14637 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64435/ Test PASSed. ---

[GitHub] spark issue #14637: [SPARK-16967] move mesos to module

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14637 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14176: [SPARK-16525][SQL] Enable Row Based HashMap in Ha...

2016-08-25 Thread clockfly
Github user clockfly commented on a diff in the pull request: https://github.com/apache/spark/pull/14176#discussion_r76339716 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -459,52 +475,91 @@ case class HashAggregateExec(

[GitHub] spark issue #14637: [SPARK-16967] move mesos to module

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14637 **[Test build #64435 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64435/consoleFull)** for PR 14637 at commit

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14638 **[Test build #64440 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64440/consoleFull)** for PR 14638 at commit

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-08-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14638 Hi, @rxin . Until now, I couldn't find more suitable class in `hive` package. I added the logic to check the Table's InputFormat. Now, `skip.header.line.count` option is applied for the table

[GitHub] spark pull request #14728: [SPARK-17165][SQL] FileStreamSource should not tr...

2016-08-25 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/14728#discussion_r76339023 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamOptions.scala --- @@ -0,0 +1,56 @@ +/* + * Licensed to the

[GitHub] spark issue #14817: [SPARK-17247][SQL]: when calcualting size of a relation ...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14817 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #14176: [SPARK-16525][SQL] Enable Row Based HashMap in Ha...

2016-08-25 Thread clockfly
Github user clockfly commented on a diff in the pull request: https://github.com/apache/spark/pull/14176#discussion_r76338880 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -459,52 +475,91 @@ case class HashAggregateExec(

[GitHub] spark pull request #14176: [SPARK-16525][SQL] Enable Row Based HashMap in Ha...

2016-08-25 Thread clockfly
Github user clockfly commented on a diff in the pull request: https://github.com/apache/spark/pull/14176#discussion_r76338652 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -279,9 +280,14 @@ case class HashAggregateExec(

[GitHub] spark pull request #14176: [SPARK-16525][SQL] Enable Row Based HashMap in Ha...

2016-08-25 Thread clockfly
Github user clockfly commented on a diff in the pull request: https://github.com/apache/spark/pull/14176#discussion_r76338535 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -279,9 +280,14 @@ case class HashAggregateExec(

[GitHub] spark pull request #14817: [SPARK-17247][SQL]: when calcualting size of a re...

2016-08-25 Thread Parth-Brahmbhatt
GitHub user Parth-Brahmbhatt opened a pull request: https://github.com/apache/spark/pull/14817 [SPARK-17247][SQL]: when calcualting size of a relation from hdfs, th… ## What changes were proposed in this pull request? when calcualting size of a relation from hdfs, the size

[GitHub] spark issue #14815: [SPARK-17244] Catalyst should not pushdown non-determini...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14815 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14815: [SPARK-17244] Catalyst should not pushdown non-determini...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14815 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64436/ Test FAILed. ---

[GitHub] spark issue #14815: [SPARK-17244] Catalyst should not pushdown non-determini...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14815 **[Test build #64436 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64436/consoleFull)** for PR 14815 at commit

[GitHub] spark issue #14655: [SPARK-16669][SQL]Adding partition prunning to Metastore...

2016-08-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14655 Thank you! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #14815: [SPARK-17244] Catalyst should not pushdown non-de...

2016-08-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14815#discussion_r76336122 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1386,15 +1386,17 @@ object EliminateOuterJoin

[GitHub] spark issue #11192: [SPARK-13257] [Improvement] Scala naive Bayes example: c...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11192 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #14777: [SPARK-17205] Literal.sql should handle Infinity ...

2016-08-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14777#discussion_r76335953 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/literals.scala --- @@ -251,8 +251,21 @@ case class Literal (value: Any,

[GitHub] spark issue #14816: [SPARK-17245] [SQL] [BRANCH-1.6] Do not rely on Hive's s...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14816 **[Test build #64439 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64439/consoleFull)** for PR 14816 at commit

[GitHub] spark issue #14816: [SPARK-17245] [SQL] [BRANCH-1.6] Do not rely on Hive's s...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14816 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14816: [SPARK-17245] [SQL] [BRANCH-1.6] Do not rely on Hive's s...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14816 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64439/ Test FAILed. ---

[GitHub] spark issue #14655: [SPARK-16669][SQL]Adding partition prunning to Metastore...

2016-08-25 Thread Parth-Brahmbhatt
Github user Parth-Brahmbhatt commented on the issue: https://github.com/apache/spark/pull/14655 @gatorsmile not sure if it will simplify much in this case as most of the complexity is in figuring out what partitions can be pruned which I don't think will go away. We will rely on hive

[GitHub] spark pull request #14777: [SPARK-17205] Literal.sql should handle Infinity ...

2016-08-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14777 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14813: [SPARK-17240][core] Make SparkConf serializable again.

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14813 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14813: [SPARK-17240][core] Make SparkConf serializable again.

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14813 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64433/ Test PASSed. ---

[GitHub] spark pull request #14815: [SPARK-17244] Catalyst should not pushdown non-de...

2016-08-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14815#discussion_r76334971 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1386,15 +1386,17 @@ object EliminateOuterJoin

[GitHub] spark pull request #14777: [SPARK-17205] Literal.sql should handle Infinity ...

2016-08-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14777#discussion_r76334925 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/literals.scala --- @@ -251,8 +251,21 @@ case class Literal (value: Any,

[GitHub] spark issue #14813: [SPARK-17240][core] Make SparkConf serializable again.

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14813 **[Test build #64433 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64433/consoleFull)** for PR 14813 at commit

[GitHub] spark issue #14777: [SPARK-17205] Literal.sql should handle Infinity and NaN

2016-08-25 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14777 LGTM - merging to master/2.0. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14816: [SPARK-17245] [SQL] [BRANCH-1.6] Do not rely on Hive's s...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14816 **[Test build #64439 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64439/consoleFull)** for PR 14816 at commit

[GitHub] spark pull request #14720: SPARK-12868: Allow Add jar to add jars from hdfs/...

2016-08-25 Thread Parth-Brahmbhatt
Github user Parth-Brahmbhatt commented on a diff in the pull request: https://github.com/apache/spark/pull/14720#discussion_r76334577 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveQuerySuite.scala --- @@ -865,6 +865,16 @@ class HiveQuerySuite extends

[GitHub] spark pull request #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-25 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r76334586 --- Diff: yarn/src/test/scala/org/apache/spark/security/IOEncryptionSuite.scala --- @@ -0,0 +1,332 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #14720: SPARK-12868: Allow Add jar to add jars from hdfs/...

2016-08-25 Thread Parth-Brahmbhatt
Github user Parth-Brahmbhatt commented on a diff in the pull request: https://github.com/apache/spark/pull/14720#discussion_r76334535 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala --- @@ -87,6 +88,9 @@ private[hive] class HiveClientImpl(

[GitHub] spark issue #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-25 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/8880 > They could be different sizes (different config options control that). We could change it so both use the same configs / same code to generate the keys, but in general if they're used for

[GitHub] spark pull request #14816: [SPARK-17245] [SQL] [BRANCH-1.6] Do not rely on H...

2016-08-25 Thread yhuai
GitHub user yhuai opened a pull request: https://github.com/apache/spark/pull/14816 [SPARK-17245] [SQL] [BRANCH-1.6] Do not rely on Hive's session state to retrieve HiveConf ## What changes were proposed in this pull request? Right now, we rely on Hive's `SessionState.get()` to

[GitHub] spark pull request #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-25 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r76334131 --- Diff: yarn/src/test/scala/org/apache/spark/security/IOEncryptionSuite.scala --- @@ -0,0 +1,332 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #14777: [SPARK-17205] Literal.sql should handle Infinity ...

2016-08-25 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/14777#discussion_r76332385 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/literals.scala --- @@ -251,8 +251,21 @@ case class Literal (value: Any,

[GitHub] spark pull request #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-25 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r76332258 --- Diff: docs/configuration.md --- @@ -559,6 +559,39 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark issue #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-25 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/8880 > Why needs to generate a new key for IO encryption? They could be different sizes (different config options control that). We could change it so both use the same configs / same code to

[GitHub] spark issue #14433: [SPARK-16829][SparkR]:sparkR sc.setLogLevel doesn't work

2016-08-25 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/14433 `args.primaryResource` is good for this purpose. I can make change similar to my initial commit but checking against `args.primaryResource`. --- If your project is set up for it, you can

[GitHub] spark issue #14812: [SPARK-17237][SQL] Remove unnecessary backticks in a piv...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14812 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-25 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/8880 Looks pretty overall. Just a high level question: Why needs to generate a new key for IO encryption? Can we just use `SecurityManager.getSecretKey`? --- If your project is set up for it, you can

[GitHub] spark issue #14812: [SPARK-17237][SQL] Remove unnecessary backticks in a piv...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14812 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64432/ Test PASSed. ---

[GitHub] spark issue #14812: [SPARK-17237][SQL] Remove unnecessary backticks in a piv...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14812 **[Test build #64432 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64432/consoleFull)** for PR 14812 at commit

[GitHub] spark pull request #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-25 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r76328447 --- Diff: core/src/main/scala/org/apache/spark/security/CryptoStreamUtils.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #14796: [SPARK-17229][SQL] PostgresDialect shouldn't wide...

2016-08-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14796 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14796: [SPARK-17229][SQL] PostgresDialect shouldn't widen float...

2016-08-25 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14796 LGTM - merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14777: [SPARK-17205] Literal.sql should handle Infinity ...

2016-08-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14777#discussion_r76326911 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/literals.scala --- @@ -251,8 +251,21 @@ case class Literal (value: Any,

[GitHub] spark issue #14270: [SPARK-5847][CORE] Allow for configuring MetricsSystem's...

2016-08-25 Thread markgrover
Github user markgrover commented on the issue: https://github.com/apache/spark/pull/14270 Sure. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #14815: [SPARK-17244] Catalyst should not pushdown non-determini...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14815 **[Test build #64437 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64437/consoleFull)** for PR 14815 at commit

[GitHub] spark issue #14811: [SPARK-17231][CORE] Avoid building debug or trace log me...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14811 **[Test build #64438 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64438/consoleFull)** for PR 14811 at commit

[GitHub] spark issue #14811: [SPARK-17231][CORE] Avoid building debug or trace log me...

2016-08-25 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/14811 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #14811: [SPARK-17231][CORE] Avoid building debug or trace log me...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14811 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14811: [SPARK-17231][CORE] Avoid building debug or trace log me...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14811 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64431/ Test FAILed. ---

[GitHub] spark issue #14811: [SPARK-17231][CORE] Avoid building debug or trace log me...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14811 **[Test build #64431 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64431/consoleFull)** for PR 14811 at commit

[GitHub] spark issue #14270: [SPARK-5847][CORE] Allow for configuring MetricsSystem's...

2016-08-25 Thread Ianwww
Github user Ianwww commented on the issue: https://github.com/apache/spark/pull/14270 Can this configuration be set in spark-defaults.conf as spark.metrics.namespace=${spark.app.name} @markgrover --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-25 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r76323741 --- Diff: core/src/main/scala/org/apache/spark/serializer/SerializerManager.scala --- @@ -103,16 +108,44 @@ private[spark] class

[GitHub] spark pull request #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-25 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r76323295 --- Diff: core/src/main/scala/org/apache/spark/util/collection/ExternalSorter.scala --- @@ -522,8 +521,9 @@ private[spark] class ExternalSorter[K, V, C](

[GitHub] spark pull request #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-25 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r76323076 --- Diff: core/src/main/scala/org/apache/spark/serializer/SerializerManager.scala --- @@ -103,16 +108,44 @@ private[spark] class

[GitHub] spark issue #14815: [SPARK-17244] Catalyst should not pushdown non-determini...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14815 **[Test build #64436 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64436/consoleFull)** for PR 14815 at commit

[GitHub] spark issue #14815: [SPARK-17244] Catalyst should not pushdown non-determini...

2016-08-25 Thread sameeragarwal
Github user sameeragarwal commented on the issue: https://github.com/apache/spark/pull/14815 cc @brkyvz who found this bug --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

<    1   2   3   4   5   6   >