[GitHub] spark issue #10881: [SPARK-12967][Netty] Avoid NettyRpc error message during...

2016-11-03 Thread 1236897
Github user 1236897 commented on the issue: https://github.com/apache/spark/pull/10881 @ zsxwing if I just ignore it, will it avoid my issue? my issue is I need complete my job within 5 mins. but the issue "RpcEnv already stopped" waste a lot of time to disconnect and make my job

[GitHub] spark pull request #15637: [SPARK-18000] [SQL] Aggregation function for comp...

2016-11-03 Thread ron8hu
Github user ron8hu commented on a diff in the pull request: https://github.com/apache/spark/pull/15637#discussion_r86410847 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/MapAggregate.scala --- @@ -0,0 +1,332 @@ +/* + * Licensed

[GitHub] spark issue #15671: [SPARK-18206][ML]Add instrumentation logs to ML training...

2016-11-03 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/15671 @jkbradley Thanks for bringing that up. I'm ok with alternate solutions provided they don't require someone to remember to manually add or manually except a new param, and that we can ensure that we

[GitHub] spark issue #15743: [SPARK-18236] Reduce duplicate objects in Spark UI and H...

2016-11-03 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/15743 @srowen, that's a great tip about `UseStringDeduplication`; I'll give this a try in my deployment. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #15637: [SPARK-18000] [SQL] Aggregation function for comp...

2016-11-03 Thread ron8hu
Github user ron8hu commented on a diff in the pull request: https://github.com/apache/spark/pull/15637#discussion_r86408951 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/MapAggregate.scala --- @@ -0,0 +1,332 @@ +/* + * Licensed

[GitHub] spark pull request #15743: [SPARK-18236] Reduce duplicate objects in Spark U...

2016-11-03 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/15743#discussion_r86410060 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1089,7 +1089,7 @@ class DAGScheduler( // To avoid UI

[GitHub] spark issue #10881: [SPARK-12967][Netty] Avoid NettyRpc error message during...

2016-11-03 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/10881 @1236897 If you don't want to build Spark, it's fine to just catch this special exception thrown from `SparkContxt.stop` and ignore it. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request #15637: [SPARK-18000] [SQL] Aggregation function for comp...

2016-11-03 Thread ron8hu
Github user ron8hu commented on a diff in the pull request: https://github.com/apache/spark/pull/15637#discussion_r86409288 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/MapAggregate.scala --- @@ -0,0 +1,332 @@ +/* + * Licensed

[GitHub] spark pull request #15637: [SPARK-18000] [SQL] Aggregation function for comp...

2016-11-03 Thread ron8hu
Github user ron8hu commented on a diff in the pull request: https://github.com/apache/spark/pull/15637#discussion_r86409090 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/MapAggregate.scala --- @@ -0,0 +1,332 @@ +/* + * Licensed

[GitHub] spark issue #10881: [SPARK-12967][Netty] Avoid NettyRpc error message during...

2016-11-03 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/10881 > compile the code through ecplise. No. Take a look at this page about how to build Spark: http://spark.apache.org/docs/1.6.2/building-spark.html > Lastly, add the output jar to

[GitHub] spark issue #15731: [SPARK-18219] Move commit protocol API (internal) from s...

2016-11-03 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/15731 Still working on it, sorry for delay. The listed works are: 1. Port `SparkHadoopWriter` to use commit protocol; 2. Write a new `SparkNewHadoopWriter` to write `RDD` through new Hadoop

[GitHub] spark pull request #15743: [SPARK-18236] Reduce duplicate objects in Spark U...

2016-11-03 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/15743#discussion_r86407389 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/UIData.scala --- @@ -197,8 +196,32 @@ private[spark] object UIData { shuffleWriteMetrics:

[GitHub] spark pull request #15743: [SPARK-18236] Reduce duplicate objects in Spark U...

2016-11-03 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/15743#discussion_r86407326 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/UIData.scala --- @@ -197,8 +196,32 @@ private[spark] object UIData { shuffleWriteMetrics:

[GitHub] spark issue #13911: [SPARK-16215][SQL] Reduce runtime overhead of a program ...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13911 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13911: [SPARK-16215][SQL] Reduce runtime overhead of a program ...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13911 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68073/ Test FAILed. ---

[GitHub] spark issue #13911: [SPARK-16215][SQL] Reduce runtime overhead of a program ...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13911 **[Test build #68073 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68073/consoleFull)** for PR 13911 at commit

[GitHub] spark issue #10881: [SPARK-12967][Netty] Avoid NettyRpc error message during...

2016-11-03 Thread 1236897
Github user 1236897 commented on the issue: https://github.com/apache/spark/pull/10881 @zsxwing could you give the link about spark 1.6.2 of github? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #15743: [SPARK-18236] Reduce duplicate objects in Spark U...

2016-11-03 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/15743#discussion_r86406738 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1089,7 +1089,7 @@ class DAGScheduler( // To avoid UI

[GitHub] spark issue #15671: [SPARK-18206][ML]Add instrumentation logs to ML training...

2016-11-03 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/15671 I don't want to truncate Param strings because it would create invalid JSON in case people want to try to catch and parse the logs. I like the idea of allowing exceptions and possibly adding

[GitHub] spark issue #15746: [SPARK-18239][SPARKR] Gradient Boosted Tree for R

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15746 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15746: [SPARK-18239][SPARKR] Gradient Boosted Tree for R

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15746 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68072/ Test PASSed. ---

[GitHub] spark issue #15746: [SPARK-18239][SPARKR] Gradient Boosted Tree for R

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15746 **[Test build #68072 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68072/consoleFull)** for PR 15746 at commit

[GitHub] spark issue #10881: [SPARK-12967][Netty] Avoid NettyRpc error message during...

2016-11-03 Thread 1236897
Github user 1236897 commented on the issue: https://github.com/apache/spark/pull/10881 @zsxwing Thank you for your reply and sorry to disturb you, cos this project is so import for me, I descrise what i need to do. Firistly, check out the spark 1.6 from Github. Secondly, use the git

[GitHub] spark issue #15755: [SPARK-15902][PySpark] Add deprecation warning if python...

2016-11-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15755 This duplicates https://github.com/apache/spark/pull/15733 -- can you close this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #10881: [SPARK-12967][Netty] Avoid NettyRpc error message during...

2016-11-03 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/10881 @1236897 You can check out Spark 1.6.2 tag and apply this patch. Then build the Spark and use this one to submit your Spark application. You can still use the Spark maven artifact to build your

[GitHub] spark issue #15755: [SPARK-15902][PySpark] Add deprecation warning if python...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15755 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #15755: [SPARK-15902][PySpark] Add deprecation warning if...

2016-11-03 Thread krishnakalyan3
GitHub user krishnakalyan3 opened a pull request: https://github.com/apache/spark/pull/15755 [SPARK-15902][PySpark] Add deprecation warning if python version below Python 2.7 ## What changes were proposed in this pull request? Deprecation warning if we detect we are running in

[GitHub] spark issue #15754: [SPARK-18200][GRAPHX][FOLLOW-UP] Support zero as an init...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15754 **[Test build #68074 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68074/consoleFull)** for PR 15754 at commit

[GitHub] spark pull request #15754: [SPARK-18200][GRAPHX][FOLLOW-UP] Support zero as ...

2016-11-03 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/15754 [SPARK-18200][GRAPHX][FOLLOW-UP] Support zero as an initial capacity in OpenHashSet ## What changes were proposed in this pull request? This is a folloup PR of #15741 in order to

[GitHub] spark issue #13911: [SPARK-16215][SQL] Reduce runtime overhead of a program ...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13911 **[Test build #68073 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68073/consoleFull)** for PR 13911 at commit

[GitHub] spark issue #14750: [SPARK-17183][SQL] put hive serde table schema to table ...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14750 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68071/ Test PASSed. ---

[GitHub] spark issue #14750: [SPARK-17183][SQL] put hive serde table schema to table ...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14750 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14750: [SPARK-17183][SQL] put hive serde table schema to table ...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14750 **[Test build #68071 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68071/consoleFull)** for PR 14750 at commit

[GitHub] spark pull request #15745: [SPARK-18207][SQL] Fix a compilation error due to...

2016-11-03 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15745#discussion_r86395330 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -1658,4 +1658,49 @@ class DataFrameSuite extends QueryTest with

[GitHub] spark pull request #15745: [SPARK-18207][SQL] Fix a compilation error due to...

2016-11-03 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15745#discussion_r86394739 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala --- @@ -247,15 +247,16 @@ abstract class HashExpression[E]

[GitHub] spark issue #14750: [SPARK-17183][SQL] put hive serde table schema to table ...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14750 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68070/ Test PASSed. ---

[GitHub] spark issue #14750: [SPARK-17183][SQL] put hive serde table schema to table ...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14750 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14750: [SPARK-17183][SQL] put hive serde table schema to table ...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14750 **[Test build #68070 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68070/consoleFull)** for PR 14750 at commit

[GitHub] spark pull request #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION sho...

2016-11-03 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15704#discussion_r86393914 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -243,7 +243,7 @@ partitionSpec ;

[GitHub] spark issue #15746: [SPARK-18239][SPARKR] Gradient Boosted Tree for R

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15746 **[Test build #68072 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68072/consoleFull)** for PR 15746 at commit

[GitHub] spark issue #14498: [SPARK-16904] [SQL] Removal of Hive Built-in Hash Functi...

2016-11-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14498 @rxin Sure, will do it this weekend. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION sho...

2016-11-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/15704#discussion_r86389182 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -418,27 +419,56 @@ case class

[GitHub] spark issue #15523: [SPARK-17981] [SPARK-17957] [SQL] Fix Incorrect Nullabil...

2016-11-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15523 Sure, @hvanhovell will do it. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15590: [SPARK-17949][SQL] A JVM object based aggregate o...

2016-11-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15590 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #15741: [SPARK-18200][GRAPHX] Support zero as an initial ...

2016-11-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/15741#discussion_r86388101 --- Diff: core/src/main/scala/org/apache/spark/util/collection/OpenHashSet.scala --- @@ -271,8 +271,12 @@ class OpenHashSet[@specialized(Long, Int) T:

[GitHub] spark pull request #11105: [SPARK-12469][CORE] Data Property accumulators fo...

2016-11-03 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/11105#discussion_r86381839 --- Diff: core/src/main/scala/org/apache/spark/util/AccumulatorV2.scala --- @@ -42,18 +60,43 @@ private[spark] case class AccumulatorMetadata( * `OUT`

[GitHub] spark issue #15590: [SPARK-17949][SQL] A JVM object based aggregate operator

2016-11-03 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/15590 LGTM. Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION sho...

2016-11-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/15704#discussion_r86386317 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -418,27 +419,56 @@ case class

[GitHub] spark pull request #11105: [SPARK-12469][CORE] Data Property accumulators fo...

2016-11-03 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/11105#discussion_r86384551 --- Diff: core/src/main/scala/org/apache/spark/util/AccumulatorV2.scala --- @@ -306,18 +428,19 @@ class LongAccumulator extends AccumulatorV2[jl.Long,

[GitHub] spark pull request #11105: [SPARK-12469][CORE] Data Property accumulators fo...

2016-11-03 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/11105#discussion_r86384061 --- Diff: core/src/main/scala/org/apache/spark/util/AccumulatorV2.scala --- @@ -22,17 +22,36 @@ import java.io.ObjectInputStream import

[GitHub] spark pull request #11105: [SPARK-12469][CORE] Data Property accumulators fo...

2016-11-03 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/11105#discussion_r86383782 --- Diff: core/src/main/scala/org/apache/spark/rdd/ShuffledRDD.scala --- @@ -104,10 +105,26 @@ class ShuffledRDD[K: ClassTag, V: ClassTag, C: ClassTag](

[GitHub] spark pull request #11105: [SPARK-12469][CORE] Data Property accumulators fo...

2016-11-03 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/11105#discussion_r84380686 --- Diff: core/src/main/scala/org/apache/spark/util/AccumulatorV2.scala --- @@ -136,15 +179,76 @@ abstract class AccumulatorV2[IN, OUT] extends Serializable

[GitHub] spark pull request #11105: [SPARK-12469][CORE] Data Property accumulators fo...

2016-11-03 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/11105#discussion_r86384641 --- Diff: core/src/main/scala/org/apache/spark/util/AccumulatorV2.scala --- @@ -383,16 +506,7 @@ class DoubleAccumulator extends AccumulatorV2[jl.Double,

[GitHub] spark pull request #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION sho...

2016-11-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/15704#discussion_r86385850 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -204,6 +207,41 @@ class AstBuilder extends

[GitHub] spark pull request #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION sho...

2016-11-03 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15704#discussion_r86385701 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -418,27 +419,56 @@ case class AlterTableRenamePartitionCommand(

[GitHub] spark pull request #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION sho...

2016-11-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/15704#discussion_r86385537 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -243,7 +243,7 @@ partitionSpec ;

[GitHub] spark issue #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-11-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15704 Thank you, @hvanhovell . I'll revise the PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #15741: [SPARK-18200][GRAPHX] Support zero as an initial ...

2016-11-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/15741#discussion_r86382636 --- Diff: core/src/main/scala/org/apache/spark/util/collection/OpenHashSet.scala --- @@ -271,8 +271,12 @@ class OpenHashSet[@specialized(Long, Int) T:

[GitHub] spark pull request #15741: [SPARK-18200][GRAPHX] Support zero as an initial ...

2016-11-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/15741#discussion_r86382197 --- Diff: core/src/main/scala/org/apache/spark/util/collection/OpenHashSet.scala --- @@ -271,8 +271,12 @@ class OpenHashSet[@specialized(Long, Int) T:

[GitHub] spark pull request #15741: [SPARK-18200][GRAPHX] Support zero as an initial ...

2016-11-03 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15741#discussion_r86379957 --- Diff: core/src/main/scala/org/apache/spark/util/collection/OpenHashSet.scala --- @@ -271,8 +271,12 @@ class OpenHashSet[@specialized(Long, Int) T:

[GitHub] spark pull request #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION sho...

2016-11-03 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15704#discussion_r86377116 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -418,27 +419,56 @@ case class AlterTableRenamePartitionCommand(

[GitHub] spark pull request #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION sho...

2016-11-03 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15704#discussion_r86378445 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -418,27 +419,56 @@ case class AlterTableRenamePartitionCommand(

[GitHub] spark issue #14995: [Test Only][SPARK-6235][CORE]Address various 2G limits

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14995 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68069/ Test PASSed. ---

[GitHub] spark issue #14995: [Test Only][SPARK-6235][CORE]Address various 2G limits

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14995 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14995: [Test Only][SPARK-6235][CORE]Address various 2G limits

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14995 **[Test build #68069 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68069/consoleFull)** for PR 14995 at commit

[GitHub] spark pull request #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION sho...

2016-11-03 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15704#discussion_r86376440 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -204,6 +207,41 @@ class AstBuilder extends

[GitHub] spark issue #11119: [SPARK-10780][ML] Add an initial model to kmeans

2016-11-03 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/9 @yinxusen Status update? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION sho...

2016-11-03 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15704#discussion_r86374321 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -243,7 +243,7 @@ partitionSpec ;

[GitHub] spark issue #15523: [SPARK-17981] [SPARK-17957] [SQL] Fix Incorrect Nullabil...

2016-11-03 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15523 Merging to master/2.1. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #15523: [SPARK-17981] [SPARK-17957] [SQL] Fix Incorrect N...

2016-11-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15523 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15523: [SPARK-17981] [SPARK-17957] [SQL] Fix Incorrect Nullabil...

2016-11-03 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15523 @gatorsmile can you open a PR for 2.0 if we also need to port it to that branch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip...

2016-11-03 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/15659#discussion_r86365769 --- Diff: python/pyspark/find_spark_home.py --- @@ -0,0 +1,66 @@ +#!/usr/bin/python + +# +# Licensed to the Apache Software Foundation (ASF)

[GitHub] spark issue #15671: [SPARK-18206][ML]Add instrumentation logs to ML training...

2016-11-03 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/15671 I created [SPARK-18253](https://issues.apache.org/jira/browse/SPARK-18253) to track it. We may have to get to it after 2.1 QA period. --- If your project is set up for it, you can reply to this

[GitHub] spark issue #15746: [SPARK-18239][SPARKR] Gradient Boosted Tree for R

2016-11-03 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/15746 Sure, I can make a pass tomorrow. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15746: [SPARK-18239][SPARKR] Gradient Boosted Tree for R

2016-11-03 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15746 @mengxr @yanboliang Could you review this ? I'll try to take a look by end of this week. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request #15628: [SPARK-17471][ML] Add compressed method to ML mat...

2016-11-03 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/15628#discussion_r86360982 --- Diff: mllib-local/src/main/scala/org/apache/spark/ml/linalg/Matrices.scala --- @@ -1076,4 +1240,15 @@ object Matrices {

[GitHub] spark pull request #15692: [SPARK-18177][ML][PYSPARK] Add missing 'subsampli...

2016-11-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15692 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #15044: [SQL][SPARK-17490] Optimize SerializeFromObject()...

2016-11-03 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15044#discussion_r86358423 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/RowEncoder.scala --- @@ -119,18 +119,36 @@ object RowEncoder {

[GitHub] spark issue #14750: [SPARK-17183][SQL] put hive serde table schema to table ...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14750 **[Test build #68071 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68071/consoleFull)** for PR 14750 at commit

[GitHub] spark pull request #15741: [SPARK-18200][GRAPHX] Support zero as an initial ...

2016-11-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/15741#discussion_r86359753 --- Diff: core/src/main/scala/org/apache/spark/util/collection/OpenHashSet.scala --- @@ -271,8 +271,12 @@ class OpenHashSet[@specialized(Long, Int) T:

[GitHub] spark issue #15753: Dev advait

2016-11-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15753 @advaitraut it seems mistakenly opened. We might have to close this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #15692: [SPARK-18177][ML][PYSPARK] Add missing 'subsamplingRate'...

2016-11-03 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/15692 LGTM, merged into master and branch-2.1. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #15044: [SQL][SPARK-17490] Optimize SerializeFromObject()...

2016-11-03 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15044#discussion_r86358654 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -993,7 +993,7 @@ case class

[GitHub] spark pull request #15044: [SQL][SPARK-17490] Optimize SerializeFromObject()...

2016-11-03 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15044#discussion_r86358543 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/RowEncoder.scala --- @@ -193,7 +211,8 @@ object RowEncoder { // as

[GitHub] spark pull request #15628: [SPARK-17471][ML] Add compressed method to ML mat...

2016-11-03 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/15628#discussion_r86358529 --- Diff: mllib-local/src/main/scala/org/apache/spark/ml/linalg/Matrices.scala --- @@ -1076,4 +1240,15 @@ object Matrices {

[GitHub] spark issue #14750: [SPARK-17183][SQL] put hive serde table schema to table ...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14750 **[Test build #68070 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68070/consoleFull)** for PR 14750 at commit

[GitHub] spark issue #15750: [SPARK-18244][SQL] Rename partitionProviderIsHive -> tra...

2016-11-03 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15750 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #14937: [SPARK-8519][SPARK-11560] [ML] [MLlib] Optimize KMeans i...

2016-11-03 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/14937 @sethah Yeah, I agree it's better to run more test against large-scale data. If the number of feature or cluster is large, the center array slice cost and some other place can be optimized which

[GitHub] spark issue #15011: [SPARK-17122][SQL]support drop current database

2016-11-03 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15011 shall we backport it to 2.0? it works in spark 1.6 and earlier. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #12904: [SPARK-15125][SQL] Changing CSV data source mapping of e...

2016-11-03 Thread antoniobarbuzzi
Github user antoniobarbuzzi commented on the issue: https://github.com/apache/spark/pull/12904 I agree with @HyukjinKwon and comment https://github.com/apache/spark/pull/12904#issuecomment-217605160, except for third use case, i.e. I'd rather do this: With the option,

[GitHub] spark issue #14995: [Test Only][SPARK-6235][CORE]Address various 2G limits

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14995 **[Test build #68069 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68069/consoleFull)** for PR 14995 at commit

[GitHub] spark issue #15753: Dev advait

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15753 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #15753: Dev advait

2016-11-03 Thread advaitraut
GitHub user advaitraut opened a pull request: https://github.com/apache/spark/pull/15753 Dev advait ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested? (Please explain how this

[GitHub] spark issue #15752: [SPARK-18250] [SQL] Minor fixes to UTF8String

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15752 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15752: [SPARK-18250] [SQL] Minor fixes to UTF8String

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15752 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68068/ Test PASSed. ---

[GitHub] spark issue #15752: [SPARK-18250] [SQL] Minor fixes to UTF8String

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15752 **[Test build #68068 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68068/consoleFull)** for PR 15752 at commit

[GitHub] spark issue #15432: [SPARK-17854][SQL] rand/randn allows null/long as input ...

2016-11-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15432 I will rebase and clean up this tomorrow. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #15751: [SPARK-18246][SQL] Throws an exception before execution ...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15751 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15751: [SPARK-18246][SQL] Throws an exception before execution ...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15751 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68067/ Test PASSed. ---

[GitHub] spark issue #15751: [SPARK-18246][SQL] Throws an exception before execution ...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15751 **[Test build #68067 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68067/consoleFull)** for PR 15751 at commit

<    1   2   3   4   5   6   >