[GitHub] spark pull request #16192: [SPARK-18764][Core]Add a warning log when skippin...

2016-12-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16192 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16175: [SPARK-17460][SQL]check if statistics.sizeInBytes >=0 in...

2016-12-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16175 I checked the code base. We are having multiple potential overflow issues in `sizeInBytes` of `Statistics`. We might need a fix for all the other places. --- If your project is set up for it,

[GitHub] spark issue #16192: [SPARK-18764][Core]Add a warning log when skipping a cor...

2016-12-07 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16192 Thanks. Merging to master and 2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #16175: [SPARK-17460][SQL]check if statistics.sizeInBytes...

2016-12-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16175#discussion_r91352281 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala --- @@ -1110,6 +1110,17 @@ class DatasetSuite extends QueryTest with

[GitHub] spark pull request #16197: [SPARK-17760][SQL][Backport] AnalysisException wi...

2016-12-07 Thread aray
Github user aray closed the pull request at: https://github.com/apache/spark/pull/16197 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16189: [SPARK-18761][CORE][WIP] Introduce "task reaper" to over...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16189 **[Test build #69803 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69803/consoleFull)** for PR 16189 at commit

[GitHub] spark pull request #16189: [SPARK-18761][CORE][WIP] Introduce "task reaper" ...

2016-12-07 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/16189#discussion_r91347493 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -432,6 +435,57 @@ private[spark] class Executor( } /**

[GitHub] spark issue #16179: [SPARK-18752][hive] "isSrcLocal" value should be set fro...

2016-12-07 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16179 I think I know what the problem is, this will require some test changes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #16175: [SPARK-17460][SQL]check if statistics.sizeInBytes...

2016-12-07 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/16175#discussion_r91345742 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala --- @@ -1110,6 +1110,16 @@ class DatasetSuite extends QueryTest with

[GitHub] spark issue #16190: [SPARK-18762][WEBUI] Web UI should be http:4040 instead ...

2016-12-07 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/16190 @sarutak Do you know why we needed this change to fix ssl-enabled history server in #15611? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #16119: [SPARK-18687][Pyspark][SQL]Backward compatibility - crea...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16119 **[Test build #69802 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69802/consoleFull)** for PR 16119 at commit

[GitHub] spark issue #16119: [SPARK-18687][Pyspark][SQL]Backward compatibility - crea...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16119 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69802/ Test FAILed. ---

[GitHub] spark issue #16119: [SPARK-18687][Pyspark][SQL]Backward compatibility - crea...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16119 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16119: [SPARK-18687][Pyspark][SQL]Backward compatibility - crea...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16119 **[Test build #69802 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69802/consoleFull)** for PR 16119 at commit

[GitHub] spark issue #16197: [SPARK-17760][SQL][Backport] AnalysisException with data...

2016-12-07 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16197 Merging to 2.0. Thanks! Can you close? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16194: [SPARK-18767][ML] Unify Models' toString methods

2016-12-07 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/16194 I haven't really checked about it, but is it possible to just provide the default `toString` implementation in some upstream abstract class, like Model? And we can do the similar thing for

[GitHub] spark issue #16119: [SPARK-18687][Pyspark][SQL]Backward compatibility - crea...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16119 **[Test build #69801 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69801/consoleFull)** for PR 16119 at commit

[GitHub] spark issue #16119: [SPARK-18687][Pyspark][SQL]Backward compatibility - crea...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16119 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69801/ Test FAILed. ---

[GitHub] spark issue #16119: [SPARK-18687][Pyspark][SQL]Backward compatibility - crea...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16119 Build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16197: [SPARK-17760][SQL][Backport] AnalysisException with data...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16197 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16197: [SPARK-17760][SQL][Backport] AnalysisException with data...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16197 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69798/ Test PASSed. ---

[GitHub] spark issue #16119: [SPARK-18687][Pyspark][SQL]Backward compatibility - crea...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16119 **[Test build #69801 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69801/consoleFull)** for PR 16119 at commit

[GitHub] spark issue #16197: [SPARK-17760][SQL][Backport] AnalysisException with data...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16197 **[Test build #69798 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69798/consoleFull)** for PR 16197 at commit

[GitHub] spark pull request #16150: [SPARK-18349][SparkR]:Update R API documentation ...

2016-12-07 Thread wangmiao1981
Github user wangmiao1981 commented on a diff in the pull request: https://github.com/apache/spark/pull/16150#discussion_r91341471 --- Diff: R/pkg/R/mllib.R --- @@ -661,7 +665,10 @@ setMethod("fitted", signature(object = "KMeansModel"), # Get the summary of a k-means model

[GitHub] spark issue #16114: [SPARK-18620][Streaming][Kinesis] Flatten input rates in...

2016-12-07 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/16114 I will need to take a deeper look at this to remember the code. I'm not sure but there may be some issues with the checkpointing happening to the WriteAheadLog and DynamoDB. Going to come back to

[GitHub] spark pull request #16114: [SPARK-18620][Streaming][Kinesis] Flatten input r...

2016-12-07 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/16114#discussion_r91339332 --- Diff: external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisRecordProcessor.scala --- @@ -68,7 +68,16 @@ private[kinesis] class

[GitHub] spark issue #16000: [SPARK-18537][Web UI]Add a REST api to spark streaming

2016-12-07 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16000 The API should be one; it doesn't belong to core or streaming, it belongs to Spark. Parts of it would be coming from core, parts of it would be coming from streaming. Just like the UI. And

[GitHub] spark pull request #16189: [SPARK-18761][CORE][WIP] Introduce "task reaper" ...

2016-12-07 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/16189#discussion_r91337695 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -432,6 +435,57 @@ private[spark] class Executor( } /**

[GitHub] spark pull request #16189: [SPARK-18761][CORE][WIP] Introduce "task reaper" ...

2016-12-07 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/16189#discussion_r91337585 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -432,6 +435,57 @@ private[spark] class Executor( } /**

[GitHub] spark pull request #14638: [SPARK-11374][SQL] Support `skip.header.line.coun...

2016-12-07 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/14638#discussion_r91332545 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala --- @@ -122,10 +125,22 @@ class HadoopTableReader( val attrsWithIndex

[GitHub] spark issue #16169: [SPARK-18326][SPARKR][ML] Review SparkR ML wrappers API ...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16169 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69800/ Test PASSed. ---

[GitHub] spark issue #16169: [SPARK-18326][SPARKR][ML] Review SparkR ML wrappers API ...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16169 **[Test build #69800 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69800/consoleFull)** for PR 16169 at commit

[GitHub] spark pull request #14638: [SPARK-11374][SQL] Support `skip.header.line.coun...

2016-12-07 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/14638#discussion_r91331724 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala --- @@ -113,6 +113,9 @@ class HadoopTableReader( val tablePath

[GitHub] spark pull request #16175: [SPARK-17460][SQL]check if statistics.sizeInBytes...

2016-12-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16175#discussion_r91329867 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala --- @@ -1110,6 +1110,16 @@ class DatasetSuite extends QueryTest with

[GitHub] spark issue #16197: [SPARK-17760][SQL][Backport] AnalysisException with data...

2016-12-07 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16197 LGTM - pending jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16148: [SPARK-18325][SparkR][ML] SparkR ML wrappers example cod...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16148 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69799/ Test PASSed. ---

[GitHub] spark issue #16148: [SPARK-18325][SparkR][ML] SparkR ML wrappers example cod...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16148 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16148: [SPARK-18325][SparkR][ML] SparkR ML wrappers example cod...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16148 **[Test build #69799 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69799/consoleFull)** for PR 16148 at commit

[GitHub] spark issue #16169: [SPARK-18326][SPARKR][ML] Review SparkR ML wrappers API ...

2016-12-07 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/16169 @felixcheung This is a good question. It's reasonable to call ```.setProbabilityCol(probabilityCol)``` on a model or a pipeline model, and this should be handled at model transformation

[GitHub] spark issue #15987: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15987 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69797/ Test PASSed. ---

[GitHub] spark issue #15987: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15987 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15987: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15987 **[Test build #69797 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69797/consoleFull)** for PR 15987 at commit

[GitHub] spark issue #16169: [SPARK-18326][SPARKR][ML] Review SparkR ML wrappers API ...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16169 **[Test build #69800 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69800/consoleFull)** for PR 16169 at commit

[GitHub] spark pull request #16148: [SPARK-18325][SparkR][ML] SparkR ML wrappers exam...

2016-12-07 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/16148#discussion_r91314970 --- Diff: examples/src/main/r/ml/ml.R --- @@ -0,0 +1,63 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark issue #16148: [SPARK-18325][SparkR][ML] SparkR ML wrappers example cod...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16148 **[Test build #69799 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69799/consoleFull)** for PR 16148 at commit

[GitHub] spark pull request #16148: [SPARK-18325][SparkR][ML] SparkR ML wrappers exam...

2016-12-07 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/16148#discussion_r91313338 --- Diff: examples/src/main/r/ml/glm.R --- @@ -0,0 +1,56 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request #16196: [SPARK-18231] Optimise SizeEstimator implementati...

2016-12-07 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/16196#discussion_r91309679 --- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala --- @@ -119,8 +126,6 @@ object SizeEstimator extends Logging { }

[GitHub] spark pull request #16148: [SPARK-18325][SparkR][ML] SparkR ML wrappers exam...

2016-12-07 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/16148#discussion_r91310987 --- Diff: docs/sparkr.md --- @@ -512,39 +512,33 @@ head(teenagers) # Machine Learning -SparkR supports the following machine learning

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14638 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69796/ Test PASSed. ---

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14638 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14638 **[Test build #69796 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69796/consoleFull)** for PR 14638 at commit

[GitHub] spark pull request #16196: [SPARK-18231] Optimise SizeEstimator implementati...

2016-12-07 Thread a-roberts
Github user a-roberts commented on a diff in the pull request: https://github.com/apache/spark/pull/16196#discussion_r91309585 --- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala --- @@ -89,7 +90,13 @@ object SizeEstimator extends Logging { // A

[GitHub] spark pull request #16196: [SPARK-18231] Optimise SizeEstimator implementati...

2016-12-07 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/16196#discussion_r91309138 --- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala --- @@ -89,7 +90,13 @@ object SizeEstimator extends Logging { // A cache

[GitHub] spark issue #16197: [SPARK-17760][SQL][Backport] AnalysisException with data...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16197 **[Test build #69798 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69798/consoleFull)** for PR 16197 at commit

[GitHub] spark pull request #16197: [SPARK-17760][SQL][Backport] AnalysisException wi...

2016-12-07 Thread aray
GitHub user aray opened a pull request: https://github.com/apache/spark/pull/16197 [SPARK-17760][SQL][Backport] AnalysisException with dataframe pivot when groupBy column is not attribute ## What changes were proposed in this pull request? Backport of #16177 to branch-2.0

[GitHub] spark pull request #16196: [SPARK-18231] Optimise SizeEstimator implementati...

2016-12-07 Thread a-roberts
Github user a-roberts commented on a diff in the pull request: https://github.com/apache/spark/pull/16196#discussion_r91305043 --- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala --- @@ -243,47 +253,59 @@ object SizeEstimator extends Logging { arrSize

[GitHub] spark pull request #16196: [SPARK-18231] Optimise SizeEstimator implementati...

2016-12-07 Thread a-roberts
Github user a-roberts commented on a diff in the pull request: https://github.com/apache/spark/pull/16196#discussion_r91301705 --- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala --- @@ -89,7 +90,13 @@ object SizeEstimator extends Logging { // A

[GitHub] spark issue #14365: [SPARK-16628][SQL] Translate file-based relation schema ...

2016-12-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14365 It seems to exist some cases like the following (on the current master). On hive, ```sql CREATE TABLE t1 (a string) PARTITIONED BY (b string) STORED AS ORC; INSERT INTO

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14638 **[Test build #69796 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69796/consoleFull)** for PR 14638 at commit

[GitHub] spark issue #15987: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15987 **[Test build #69797 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69797/consoleFull)** for PR 15987 at commit

[GitHub] spark issue #16196: [SPARK-18231] Optimise SizeEstimator implementation

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16196 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16196: [SPARK-18231] Optimise SizeEstimator implementation

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16196 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69795/ Test FAILed. ---

[GitHub] spark issue #15987: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-12-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15987 Retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-12-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14638 Retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #15664: [SPARK-18123][SQL] Use db column names instead of...

2016-12-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/15664#discussion_r91287287 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala --- @@ -112,7 +112,25 @@ object JdbcUtils extends

[GitHub] spark issue #16196: [SPARK-18231] Optimise SizeEstimator implementation

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16196 **[Test build #69795 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69795/consoleFull)** for PR 16196 at commit

[GitHub] spark pull request #16193: [SPARK-18766] [SQL] Push Down Filter Through Batc...

2016-12-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16193#discussion_r91285472 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ExtractPythonUDFs.scala --- @@ -166,3 +174,40 @@ object ExtractPythonUDFs

[GitHub] spark pull request #16177: [SPARK-17760][SQL] AnalysisException with datafra...

2016-12-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16177 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16177: [SPARK-17760][SQL] AnalysisException with dataframe pivo...

2016-12-07 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16177 @aray could you open a backport for branch 2.0 if you feel that we should also fix it there? --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #16177: [SPARK-17760][SQL] AnalysisException with dataframe pivo...

2016-12-07 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16177 LGTM. Merging to master/2.1. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16193: [SPARK-18766] [SQL] Push Down Filter Through Batc...

2016-12-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16193#discussion_r91284534 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/python/BatchEvalPythonExecSuite.scala --- @@ -0,0 +1,99 @@ +/* + * Licensed

[GitHub] spark pull request #15722: [SPARK-18208] [Shuffle] Executor OOM due to a gro...

2016-12-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15722 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16006: [SPARK-18580] [DStreams] [external/kafka-0-10] Use spark...

2016-12-07 Thread omuravskiy
Github user omuravskiy commented on the issue: https://github.com/apache/spark/pull/16006 Tests added --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #15722: [SPARK-18208] [Shuffle] Executor OOM due to a growing Lo...

2016-12-07 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15722 LGTM. Merging to master/2.1. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16056: [SPARK-18623][SQL] Add `returnNullable` to `StaticInvoke...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16056 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69794/ Test PASSed. ---

[GitHub] spark issue #15722: [SPARK-18208] [Shuffle] Executor OOM due to a growing Lo...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15722 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69793/ Test PASSed. ---

[GitHub] spark issue #15722: [SPARK-18208] [Shuffle] Executor OOM due to a growing Lo...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15722 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15722: [SPARK-18208] [Shuffle] Executor OOM due to a growing Lo...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15722 **[Test build #69793 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69793/consoleFull)** for PR 15722 at commit

[GitHub] spark issue #16056: [SPARK-18623][SQL] Add `returnNullable` to `StaticInvoke...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16056 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16056: [SPARK-18623][SQL] Add `returnNullable` to `StaticInvoke...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16056 **[Test build #69794 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69794/consoleFull)** for PR 16056 at commit

[GitHub] spark issue #16190: [SPARK-18762][WEBUI] Web UI should be http:4040 instead ...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16190 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16190: [SPARK-18762][WEBUI] Web UI should be http:4040 instead ...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16190 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69790/ Test PASSed. ---

[GitHub] spark issue #16190: [SPARK-18762][WEBUI] Web UI should be http:4040 instead ...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16190 **[Test build #69790 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69790/consoleFull)** for PR 16190 at commit

[GitHub] spark issue #16043: [SPARK-18601][SQL] Simplify Create/Get complex expressio...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16043 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16043: [SPARK-18601][SQL] Simplify Create/Get complex expressio...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16043 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69788/ Test PASSed. ---

[GitHub] spark issue #16043: [SPARK-18601][SQL] Simplify Create/Get complex expressio...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16043 **[Test build #69788 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69788/consoleFull)** for PR 16043 at commit

[GitHub] spark pull request #16196: [SPARK-18231] Optimise SizeEstimator implementati...

2016-12-07 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/16196#discussion_r91268810 --- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala --- @@ -119,8 +126,6 @@ object SizeEstimator extends Logging { }

[GitHub] spark pull request #16196: [SPARK-18231] Optimise SizeEstimator implementati...

2016-12-07 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/16196#discussion_r91268937 --- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala --- @@ -89,7 +90,13 @@ object SizeEstimator extends Logging { // A cache

[GitHub] spark pull request #16196: [SPARK-18231] Optimise SizeEstimator implementati...

2016-12-07 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/16196#discussion_r91270660 --- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala --- @@ -243,47 +253,59 @@ object SizeEstimator extends Logging { arrSize

[GitHub] spark pull request #16196: [SPARK-18231] Optimise SizeEstimator implementati...

2016-12-07 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/16196#discussion_r91271510 --- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala --- @@ -316,62 +338,40 @@ object SizeEstimator extends Logging { */

[GitHub] spark pull request #16196: [SPARK-18231] Optimise SizeEstimator implementati...

2016-12-07 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/16196#discussion_r91270825 --- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala --- @@ -243,47 +253,59 @@ object SizeEstimator extends Logging { arrSize

[GitHub] spark issue #16192: [SPARK-18764][Core]Add a warning log when skipping a cor...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16192 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69785/ Test PASSed. ---

[GitHub] spark issue #16192: [SPARK-18764][Core]Add a warning log when skipping a cor...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16192 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16109 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69786/ Test PASSed. ---

[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16109 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16192: [SPARK-18764][Core]Add a warning log when skipping a cor...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16192 **[Test build #69785 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69785/consoleFull)** for PR 16192 at commit

[GitHub] spark issue #16122: [SPARK-18681][SQL] Fix filtering to compatible with part...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16122 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16109 **[Test build #69786 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69786/consoleFull)** for PR 16109 at commit

[GitHub] spark issue #16122: [SPARK-18681][SQL] Fix filtering to compatible with part...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16122 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69792/ Test PASSed. ---

[GitHub] spark issue #16122: [SPARK-18681][SQL] Fix filtering to compatible with part...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16122 **[Test build #69792 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69792/consoleFull)** for PR 16122 at commit

<    1   2   3   4   5   6   >