[GitHub] spark issue #16248: [SPARK-18810][SPARKR] SparkR install.spark does not work...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16248 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16248: [SPARK-18810][SPARKR] SparkR install.spark does not work...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16248 **[Test build #70009 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70009/consoleFull)** for PR 16248 at commit

[GitHub] spark issue #16248: [SPARK-18810][SPARKR] SparkR install.spark does not work...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16248 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70009/ Test PASSed. ---

[GitHub] spark issue #16104: [SPARK-18675][SQL] CTAS for hive serde table should work...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16104 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16104: [SPARK-18675][SQL] CTAS for hive serde table should work...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16104 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70006/ Test PASSed. ---

[GitHub] spark issue #16104: [SPARK-18675][SQL] CTAS for hive serde table should work...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16104 **[Test build #70006 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70006/consoleFull)** for PR 16104 at commit

[GitHub] spark pull request #16219: [SPARK-18790][SS] Keep a general offset history o...

2016-12-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16219 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16219: [SPARK-18790][SS] Keep a general offset history of strea...

2016-12-11 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16219 Thanks! Merging to master and 2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #13909: [SPARK-16213][SQL] Reduce runtime overhead of a program ...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13909 **[Test build #70010 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70010/consoleFull)** for PR 13909 at commit

[GitHub] spark issue #16219: [SPARK-18790][SS] Keep a general offset history of strea...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16219 **[Test build #3493 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3493/consoleFull)** for PR 16219 at commit

[GitHub] spark issue #16220: [SPARK-18796][SS]StreamingQueryManager should not block ...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16220 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70003/ Test PASSed. ---

[GitHub] spark issue #16220: [SPARK-18796][SS]StreamingQueryManager should not block ...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16220 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16220: [SPARK-18796][SS]StreamingQueryManager should not block ...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16220 **[Test build #70003 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70003/consoleFull)** for PR 16220 at commit

[GitHub] spark issue #16251: [SPARK-18826][SS]Add 'newestFirst' option to FileStreamS...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16251 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70002/ Test PASSed. ---

[GitHub] spark issue #16251: [SPARK-18826][SS]Add 'newestFirst' option to FileStreamS...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16251 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16251: [SPARK-18826][SS]Add 'newestFirst' option to FileStreamS...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16251 **[Test build #70002 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70002/consoleFull)** for PR 16251 at commit

[GitHub] spark pull request #15915: [SPARK-18485][CORE] Underlying integer overflow w...

2016-12-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15915#discussion_r91893984 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -331,7 +332,7 @@ private[spark] class MemoryStore( var

[GitHub] spark pull request #15915: [SPARK-18485][CORE] Underlying integer overflow w...

2016-12-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15915#discussion_r91892123 --- Diff: core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala --- @@ -78,6 +80,7 @@ private[spark] class TorrentBroadcast[T: ClassTag](obj:

[GitHub] spark issue #16248: [SPARK-18810][SPARKR] SparkR install.spark does not work...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16248 **[Test build #70009 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70009/consoleFull)** for PR 16248 at commit

[GitHub] spark pull request #16248: [SPARK-18810][SPARKR] SparkR install.spark does n...

2016-12-11 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/16248#discussion_r91891490 --- Diff: R/pkg/R/utils.R --- @@ -851,3 +851,12 @@ rbindRaws <- function(inputData){ out[!rawcolumns] <- lapply(out[!rawcolumns], unlist)

[GitHub] spark issue #16030: [SPARK-18108][SQL] Fix a bug to fail partition schema in...

2016-12-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16030 the new behavior LGTM, but I'm not sure if we still need to keep the old behavior --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #16219: [SPARK-18790][SS] Keep a general offset history of strea...

2016-12-11 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16219 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #16142: [SPARK-18716][CORE] Restrict the disk usage of sp...

2016-12-11 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/16142#discussion_r91889826 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -90,6 +91,10 @@ private[spark] class EventLoggingListener(

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-12-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14638 Do you mean to replace the current whole `TableReader.scala` which was introduced in SPARK-1251 ? I guessed Spark chose this direct access approach for the performance issue at that time.

[GitHub] spark pull request #15915: [SPARK-18485][CORE] Underlying integer overflow w...

2016-12-11 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/15915#discussion_r9169 --- Diff: core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala --- @@ -78,6 +80,7 @@ private[spark] class TorrentBroadcast[T:

[GitHub] spark issue #15915: [SPARK-18485][CORE] Underlying integer overflow when cre...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15915 **[Test build #70008 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70008/consoleFull)** for PR 15915 at commit

[GitHub] spark issue #15915: [SPARK-18485][CORE] Underlying integer overflow when cre...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15915 **[Test build #70007 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70007/consoleFull)** for PR 15915 at commit

[GitHub] spark issue #16245: [SPARK-18824][SQL] Add optimizer rule to reorder Filter ...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16245 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16245: [SPARK-18824][SQL] Add optimizer rule to reorder Filter ...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16245 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/6/ Test PASSed. ---

[GitHub] spark issue #16146: [SPARK-18091] [SQL] [BACKPORT-1.6] Deep if expressions c...

2016-12-11 Thread kapilsingh5050
Github user kapilsingh5050 commented on the issue: https://github.com/apache/spark/pull/16146 Yes, I'll do that but the test failures here are different. I'm still to figure out the root cause. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #16245: [SPARK-18824][SQL] Add optimizer rule to reorder Filter ...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16245 **[Test build #6 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/6/consoleFull)** for PR 16245 at commit

[GitHub] spark issue #16104: [SPARK-18675][SQL] CTAS for hive serde table should work...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16104 **[Test build #70006 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70006/consoleFull)** for PR 16104 at commit

[GitHub] spark issue #16146: [SPARK-18091] [SQL] [BACKPORT-1.6] Deep if expressions c...

2016-12-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16146 hi @kapilsingh5050 can you also include https://github.com/apache/spark/pull/16244? It does fix the maven tests. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #16104: [SPARK-18675][SQL] CTAS for hive serde table should work...

2016-12-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16104 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #16104: [SPARK-18675][SQL] CTAS for hive serde table shou...

2016-12-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16104#discussion_r91886939 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -121,21 +121,61 @@ case class

[GitHub] spark issue #16245: [SPARK-18824][SQL] Add optimizer rule to reorder Filter ...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16245 **[Test build #70005 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70005/consoleFull)** for PR 16245 at commit

[GitHub] spark pull request #16104: [SPARK-18675][SQL] CTAS for hive serde table shou...

2016-12-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16104#discussion_r91886458 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -121,21 +121,61 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request #16189: [SPARK-18761][CORE] Introduce "task reaper" to ov...

2016-12-11 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/16189#discussion_r91886381 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -432,6 +458,78 @@ private[spark] class Executor( } /**

[GitHub] spark issue #16189: [SPARK-18761][CORE] Introduce "task reaper" to oversee t...

2016-12-11 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/16189 @lins05, I'll see if there's a way to get a nicer executor exit status to be reported back to the driver. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #16252: [SPARK-18827][Core] Fix cannot read broadcast on disk

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16252 **[Test build #70004 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70004/consoleFull)** for PR 16252 at commit

[GitHub] spark pull request #16189: [SPARK-18761][CORE] Introduce "task reaper" to ov...

2016-12-11 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/16189#discussion_r91886287 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -432,6 +458,78 @@ private[spark] class Executor( } /**

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-12-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14638 Is there any high-level API that we can use to read hive tables? The current hive table reader is so low-level that we have to support features like `skip.header.line.count`, and I think it

[GitHub] spark pull request #16245: [SPARK-18824][SQL] Add optimizer rule to reorder ...

2016-12-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16245#discussion_r91886133 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -514,6 +514,25 @@ case class OptimizeCodegen(conf:

[GitHub] spark pull request #16252: [SPARK-18827][Core] Fix cannot read broadcast on ...

2016-12-11 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/16252 [SPARK-18827][Core] Fix cannot read broadcast on disk ## What changes were proposed in this pull request? Fix cannot read broadcast on disk ## How was this patch tested? Add

[GitHub] spark issue #15915: [SPARK-18485][CORE] Underlying integer overflow when cre...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15915 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15915: [SPARK-18485][CORE] Underlying integer overflow when cre...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15915 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70001/ Test FAILed. ---

[GitHub] spark issue #15915: [SPARK-18485][CORE] Underlying integer overflow when cre...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15915 **[Test build #70001 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70001/consoleFull)** for PR 15915 at commit

[GitHub] spark issue #16135: [SPARK-18700][SQL] Add StripedLock for each table's rela...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16135 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69998/ Test PASSed. ---

[GitHub] spark issue #16135: [SPARK-18700][SQL] Add StripedLock for each table's rela...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16135 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16135: [SPARK-18700][SQL] Add StripedLock for each table's rela...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16135 **[Test build #69998 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69998/consoleFull)** for PR 16135 at commit

[GitHub] spark issue #16219: [SPARK-18790][SS] Keep a general offset history of strea...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16219 **[Test build #3493 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3493/consoleFull)** for PR 16219 at commit

[GitHub] spark issue #16220: [SPARK-18796][SS]StreamingQueryManager should not block ...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16220 **[Test build #70003 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70003/consoleFull)** for PR 16220 at commit

[GitHub] spark issue #16220: [SPARK-18796][SS]StreamingQueryManager should not block ...

2016-12-11 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16220 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #16251: [SPARK-18826][SS]Add 'newestFirst' option to FileStreamS...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16251 **[Test build #70002 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70002/consoleFull)** for PR 16251 at commit

[GitHub] spark pull request #16251: [SPARK-18826][SS]Add 'newestFirst' option to File...

2016-12-11 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/16251 [SPARK-18826][SS]Add 'newestFirst' option to FileStreamSource ## What changes were proposed in this pull request? When starting a stream with a lot of backfill and maxFilesPerTrigger, the

[GitHub] spark pull request #16248: [SPARK-18810][SPARKR] SparkR install.spark does n...

2016-12-11 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/16248#discussion_r91883327 --- Diff: R/pkg/R/utils.R --- @@ -851,3 +851,12 @@ rbindRaws <- function(inputData){ out[!rawcolumns] <- lapply(out[!rawcolumns], unlist) out

[GitHub] spark pull request #16248: [SPARK-18810][SPARKR] SparkR install.spark does n...

2016-12-11 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/16248#discussion_r91883219 --- Diff: R/pkg/R/install.R --- @@ -104,7 +113,12 @@ install.spark <- function(hadoopVersion = "2.7", mirrorUrl = NULL, if (tarExists && !overwrite)

[GitHub] spark issue #16249: [SPARKR] Refactor scripts for R

2016-12-11 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/16249 Can we open a JIRA for this ? Its good to track this change --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #16214: [SPARK-18325][SPARKR] Add example for using nativ...

2016-12-11 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/16214#discussion_r91881854 --- Diff: examples/src/main/r/native-r-package.R --- @@ -0,0 +1,68 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark issue #16214: [SPARK-18325][SPARKR] Add example for using native R pac...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16214 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16214: [SPARK-18325][SPARKR] Add example for using native R pac...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16214 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/7/ Test PASSed. ---

[GitHub] spark issue #16214: [SPARK-18325][SPARKR] Add example for using native R pac...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16214 **[Test build #7 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/7/consoleFull)** for PR 16214 at commit

[GitHub] spark pull request #16245: [SPARK-18824][SQL] Add optimizer rule to reorder ...

2016-12-11 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/16245#discussion_r91881557 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -514,6 +514,25 @@ case class OptimizeCodegen(conf:

[GitHub] spark pull request #13909: [SPARK-16213][SQL] Reduce runtime overhead of a p...

2016-12-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/13909#discussion_r91881499 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/BufferHolder.java --- @@ -57,6 +60,12 @@ public

[GitHub] spark pull request #13909: [SPARK-16213][SQL] Reduce runtime overhead of a p...

2016-12-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/13909#discussion_r91881279 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -56,33 +58,93 @@ case class

[GitHub] spark issue #16195: [Spark-18765] [CORE] Make values for spark.yarn.{am|driv...

2016-12-11 Thread daisukebe
Github user daisukebe commented on the issue: https://github.com/apache/spark/pull/16195 @vanzin , 2.0 already has this capability per https://issues.apache.org/jira/browse/SPARK-529, thus my patch targets on 1.6. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request #16214: [SPARK-18325][SPARKR] Add example for using nativ...

2016-12-11 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/16214#discussion_r91881006 --- Diff: docs/sparkr.md --- @@ -472,21 +472,17 @@ should fit in a single machine. If that is not the case they can do something li `dapply`

[GitHub] spark pull request #13909: [SPARK-16213][SQL] Reduce runtime overhead of a p...

2016-12-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/13909#discussion_r91880888 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/BufferHolder.java --- @@ -18,8 +18,11 @@ package

[GitHub] spark pull request #13909: [SPARK-16213][SQL] Reduce runtime overhead of a p...

2016-12-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/13909#discussion_r91880870 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/BufferHolder.java --- @@ -18,8 +18,11 @@ package

[GitHub] spark pull request #15915: [SPARK-18485][CORE] Underlying integer overflow w...

2016-12-11 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/15915#discussion_r91880781 --- Diff: core/src/main/scala/org/apache/spark/memory/MemoryManager.scala --- @@ -223,8 +222,10 @@ private[spark] abstract class MemoryManager(

[GitHub] spark issue #16245: [SQL][WIP] Add optimizer rule to reorder Filter predicat...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16245 **[Test build #6 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/6/consoleFull)** for PR 16245 at commit

[GitHub] spark issue #16214: [SPARK-18325][SPARKR] Add example for using native R pac...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16214 **[Test build #7 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/7/consoleFull)** for PR 16214 at commit

[GitHub] spark issue #15915: [SPARK-18485][CORE] Underlying integer overflow when cre...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15915 **[Test build #70001 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70001/consoleFull)** for PR 15915 at commit

[GitHub] spark issue #16135: [SPARK-18700][SQL] Add StripedLock for each table's rela...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16135 **[Test build #69998 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69998/consoleFull)** for PR 16135 at commit

[GitHub] spark pull request #16135: [SPARK-18700][SQL] Add StripedLock for each table...

2016-12-11 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/16135#discussion_r91879183 --- Diff: core/src/main/scala/org/apache/spark/metrics/source/StaticSources.scala --- @@ -105,6 +111,7 @@ object HiveCatalogMetrics extends Source {

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-12-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14638 Thank you so much, @jamartinh , @srowen , @HyukjinKwon , and @gatorsmile . We can distinguish the two existing problems separately here. First, **a)** Spark returns incorrect

[GitHub] spark issue #16086: [SPARK-18653][SQL] Fix incorrect space padding for unico...

2016-12-11 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/16086 I am thinking about an simpler approach. However, it is fine to close for now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #15297: [SPARK-9862]Handling data skew

2016-12-11 Thread YuhuWang2002
Github user YuhuWang2002 commented on the issue: https://github.com/apache/spark/pull/15297 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15620: [SPARK-18091] [SQL] Deep if expressions cause Generated ...

2016-12-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15620 the test is good now https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-branch-2.0-test-maven-hadoop-2.2/ --- If your project is set up for it, you can reply

[GitHub] spark issue #11045: [SPARK-8321][SQL][WIP] Authorization Support(on all oper...

2016-12-11 Thread winningsix
Github user winningsix commented on the issue: https://github.com/apache/spark/pull/11045 @yaooqinn yes, the validation is working on server side. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #15915: [SPARK-18485][CORE] Underlying integer overflow when cre...

2016-12-11 Thread uncleGen
Github user uncleGen commented on the issue: https://github.com/apache/spark/pull/15915 @srowen Sorry for the delay, I will update it as soon as possible on the basis of comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #15904: [SPARK-18470][STREAMING][WIP] Provide Spark Streaming Mo...

2016-12-11 Thread uncleGen
Github user uncleGen commented on the issue: https://github.com/apache/spark/pull/15904 @vanzin Sorry for the delay, I will update as soon as possible on the basis of your comments. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #16240: [SPARK-16792][SQL] Dataset containing a Case Class with ...

2016-12-11 Thread michalsenkyr
Github user michalsenkyr commented on the issue: https://github.com/apache/spark/pull/16240 Possible optimization: Instead of conversions using `to`, we can use `Builder`s. This way we could get rid of the conversion overhead. This would require adding a new codegen method that would

[GitHub] spark issue #16249: [SPARKR] Refactor scripts for R

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16249 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16249: [SPARKR] Refactor scripts for R

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16249 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69996/ Test PASSed. ---

[GitHub] spark issue #16249: [SPARKR] Refactor scripts for R

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16249 **[Test build #69996 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69996/consoleFull)** for PR 16249 at commit

[GitHub] spark issue #16180: [DOCS][MINOR] Clarify Where AccumulatorV2s are Displayed

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16180 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69997/ Test PASSed. ---

[GitHub] spark issue #16180: [DOCS][MINOR] Clarify Where AccumulatorV2s are Displayed

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16180 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16180: [DOCS][MINOR] Clarify Where AccumulatorV2s are Displayed

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16180 **[Test build #69997 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69997/consoleFull)** for PR 16180 at commit

[GitHub] spark issue #13909: [SPARK-16213][SQL] Reduce runtime overhead of a program ...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13909 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13909: [SPARK-16213][SQL] Reduce runtime overhead of a program ...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13909 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69995/ Test PASSed. ---

[GitHub] spark issue #13909: [SPARK-16213][SQL] Reduce runtime overhead of a program ...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13909 **[Test build #69995 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69995/consoleFull)** for PR 13909 at commit

[GitHub] spark issue #16180: [DOCS][MINOR] Clarify Where AccumulatorV2s are Displayed

2016-12-11 Thread anabranch
Github user anabranch commented on the issue: https://github.com/apache/spark/pull/16180 @srowen completed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #16180: [DOCS][MINOR] Clarify Where AccumulatorV2s are Displayed

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16180 **[Test build #69997 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69997/consoleFull)** for PR 16180 at commit

[GitHub] spark pull request #16180: [DOCS][MINOR] Clarify Where AccumulatorV2s are Di...

2016-12-11 Thread anabranch
Github user anabranch commented on a diff in the pull request: https://github.com/apache/spark/pull/16180#discussion_r91865764 --- Diff: docs/programming-guide.md --- @@ -1345,14 +1345,17 @@ therefore be efficiently supported in parallel. They can be used to implement co

[GitHub] spark issue #16249: [SPARKR] Refactor scripts for R

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16249 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69994/ Test PASSed. ---

[GitHub] spark issue #16249: [SPARKR] Refactor scripts for R

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16249 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16249: [SPARKR] Refactor scripts for R

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16249 **[Test build #69994 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69994/consoleFull)** for PR 16249 at commit

[GitHub] spark issue #16250: [CORE][MINOR] Stylistic changes in DAGScheduler (to ease...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16250 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16250: [CORE][MINOR] Stylistic changes in DAGScheduler (to ease...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16250 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69991/ Test PASSed. ---

  1   2   3   >