[GitHub] spark issue #21589: [SPARK-24591][CORE] Number of cores and executors in the...

2018-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21589 **[Test build #92907 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92907/testReport)** for PR 21589 at commit

[GitHub] spark issue #21745: [SPARK-24781][SQL] Using a reference from Dataset in Fil...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21745 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21745: [SPARK-24781][SQL] Using a reference from Dataset in Fil...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21745 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92911/ Test FAILed. ---

[GitHub] spark issue #21745: [SPARK-24781][SQL] Using a reference from Dataset in Fil...

2018-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21745 **[Test build #92911 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92911/testReport)** for PR 21745 at commit

[GitHub] spark pull request #21752: [SPARK-24788][SQL] fixed UnresolvedException when...

2018-07-11 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21752#discussion_r201916394 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala --- @@ -717,4 +717,9 @@ class DataFrameAggregateSuite extends QueryTest

[GitHub] spark pull request #21752: [SPARK-24788][SQL] fixed UnresolvedException when...

2018-07-11 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21752#discussion_r201916183 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala --- @@ -717,4 +717,9 @@ class DataFrameAggregateSuite extends QueryTest

[GitHub] spark pull request #21752: [SPARK-24788][SQL] fixed UnresolvedException when...

2018-07-11 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21752#discussion_r201916132 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala --- @@ -717,4 +717,9 @@ class DataFrameAggregateSuite extends QueryTest

[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...

2018-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21747 **[Test build #92914 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92914/testReport)** for PR 21747 at commit

[GitHub] spark issue #21156: [SPARK-24087][SQL] Avoid shuffle when join keys are a su...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21156 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21752: [SPARK-24788][SQL] fixed UnresolvedException when...

2018-07-11 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21752#discussion_r201915580 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala --- @@ -717,4 +717,9 @@ class DataFrameAggregateSuite extends QueryTest

[GitHub] spark issue #21156: [SPARK-24087][SQL] Avoid shuffle when join keys are a su...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21156 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92909/ Test PASSed. ---

[GitHub] spark issue #21156: [SPARK-24087][SQL] Avoid shuffle when join keys are a su...

2018-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21156 **[Test build #92909 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92909/testReport)** for PR 21156 at commit

[GitHub] spark issue #21752: [SPARK-24788][SQL] fixed UnresolvedException when toStri...

2018-07-11 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21752 Can you describe the issue detail instead of the jira link? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #21745: [SPARK-24781][SQL] Using a reference from Dataset in Fil...

2018-07-11 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21745 Actually, the very first time we introduce this bug was at 7463a88be6eecf14ba0a87048a975a06b03b928b. We added `!f.resolved &&` in `ResolveMissingReferences` rule. ---

[GitHub] spark issue #21749: [SPARK-24785] [SHELL] Making sure REPL prints Spark UI i...

2018-07-11 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21749 @skonto perhaps you folks can give some guidance on this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #21749: [SPARK-24785] [SHELL] Making sure REPL prints Spark UI i...

2018-07-11 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21749 agreed, it's not ideal to use reflection... --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...

2018-07-11 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21747 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21710: [SPARK-24207][R]add R API for PrefixSpan

2018-07-11 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21710 we could, but we tend to forget about them until the release, so I'd prefer we don't and have everything in one PR. not ideal, we could leave out code example, or maybe even vigenette,

[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21747 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21747 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92912/ Test FAILed. ---

[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...

2018-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21747 **[Test build #92912 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92912/testReport)** for PR 21747 at commit

[GitHub] spark issue #21745: [SPARK-24781][SQL] Using a reference from Dataset in Fil...

2018-07-11 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21745 I tried to checkout the commit 82183f7b57f2a93e646c56a9e37fac64b348ff0b which is before AnalysisBarrier commit. ```scala scala> val df = Seq(("test1", 0), ("test2", 1)).toDF("name",

[GitHub] spark pull request #21745: [SPARK-24781][SQL] Using a reference from Dataset...

2018-07-11 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21745#discussion_r201914482 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1163,7 +1165,8 @@ class Analyzer( case

[GitHub] spark issue #21741: [SPARK-24718][SQL] Timestamp support pushdown to parquet...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21741 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92908/ Test PASSed. ---

[GitHub] spark issue #21741: [SPARK-24718][SQL] Timestamp support pushdown to parquet...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21741 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21589: [SPARK-24591][CORE] Number of cores and executors...

2018-07-11 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21589#discussion_r201914125 --- Diff: R/pkg/R/context.R --- @@ -435,3 +435,31 @@ setCheckpointDir <- function(directory) { sc <- getSparkContext()

[GitHub] spark issue #21741: [SPARK-24718][SQL] Timestamp support pushdown to parquet...

2018-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21741 **[Test build #92908 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92908/testReport)** for PR 21741 at commit

[GitHub] spark pull request #21589: [SPARK-24591][CORE] Number of cores and executors...

2018-07-11 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21589#discussion_r201913817 --- Diff: R/pkg/R/context.R --- @@ -435,3 +435,31 @@ setCheckpointDir <- function(directory) { sc <- getSparkContext()

[GitHub] spark pull request #21589: [SPARK-24591][CORE] Number of cores and executors...

2018-07-11 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21589#discussion_r201913578 --- Diff: R/pkg/R/context.R --- @@ -435,3 +435,31 @@ setCheckpointDir <- function(directory) { sc <- getSparkContext()

[GitHub] spark pull request #21753: [SPARK-24790][SQL] Allow complex aggregate expres...

2018-07-11 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21753#discussion_r201912569 --- Diff: sql/core/src/test/resources/sql-tests/results/pivot.sql.out --- @@ -192,3 +192,33 @@ struct<> -- !query 12 output

[GitHub] spark issue #21753: [SPARK-24790][SQL] Allow complex aggregate expressions i...

2018-07-11 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21753 Actually, this is a bug? In the current master, the exception says; ``` org.apache.spark.sql.AnalysisException Aggregate expression required for pivot, found 'CEIL(sum(cast(earnings#x as

[GitHub] spark issue #21745: [SPARK-24781][SQL] Using a reference from Dataset in Fil...

2018-07-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21745 We might need to get rid of AnalysisBarrier in the next release. This already caused at least three regressions in 2.3 --- -

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-07-11 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r201904049 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3261,3 +3261,322 @@ case class

[GitHub] spark issue #21667: [SPARK-24691][SQL]Dispatch the type support check in Fil...

2018-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21667 **[Test build #92913 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92913/testReport)** for PR 21667 at commit

[GitHub] spark issue #21667: [SPARK-24691][SQL]Dispatch the type support check in Fil...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21667 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21667: [SPARK-24691][SQL]Dispatch the type support check in Fil...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21667 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/889/

[GitHub] spark pull request #21704: [SPARK-24734][SQL] Fix containsNull of Concat for...

2018-07-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21704#discussion_r201902635 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -48,7 +48,8 @@ case class

[GitHub] spark issue #21745: [SPARK-24781][SQL] Using a reference from Dataset in Fil...

2018-07-11 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21745 @gatorsmile It seems the AnalysisBarrier commit causes this error, so v2.2 does not have this issue; ``` scala> df.select(df("name")).filter(df("id") === 0).explain(true) == Parsed

[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...

2018-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21747 **[Test build #92912 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92912/testReport)** for PR 21747 at commit

[GitHub] spark pull request #21704: [SPARK-24734][SQL] Fix containsNull of Concat for...

2018-07-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21704#discussion_r201902242 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -610,27 +636,27 @@ object TypeCoercion {

[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...

2018-07-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21747 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...

2018-07-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21747 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21745: [SPARK-24781][SQL] Using a reference from Dataset in Fil...

2018-07-11 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21745 Sorry replying via email. The previously failed test case has a GROUPING with resolved references. Since it's unresolved itself, the rule will go through underlying Project and

[GitHub] spark pull request #21745: [SPARK-24781][SQL] Using a reference from Dataset...

2018-07-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21745#discussion_r201901412 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1163,7 +1165,8 @@ class Analyzer(

[GitHub] spark issue #21061: [SPARK-23914][SQL] Add array_union function

2018-07-11 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21061 SGTM On Thu, Jul 12, 2018, 12:19 PM Takuya UESHIN wrote: > *@ueshin* commented on this pull request. > -- > > In >

[GitHub] spark pull request #21745: [SPARK-24781][SQL] Using a reference from Dataset...

2018-07-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21745#discussion_r201900877 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1163,7 +1165,8 @@ class Analyzer(

[GitHub] spark pull request #21704: [SPARK-24734][SQL] Fix containsNull of Concat for...

2018-07-11 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21704#discussion_r201900374 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -533,15 +559,15 @@ object TypeCoercion {

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-07-11 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r201898932 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3261,3 +3261,322 @@ case class

[GitHub] spark issue #21608: [SPARK-24626] [SQL] Improve location size calculation in...

2018-07-11 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21608 you'd be better to do that. cc: @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21745: [SPARK-24781][SQL] Using a reference from Dataset in Fil...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21745 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21745: [SPARK-24781][SQL] Using a reference from Dataset in Fil...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21745 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/888/

[GitHub] spark issue #21745: [SPARK-24781][SQL] Using a reference from Dataset in Fil...

2018-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21745 **[Test build #92911 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92911/testReport)** for PR 21745 at commit

[GitHub] spark issue #21700: [SPARK-24717][SS] Split out max retain version of state ...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21700 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92904/ Test PASSed. ---

[GitHub] spark issue #21700: [SPARK-24717][SS] Split out max retain version of state ...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21700 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21754: [SPARK-24705][SQL] Cannot reuse an exchange operator wit...

2018-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21754 **[Test build #92910 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92910/testReport)** for PR 21754 at commit

[GitHub] spark issue #21754: [SPARK-24705][SQL] Cannot reuse an exchange operator wit...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21754 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/887/

[GitHub] spark issue #21700: [SPARK-24717][SS] Split out max retain version of state ...

2018-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21700 **[Test build #92904 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92904/testReport)** for PR 21700 at commit

[GitHub] spark issue #21754: [SPARK-24705][SQL] Cannot reuse an exchange operator wit...

2018-07-11 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21754 spark-2.2 and 2.3 has the same issue, so we probably need to backport this fix there, too. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21754: [SPARK-24705][SQL] Cannot reuse an exchange operator wit...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21754 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21754: [SPARK-24705][SQL] Cannot reuse an exchange opera...

2018-07-11 Thread maropu
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/21754 [SPARK-24705][SQL] Cannot reuse an exchange operator with an adaptive execution coordinator ## What changes were proposed in this pull request? This pr fixed a bug to wrongly reuse an exchange

[GitHub] spark issue #21753: [SPARK-24790][SQL] Allow complex aggregate expressions i...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21753 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21753: [SPARK-24790][SQL] Allow complex aggregate expressions i...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21753 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92901/ Test PASSed. ---

[GitHub] spark issue #21131: [SPARK-23433][CORE] Late zombie task completions update ...

2018-07-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21131 hmm, will we have a problem for shuffle here? Assuming a shuffle stage has 2 task sets, one is active, one is zombie. Both of them have running tasks. If a task from zombie task set

[GitHub] spark issue #21753: [SPARK-24790][SQL] Allow complex aggregate expressions i...

2018-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21753 **[Test build #92901 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92901/testReport)** for PR 21753 at commit

[GitHub] spark issue #21131: [SPARK-23433][CORE] Late zombie task completions update ...

2018-07-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21131 hmm, will we have a problem for shuffle here? Assuming a shuffle stage has 2 tasksets, one is zombie, one is normal. Both of them have running tasks. if a task in zombie taskset finishes,

[GitHub] spark issue #21719: [SPARK-24747][ML] Make Instrumentation class more flexib...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21719 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21719: [SPARK-24747][ML] Make Instrumentation class more flexib...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21719 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92906/ Test FAILed. ---

[GitHub] spark issue #21719: [SPARK-24747][ML] Make Instrumentation class more flexib...

2018-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21719 **[Test build #92906 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92906/testReport)** for PR 21719 at commit

[GitHub] spark issue #18139: [SPARK-20787][PYTHON] PySpark can't handle datetimes bef...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18139 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21131: [SPARK-23433][CORE] Late zombie task completions update ...

2018-07-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21131 ah i see. Does it only apply to the result stage? IIRC shuffle stage tracks shuffle epoch and will ignore the tasks from a killed stage. ---

[GitHub] spark pull request #21741: [SPARK-24718][SQL] Timestamp support pushdown to ...

2018-07-11 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21741#discussion_r201889164 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala --- @@ -387,6 +389,82 @@ class

[GitHub] spark pull request #21542: [SPARK-24529][Build][test-maven] Add spotbugs int...

2018-07-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21542 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21741: [SPARK-24718][SQL] Timestamp support pushdown to parquet...

2018-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21741 **[Test build #92908 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92908/testReport)** for PR 21741 at commit

[GitHub] spark issue #21156: [SPARK-24087][SQL] Avoid shuffle when join keys are a su...

2018-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21156 **[Test build #92909 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92909/testReport)** for PR 21156 at commit

[GitHub] spark issue #21741: [SPARK-24718][SQL] Timestamp support pushdown to parquet...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21741 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21741: [SPARK-24718][SQL] Timestamp support pushdown to parquet...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21741 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/886/

[GitHub] spark issue #21156: [SPARK-24087][SQL] Avoid shuffle when join keys are a su...

2018-07-11 Thread yucai
Github user yucai commented on the issue: https://github.com/apache/spark/pull/21156 @maryannxue how about this way? Any better idea? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21542: [SPARK-24529][Build][test-maven] Add spotbugs into maven...

2018-07-11 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21542 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21589: [SPARK-24591][CORE] Number of cores and executors...

2018-07-11 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21589#discussion_r201887843 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -2336,6 +2336,18 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark issue #21589: [SPARK-24591][CORE] Number of cores and executors in the...

2018-07-11 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21589 LGTM otherwise --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21589: [SPARK-24591][CORE] Number of cores and executors...

2018-07-11 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21589#discussion_r201887513 --- Diff: core/src/main/scala/org/apache/spark/api/java/JavaSparkContext.scala --- @@ -128,6 +128,18 @@ class JavaSparkContext(val sc: SparkContext)

[GitHub] spark pull request #21589: [SPARK-24591][CORE] Number of cores and executors...

2018-07-11 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21589#discussion_r201887443 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -2336,6 +2336,18 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark issue #21719: [SPARK-24747][ML] Make Instrumentation class more flexib...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21719 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21719: [SPARK-24747][ML] Make Instrumentation class more flexib...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21719 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/885/

[GitHub] spark pull request #21589: [SPARK-24591][CORE] Number of cores and executors...

2018-07-11 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21589#discussion_r201887249 --- Diff: R/pkg/R/context.R --- @@ -435,3 +435,31 @@ setCheckpointDir <- function(directory) { sc <- getSparkContext()

[GitHub] spark pull request #21589: [SPARK-24591][CORE] Number of cores and executors...

2018-07-11 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21589#discussion_r201887142 --- Diff: python/pyspark/context.py --- @@ -406,6 +406,22 @@ def defaultMinPartitions(self): """ return

[GitHub] spark pull request #21589: [SPARK-24591][CORE] Number of cores and executors...

2018-07-11 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21589#discussion_r201887215 --- Diff: python/pyspark/context.py --- @@ -406,6 +406,22 @@ def defaultMinPartitions(self): """ return

[GitHub] spark issue #21719: [SPARK-24747][ML] Make Instrumentation class more flexib...

2018-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21719 **[Test build #92906 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92906/testReport)** for PR 21719 at commit

[GitHub] spark issue #21589: [SPARK-24591][CORE] Number of cores and executors in the...

2018-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21589 **[Test build #92907 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92907/testReport)** for PR 21589 at commit

[GitHub] spark pull request #21184: [WIP][SPARK-24051][SQL] Replace Aliases with the ...

2018-07-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21184#discussion_r201886545 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -284,6 +288,80 @@ class Analyzer( } }

[GitHub] spark issue #21589: [SPARK-24591][CORE] Number of cores and executors in the...

2018-07-11 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21589 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21131: [SPARK-23433][CORE] Late zombie task completions update ...

2018-07-11 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/21131 >>Fetch failure lead to multiple tasksets which are active for a given stage. >How can this happen? the TaskSetManager will mark itself as zombie when it receives a fetch failed. We

[GitHub] spark issue #21440: [SPARK-24307][CORE] Support reading remote cached partit...

2018-07-11 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/21440 @mridulm @jerryshao maybe you would be interested in reviewing this as well? --- - To unsubscribe, e-mail:

[GitHub] spark issue #21583: [SPARK-23984][K8S][Test] Added Integration Tests for PyS...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21583 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21583: [SPARK-23984][K8S][Test] Added Integration Tests for PyS...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21583 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/884/

[GitHub] spark issue #21583: [SPARK-23984][K8S][Test] Added Integration Tests for PyS...

2018-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21583 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/884/ ---

[GitHub] spark pull request #21730: [SPARK-24761][SQL] Adding of isModifiable() to Ru...

2018-07-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21730 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21589: [SPARK-24591][CORE] Number of cores and executors in the...

2018-07-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21589 cc @jiangxb1987 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21751: [SPARK-24208][SQL][FOLLOWUP] Move test cases to p...

2018-07-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21751#discussion_r201879749 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala --- @@ -557,4 +558,21 @@ class AnalysisSuite extends

[GitHub] spark pull request #21736: [SPARK-24782][SQL] Simplify conf retrieval in SQL...

2018-07-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21736 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

  1   2   3   4   5   6   7   >