[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19082 Nope. You can wait for the answers from the JVM experts. : ) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19287: [SPARK-22074][Core] Task killed by other attempt task sh...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19287 **[Test build #82546 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82546/testReport)** for PR 19287 at commit [`1c8c849`](https://github.com/apache/spark/commit/1c8c84937e85302f2ac48bcbdbdb5507c9b445e4). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19287: [SPARK-22074][Core] Task killed by other attempt task sh...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19287 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82546/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19287: [SPARK-22074][Core] Task killed by other attempt task sh...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19287 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19082 Ok. Any comments from @kiszk and @rednaxelafx on this topic? :) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19287: [SPARK-22074][Core] Task killed by other attempt task sh...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19287 LGTM, merging to master. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #19287: [SPARK-22074][Core] Task killed by other attempt ...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19287 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #19444: [SPARK-22214][SQL] Refactor the list hive partiti...
Github user jiangxb1987 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19444#discussion_r143390226
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala
---
@@ -405,6 +405,11 @@ object CatalogTypes {
* Specifications of a table partition. Mapping column name to column
value.
*/
type TablePartitionSpec = Map[String, String]
+
+ /**
+ * Initialize an empty spec.
+ */
+ lazy val emptyTablePartitionSpec: TablePartitionSpec = Map.empty[String,
String]
--- End diff --
We wanted to refer the val `emptyTablePartitionSpec` as
`TablePartitionSpec`, not `Map[String, String]`, though they are equal.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19399: [SPARK-22175][WEB-UI] Add status column to history page
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/19399 Nice, I think it should be merged. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19082 The relation is very clear. Let me copy what I said above > Just imagine we have two nodes, we want to do a horizontal/ring cut. Basically, in this scenario, horizontal/ring cutting means whether we do a whole-stage codegen or not. After horizontal/ring cutting, each operator will be in a single blocking loop. That means, whole-stage codegen is off. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19287: [SPARK-22074][Core] Task killed by other attempt task sh...
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/19287 Thanks all reviewers! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19082 Yeah, once we exceed `hugeMethodLimit`, wholestage codegen is disabled. Can you tell me how `hugeMethodLimit` makes each operator to a single blocking loop? We didn't run codegen now. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19082 > After horizontal/ring cutting, each operator will be in a single blocking loop. That means, whole-stage codegen is off. And do we have anything to cut the operators to individual single blocking loop? I didn't see we have anything to do this. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19145: [spark-21933][yarn] Spark Streaming request more executo...
Github user klion26 commented on the issue: https://github.com/apache/spark/pull/19145 sorry for the late response, IIUC, in MR, this case handled by below 1. AM received the container failed message 2. AM will check whether there are any attempts of the same task is RUNNING or SUCCEED 2.1 If step 2 returns true, then MR ignores the failed message 2.2 if step 2 returns false, then MR will request a new container and run the specified task --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19082 Sorry, it is really hard to explain it to you. Maybe you can first think about why the whole-stage codegen works better the previous solution? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19082 No. I think you should explain it because this is your concern. You just need to point out, where is the single blocking loop for the two operators? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19082 Just try to help you understand it. If you don't want it, I can keep quiet. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19082 @gatorsmile You can't just drop few words without explanation and simply say that others works don't work. I do respect your work/comments and you should respect my works too. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19082 If you really can't or don't willing to explain, we can wait for @kiszk or @rednaxelafx to explain it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #19420: [SPARK-22191] [SQL] Add hive serde example with s...
Github user crlalam commented on a diff in the pull request:
https://github.com/apache/spark/pull/19420#discussion_r143118902
--- Diff:
examples/src/main/java/org/apache/spark/examples/sql/hive/JavaSparkHiveExample.java
---
@@ -124,6 +124,13 @@ public static void main(String[] args) {
// ...
// $example off:spark_hive$
+// Hive serde's are also supported with serde properties.
+ String sqlQuery = "CREATE TABLE src_serde(key decimal(38,18), value
int) USING hive"
--- End diff --
Hi @dongjoon-hyun ,
Thanks for the review and pointing me to the coding style. Fixed coding
style issues and updated the pull request. Please take a look.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19082 OK. Based on your attitudes, I do not care it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
