[GitHub] spark pull request #19698: [SPARK-20648][core] Port JobsTab and StageTab to ...

2017-11-13 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19698#discussion_r150622074 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/JobPage.scala --- @@ -182,173 +185,183 @@ private[ui] class JobPage(parent: JobsTab) extends

[GitHub] spark pull request #19698: [SPARK-20648][core] Port JobsTab and StageTab to ...

2017-11-13 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19698#discussion_r150631869 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -856,22 +832,24 @@ private[ui] class TaskTableRowData( val logs: Map

[GitHub] spark pull request #19698: [SPARK-20648][core] Port JobsTab and StageTab to ...

2017-11-13 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19698#discussion_r150632556 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -1036,8 +1006,13 @@ private[ui] class TaskDataSource( shuffleRead

[GitHub] spark pull request #19698: [SPARK-20648][core] Port JobsTab and StageTab to ...

2017-11-13 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19698#discussion_r150626334 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/PoolPage.scala --- @@ -17,50 +17,49 @@ package org.apache.spark.ui.jobs +import

[GitHub] spark pull request #19698: [SPARK-20648][core] Port JobsTab and StageTab to ...

2017-11-13 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19698#discussion_r150622874 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/JobPage.scala --- @@ -182,173 +185,183 @@ private[ui] class JobPage(parent: JobsTab) extends

[GitHub] spark pull request #19698: [SPARK-20648][core] Port JobsTab and StageTab to ...

2017-11-13 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19698#discussion_r150564678 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -23,19 +23,21 @@ import javax.servlet.http.HttpServletRequest

[GitHub] spark pull request #19698: [SPARK-20648][core] Port JobsTab and StageTab to ...

2017-11-13 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19698#discussion_r150573812 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -285,106 +272,120 @@ private[ui] class AllJobsPage(parent: JobsTab) extends

[GitHub] spark issue #19681: [SPARK-20652][sql] Store SQL UI data in the new app stat...

2017-11-12 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19681 lgtm need to fix the merge conflict though --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #19702: [SPARK-10365][SQL] Support Parquet logical type TIMESTAM...

2017-11-09 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19702 ok sorry for taking a bit of time to look at this. tl;dr I think what you are proposing here is fine. So to be honest, I'm a bit confused about the parquet versioning -- the types I

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-11-09 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19250 @cloud-fan yes I can do that, will be next week before I get to it --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #19679: [SPARK-20647][core] Port StorageTab to the new UI backen...

2017-11-09 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19679 merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #19681: [SPARK-20652][sql] Store SQL UI data in the new a...

2017-11-09 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19681#discussion_r150094936 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListener.scala --- @@ -0,0 +1,367 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #19681: [SPARK-20652][sql] Store SQL UI data in the new a...

2017-11-09 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19681#discussion_r150091458 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLAppStatusStore.scala --- @@ -0,0 +1,172 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19681: [SPARK-20652][sql] Store SQL UI data in the new app stat...

2017-11-09 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19681 >> Are there any tests for the SQL UI in the history server? >No, because those tests are in core. I want to add some SHS integration tests into a separate module after this w

[GitHub] spark pull request #19703: [SPARK-22403][SS] Add optional checkpointLocation...

2017-11-09 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19703#discussion_r150087448 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/streaming/StructuredKafkaWordCount.scala --- @@ -46,11 +51,13 @@ object

[GitHub] spark pull request #19703: [SPARK-22403][SS] Add optional checkpointLocation...

2017-11-09 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19703#discussion_r150029549 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/streaming/StructuredKafkaWordCount.scala --- @@ -46,11 +51,13 @@ object

[GitHub] spark issue #19703: [SPARK-22403][SS] Add optional checkpointLocation argume...

2017-11-09 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19703 Jenkins, add to whitelist --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #19702: [SPARK-10365][SQL] Support Parquet logical type TIMESTAM...

2017-11-09 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19702 hey thanks for doing this @cloud-fan but I have a small request -- can we get another day to review how this works, especially in connection with somewhat recent changes in parquet to include

[GitHub] spark pull request #19681: [SPARK-20652][sql] Store SQL UI data in the new a...

2017-11-08 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19681#discussion_r149767509 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListener.scala --- @@ -0,0 +1,353 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #19681: [SPARK-20652][sql] Store SQL UI data in the new a...

2017-11-08 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19681#discussion_r149780801 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala --- @@ -420,21 +428,19 @@ class SQLMetricsSuite extends

[GitHub] spark pull request #19681: [SPARK-20652][sql] Store SQL UI data in the new a...

2017-11-08 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19681#discussion_r149778794 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLListener.scala --- @@ -89,398 +83,3 @@ private class LongLongTupleConverter extends

[GitHub] spark pull request #19681: [SPARK-20652][sql] Store SQL UI data in the new a...

2017-11-08 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19681#discussion_r149767574 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListener.scala --- @@ -0,0 +1,353 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #19681: [SPARK-20652][sql] Store SQL UI data in the new a...

2017-11-08 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19681#discussion_r149767731 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLAppStatusStore.scala --- @@ -0,0 +1,172 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #19681: [SPARK-20652][sql] Store SQL UI data in the new a...

2017-11-07 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19681#discussion_r149578181 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListener.scala --- @@ -40,7 +40,7 @@ private[sql] class

[GitHub] spark pull request #19681: [SPARK-20652][sql] Store SQL UI data in the new a...

2017-11-07 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19681#discussion_r149578074 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListener.scala --- @@ -0,0 +1,353 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19678: [SPARK-20646][core] Port executors page to new UI backen...

2017-11-07 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19678 merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #19681: [SPARK-20652][sql] Store SQL UI data in the new a...

2017-11-07 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19681#discussion_r149529181 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListener.scala --- @@ -0,0 +1,353 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #19681: [SPARK-20652][sql] Store SQL UI data in the new a...

2017-11-07 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19681#discussion_r149530162 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListener.scala --- @@ -0,0 +1,353 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19678: [SPARK-20646][core] Port executors page to new UI backen...

2017-11-07 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19678 lgtm (assuming tests pass) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-11-07 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19250 > I think we can follow what Hive/Impala did for interoperability, i.e. create a config to interpret parquet INT96 as timezone-agnostic timestamp in parquet reader of Spark. If I underst

[GitHub] spark issue #19677: [SPARK-20645][core] Port environment page to new UI back...

2017-11-07 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19677 merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #19681: [SPARK-20652][sql] Store SQL UI data in the new a...

2017-11-07 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19681#discussion_r149500473 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListener.scala --- @@ -0,0 +1,353 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #19681: [SPARK-20652][sql] Store SQL UI data in the new a...

2017-11-07 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19681#discussion_r149457439 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListener.scala --- @@ -0,0 +1,353 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #19681: [SPARK-20652][sql] Store SQL UI data in the new a...

2017-11-07 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19681#discussion_r149458539 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLListenerSuite.scala --- @@ -36,13 +36,14 @@ import

[GitHub] spark pull request #19681: [SPARK-20652][sql] Store SQL UI data in the new a...

2017-11-07 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19681#discussion_r149450470 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListener.scala --- @@ -0,0 +1,353 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19679: [SPARK-20647][core] Port StorageTab to the new UI backen...

2017-11-07 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19679 I figured out what is going on with StreamBlocks now, both before and after this change, so I'm OK with it --- - To unsubscribe

[GitHub] spark pull request #19679: [SPARK-20647][core] Port StorageTab to the new UI...

2017-11-06 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19679#discussion_r149277525 --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusListenerSuite.scala --- @@ -685,12 +700,55 @@ class AppStatusListenerSuite extends

[GitHub] spark issue #19678: [SPARK-20646][core] Port executors page to new UI backen...

2017-11-06 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19678 Also going to copy this comment from here for others: https://github.com/vanzin/spark/pull/45 >> squito 12 days ago >> what do you think about adding a helper method fo

[GitHub] spark pull request #19678: [SPARK-20646][core] Port executors page to new UI...

2017-11-06 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19678#discussion_r149275806 --- Diff: core/src/main/scala/org/apache/spark/ui/exec/ExecutorsTab.scala --- @@ -17,192 +17,46 @@ package org.apache.spark.ui.exec

[GitHub] spark pull request #19678: [SPARK-20646][core] Port executors page to new UI...

2017-11-06 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19678#discussion_r149276157 --- Diff: core/src/test/scala/org/apache/spark/ui/StagePageSuite.scala --- @@ -22,13 +22,14 @@ import javax.servlet.http.HttpServletRequest import

[GitHub] spark issue #19677: [SPARK-20645][core] Port environment page to new UI back...

2017-11-06 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19677 lgtm --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #19582: [SPARK-20644][core] Initial ground work for kvstore UI b...

2017-11-06 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19582 merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-11-06 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19250 @cloud-fan I think you misunderstand the purpose of this change. The primary purpose is actually to deal with parquet, where that option doesn't do anything. We need this for parquet

[GitHub] spark pull request #19582: [SPARK-20644][core] Initial ground work for kvsto...

2017-11-03 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19582#discussion_r148901343 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -486,8 +580,21 @@ private[history] class FsHistoryProvider(conf

[GitHub] spark pull request #19582: [SPARK-20644][core] Initial ground work for kvsto...

2017-11-03 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19582#discussion_r148879455 --- Diff: core/src/main/scala/org/apache/spark/status/storeTypes.scala --- @@ -17,12 +17,17 @@ package org.apache.spark.status +import

[GitHub] spark pull request #19582: [SPARK-20644][core] Initial ground work for kvsto...

2017-11-03 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19582#discussion_r148847694 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -528,4 +544,21 @@ private class AppStatusListener(kvstore: KVStore

[GitHub] spark pull request #19582: [SPARK-20644][core] Initial ground work for kvsto...

2017-11-03 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19582#discussion_r148836744 --- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala --- @@ -37,8 +37,11 @@ import org.apache.spark.util.kvstore.KVStore

[GitHub] spark pull request #19582: [SPARK-20644][core] Initial ground work for kvsto...

2017-11-03 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19582#discussion_r148713957 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -528,4 +544,21 @@ private class AppStatusListener(kvstore: KVStore

[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...

2017-10-30 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19388 also @mridulm I think you worked on the committer protocol before, both in spark and MR, so would be great to get your thoughts

[GitHub] spark pull request #19583: [WIP][SPARK-22339] [CORE] [NETWORK-SHUFFLE] Push ...

2017-10-27 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19583#discussion_r147522958 --- Diff: core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala --- @@ -51,7 +51,26 @@ private case class ExecutorRegistered(executorId: String

[GitHub] spark pull request #19583: [WIP][SPARK-22339] [CORE] [NETWORK-SHUFFLE] Push ...

2017-10-27 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19583#discussion_r147526776 --- Diff: core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala --- @@ -51,7 +51,26 @@ private case class ExecutorRegistered(executorId: String

[GitHub] spark pull request #19583: [WIP][SPARK-22339] [CORE] [NETWORK-SHUFFLE] Push ...

2017-10-27 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19583#discussion_r147525074 --- Diff: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala --- @@ -241,6 +243,21 @@ final class ShuffleBlockFetcherIterator

[GitHub] spark issue #19583: [WIP][SPARK-22339] [CORE] [NETWORK-SHUFFLE] Push epoch u...

2017-10-27 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19583 Jenkins, ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #19583: [WIP][SPARK-22339] [CORE] [NETWORK-SHUFFLE] Push ...

2017-10-27 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19583#discussion_r147526524 --- Diff: core/src/test/scala/org/apache/spark/HeartbeatReceiverSuite.scala --- @@ -225,6 +270,7 @@ class HeartbeatReceiverSuite Matchers.eq

[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...

2017-10-26 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19388 @rezasafi and I talked about this for a while offline. Its a lot trickier than this seems at first. Here's whats going on: 1. `SparkHadoopWriter` is pretending `rddId` is `stageId

[GitHub] spark issue #19383: [SPARK-20643][core] Add listener implementation to colle...

2017-10-26 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19383 merged to master. thanks @vanzin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #19383: [SPARK-20643][core] Add listener implementation to colle...

2017-10-25 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19383 btw, for any other potential reviews, I'm going already reviewing the rest of marcelo's commits in this project (the prs against is own repo here: https://github.com/vanzin/spark/pulls). In general

[GitHub] spark issue #19383: [SPARK-20643][core] Add listener implementation to colle...

2017-10-25 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19383 lgtm --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-10-25 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r146888728 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -0,0 +1,534 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-10-24 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r146698973 --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusListenerSuite.scala --- @@ -0,0 +1,690 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-10-24 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r146667371 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -0,0 +1,534 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-10-24 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r146689668 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -0,0 +1,534 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-10-24 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r146671188 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -0,0 +1,534 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-10-24 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r146669834 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -0,0 +1,534 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-10-24 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r146682978 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -0,0 +1,535 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-10-24 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r146663719 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -129,29 +130,17 @@ private[history] class FsHistoryProvider

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-10-24 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r146669340 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -0,0 +1,534 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-10-24 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r146694573 --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusListenerSuite.scala --- @@ -0,0 +1,690 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19194: [SPARK-20589] Allow limiting task concurrency per stage

2017-10-23 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19194 @dhruve is this still active? Sorry I was out a while, catching up on everything. --- - To unsubscribe, e-mail: reviews

[GitHub] spark pull request #19399: [SPARK-22175][WEB-UI] Add status column to histor...

2017-10-05 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19399#discussion_r142959826 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -850,6 +869,18 @@ private[history] class AppListingListener(log

[GitHub] spark issue #19287: [SPARK-22074][Core] Task killed by other attempt task sh...

2017-10-05 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19287 lgtm, thanks @xuanyuanking @jerryshao can you merge this? I will have very intermittent access for a few weeks, I'd prefer not to merge in case there is any issue that needs an urgent

[GitHub] spark issue #19194: [SPARK-20589] Allow limiting task concurrency per stage

2017-10-05 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19194 btw I'm going to have really intermittent access for a few weeks, so you don't need to wait for me to proceed with this @tgravescs

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-09-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r141960405 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -0,0 +1,535 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-09-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r141968452 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -0,0 +1,535 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-09-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r141960365 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -0,0 +1,535 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-09-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r141964610 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -0,0 +1,535 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-09-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r141962883 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -0,0 +1,535 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-09-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r141972641 --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusListenerSuite.scala --- @@ -0,0 +1,688 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-09-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r141972926 --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusListenerSuite.scala --- @@ -0,0 +1,688 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-09-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r141967159 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -0,0 +1,535 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-09-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r141955877 --- Diff: common/kvstore/src/main/java/org/apache/spark/util/kvstore/LevelDB.java --- @@ -76,7 +76,7 @@ public LevelDB(File path, KVStoreSerializer

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-09-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r141962667 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -0,0 +1,535 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-09-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r141962046 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -0,0 +1,535 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-09-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r141960672 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -42,6 +42,7 @@ import org.apache.spark.deploy.history.config

[GitHub] spark pull request #19287: [SPARK-22074][Core] Task killed by other attempt ...

2017-09-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19287#discussion_r141953125 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -744,6 +744,112 @@ class TaskSetManagerSuite extends SparkFunSuite

[GitHub] spark pull request #19287: [SPARK-22074][Core] Task killed by other attempt ...

2017-09-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19287#discussion_r141953243 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -744,6 +744,112 @@ class TaskSetManagerSuite extends SparkFunSuite

[GitHub] spark pull request #19389: [SPARK-22165][SQL] Resolve type conflicts between...

2017-09-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19389#discussion_r141947058 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetPartitionDiscoverySuite.scala --- @@ -249,6 +249,11 @@ class

[GitHub] spark pull request #19389: [SPARK-22165][SQL] Resolve type conflicts between...

2017-09-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19389#discussion_r141948681 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetPartitionDiscoverySuite.scala --- @@ -1067,4 +1098,24 @@ class

[GitHub] spark pull request #19389: [SPARK-22165][SQL] Resolve type conflicts between...

2017-09-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19389#discussion_r141949009 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -149,7 +149,7 @@ object TypeCoercion

[GitHub] spark pull request #19388: [SPARK-22162] Executors and the driver should use...

2017-09-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19388#discussion_r141924015 --- Diff: core/src/test/scala/org/apache/spark/rdd/PairRDDFunctionsSuite.scala --- @@ -524,6 +525,13 @@ class PairRDDFunctionsSuite extends SparkFunSuite

[GitHub] spark pull request #19388: [SPARK-22162] Executors and the driver should use...

2017-09-28 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19388#discussion_r141775799 --- Diff: core/src/test/scala/org/apache/spark/rdd/PairRDDFunctionsSuite.scala --- @@ -524,6 +525,13 @@ class PairRDDFunctionsSuite extends SparkFunSuite

[GitHub] spark pull request #19388: [SPARK-22162] Executors and the driver should use...

2017-09-28 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19388#discussion_r141774733 --- Diff: core/src/test/scala/org/apache/spark/rdd/PairRDDFunctionsSuite.scala --- @@ -864,6 +872,47 @@ class NewFakeFormatWithCallback() extends

[GitHub] spark pull request #19388: [SPARK-22162] Executors and the driver should use...

2017-09-28 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19388#discussion_r141774796 --- Diff: core/src/test/scala/org/apache/spark/rdd/PairRDDFunctionsSuite.scala --- @@ -864,6 +872,47 @@ class NewFakeFormatWithCallback() extends

[GitHub] spark pull request #19388: [SPARK-22162] Executors and the driver should use...

2017-09-28 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19388#discussion_r141774743 --- Diff: core/src/test/scala/org/apache/spark/rdd/PairRDDFunctionsSuite.scala --- @@ -864,6 +872,47 @@ class NewFakeFormatWithCallback() extends

[GitHub] spark pull request #19309: [SPARK-19558][sql] Add config key to register Que...

2017-09-28 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19309#discussion_r141712571 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2687,6 +2688,59 @@ private[spark] object Utils extends Logging { def

[GitHub] spark pull request #19287: [SPARK-22074][Core] Task killed by other attempt ...

2017-09-28 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19287#discussion_r141677782 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskInfo.scala --- @@ -66,6 +66,13 @@ class TaskInfo( */ var finishTime: Long = 0

[GitHub] spark pull request #19287: [SPARK-22074][Core] Task killed by other attempt ...

2017-09-28 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19287#discussion_r141676882 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskInfo.scala --- @@ -74,6 +81,10 @@ class TaskInfo( gettingResultTime = time

[GitHub] spark pull request #19287: [SPARK-22074][Core] Task killed by other attempt ...

2017-09-28 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19287#discussion_r141676828 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskInfo.scala --- @@ -66,6 +66,13 @@ class TaskInfo( */ var finishTime: Long = 0

[GitHub] spark pull request #19287: [SPARK-22074][Core] Task killed by other attempt ...

2017-09-28 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19287#discussion_r141674771 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -744,6 +744,100 @@ class TaskSetManagerSuite extends SparkFunSuite

[GitHub] spark pull request #19287: [SPARK-22074][Core] Task killed by other attempt ...

2017-09-28 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19287#discussion_r141672604 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -744,6 +744,100 @@ class TaskSetManagerSuite extends SparkFunSuite

<    6   7   8   9   10   11   12   13   14   15   >