[GitHub] spark pull request: SPARK-6433 hive tests to import spark-sql test...

2015-03-23 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/5119#discussion_r26925189 --- Diff: pom.xml --- @@ -1472,6 +1474,46 @@ groupIdorg.scalatest/groupId artifactIdscalatest-maven-plugin/artifactId

[GitHub] spark pull request: [SPARK-5682] Reuse hadoop encrypted shuffle al...

2015-03-23 Thread steveloughran
Github user steveloughran commented on the pull request: https://github.com/apache/spark/pull/4491#issuecomment-84934768 @kellyzly i don't understand why need make CryptoOutputStream.scala#close safe. Is there situation when multiple threads call this function

[GitHub] spark pull request: SPARK-6433 hive tests to import spark-sql test...

2015-03-23 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/5119#discussion_r26925330 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/QueryTest.scala --- @@ -1,140 +0,0 @@ -/* --- End diff -- yes. These are the two

[GitHub] spark pull request: SPARK-6433 hive tests to import spark-sql test...

2015-03-23 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/5119#discussion_r26925134 --- Diff: pom.xml --- @@ -1472,6 +1474,46 @@ groupIdorg.scalatest/groupId artifactIdscalatest-maven-plugin/artifactId

[GitHub] spark pull request: SPARK-6433 hive tests to import spark-sql test...

2015-03-23 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/5119#discussion_r26925089 --- Diff: pom.xml --- @@ -158,6 +158,7 @@ fasterxml.jackson.version2.4.4/fasterxml.jackson.version snappy.version1.1.1.6

[GitHub] spark pull request: SPARK-6433 patch 001: test JARs are built; sql...

2015-03-21 Thread steveloughran
GitHub user steveloughran opened a pull request: https://github.com/apache/spark/pull/5119 SPARK-6433 patch 001: test JARs are built; sql/hive pulls in spark-sql ... 1. Test JARs are built published 1. log4j.resources is explicitly excluded. Without this, downstream test run

[GitHub] spark pull request: [SPARK-5682] Reuse hadoop encrypted shuffle al...

2015-03-20 Thread steveloughran
Github user steveloughran commented on the pull request: https://github.com/apache/spark/pull/4491#issuecomment-83972651 Most of the sync changes in HADOOP-1170 was to deal with HBase's expectations. Even if you don't think it matters for its current use in spark, you should make

[GitHub] spark pull request: [SPARK-5682] Reuse hadoop encrypted shuffle al...

2015-03-19 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/4491#discussion_r26760844 --- Diff: core/src/main/scala/org/apache/spark/crypto/CipherSuite.scala --- @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-5682] Reuse hadoop encrypted shuffle al...

2015-03-19 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/4491#discussion_r26760061 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoInputStream.scala --- @@ -0,0 +1,428 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-5682] Reuse hadoop encrypted shuffle al...

2015-03-19 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/4491#discussion_r26760312 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoInputStream.scala --- @@ -0,0 +1,428 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-5682] Reuse hadoop encrypted shuffle al...

2015-03-19 Thread steveloughran
Github user steveloughran commented on the pull request: https://github.com/apache/spark/pull/4491#issuecomment-83615286 1. the `InterfaceAudience.Private` tags in Hadoop are a please don't use` hint, although if you look at YARN AMs, they end up importing using stuff which

[GitHub] spark pull request: [SPARK-5342][YARN] Allow long running Spark ap...

2015-03-17 Thread steveloughran
Github user steveloughran commented on the pull request: https://github.com/apache/spark/pull/4688#issuecomment-82491054 Following Thomas's [earlier comment](https://github.com/apache/spark/pull/4688#issuecomment-76224212) * Yes, slider does keytabs. For deploying things like

[GitHub] spark pull request: SPARK-6389 YARN app diagnostics report doesn't...

2015-03-17 Thread steveloughran
GitHub user steveloughran opened a pull request: https://github.com/apache/spark/pull/5070 SPARK-6389 YARN app diagnostics report doesn't report NPEs Trivial patch to implicitly call `Exception.toString()` over `Exception.getMessage()` —this defaults to including the exception

[GitHub] spark pull request: SPARK-6389 YARN app diagnostics report doesn't...

2015-03-17 Thread steveloughran
Github user steveloughran commented on the pull request: https://github.com/apache/spark/pull/5070#issuecomment-82419544 I think you'd be welcome to take a quick look for other occurrences in the code base that plainly need the same treatment. As I come across them

[GitHub] spark pull request: SPARK-6389 YARN app diagnostics report doesn't...

2015-03-17 Thread steveloughran
Github user steveloughran commented on the pull request: https://github.com/apache/spark/pull/5070#issuecomment-82450470 Actually, there are a couple more in the same file. I'll update this pull with the other ones --- If your project is set up for it, you can reply to this email

<    7   8   9   10   11   12