Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/1945#issuecomment-62773928
Hey @pwendell @srowen @bbossy this is actually causing SBT applications
that use the `spark-hive_2.10` module. More details can be found here:
https://issues.apache.or
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/1945
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enab
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1945#issuecomment-55473914
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/80/consoleFull)
for PR 1945 at commit
[`c32ce59`](https://github.com/a
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1945#issuecomment-55466591
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/80/consoleFull)
for PR 1945 at commit
[`c32ce59`](https://github.com/ap
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/1945#issuecomment-55466291
Yeah - LGTM pending tests.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1945#issuecomment-54694472
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project d
Github user bbossy commented on the pull request:
https://github.com/apache/spark/pull/1945#issuecomment-52290012
Created the issue: https://issues.apache.org/jira/browse/SPARK-3069 (Build
instructions in README are outdated)
@srowen: Thank you for your input!
---
If y
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/1945#issuecomment-52287539
I think it works with the invocation you describe. Honestly it's not a big
priority, this version, but nice to get it right. Want to open a JIRA to track
updating/deleting
Github user bbossy commented on the pull request:
https://github.com/apache/spark/pull/1945#issuecomment-52209453
Yeah, you're right about yarn being orthogonal to the Hadoop version.
Apart from the maven/sbt question there is another issue: The `Cloudera CDH
4.2.0 with MapRed
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/1945#issuecomment-52202237
Yeah that's out of date I believe. For example `-Phadoop-2.3` has to be
specified with `-Dhadoop.version=2.3.0`. And I think `mvn` is the primary build
now. I imagine you
Github user bbossy commented on the pull request:
https://github.com/apache/spark/pull/1945#issuecomment-52201887
The problem I see, is that if you build according to the README:
```
# Apache Hadoop 2.2.X and newer
$ sbt/sbt -Dhadoop.version=2.2.0 -Pyarn assembly
```
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/1945#issuecomment-52200382
You have to specify a Hadoop profile already, and you added the classifier
to all of them. So that's fine. Building with YARN is orthogonal, so doesn't
belong elsewhere I
Github user bbossy commented on the pull request:
https://github.com/apache/spark/pull/1945#issuecomment-52199845
Should I also add the `avro.mapred.classifier` property to the yarn
profile? Maybe even yarn-alpha and mapr?
Since now to build it according to the README one shou
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/1945#issuecomment-52187372
I've looked at this part of the build a lot and can say LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1945#issuecomment-52186236
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your pro
GitHub user bbossy opened a pull request:
https://github.com/apache/spark/pull/1945
SPARK-3039: Allow spark to be built using avro-mapred for hadoop2
SPARK-3039: Adds the maven property "avro.mapred.classifier" to build
spark-assembly with avro-mapred with support for the new Hadoop
16 matches
Mail list logo
