[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[ScrapCodes]

Github user ScrapCodes closed the pull request at:

https://github.com/apache/spark/pull/2357


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/2357#issuecomment-58573083
  
Can one of the admins verify this patch?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/2357#issuecomment-57596967
  
  [QA tests have 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21183/consoleFull)
 for   PR 2357 at commit 
[`609dd98`](https://github.com/apache/spark/commit/609dd98b7de77397dfc490c1c0a12bb9349830e5).
 * This patch **passes** unit tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/2357#issuecomment-57596973
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21183/


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/2357#issuecomment-57590659
  
  [QA tests have 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21183/consoleFull)
 for   PR 2357 at commit 
[`609dd98`](https://github.com/apache/spark/commit/609dd98b7de77397dfc490c1c0a12bb9349830e5).
 * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[ScrapCodes]

GitHub user ScrapCodes reopened a pull request:

https://github.com/apache/spark/pull/2357

[SPARK-3437][BUILD] Support crossbuilding in maven. With new 
scala-install-plugin.

Since this plugin is not deployed anywhere, for anyone trying this patch 
has to publish it locally by cloning the following repo 
https://github.com/ScrapCodes/scala-install-plugin.  And then running `mvn 
install`. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ScrapCodes/spark-1 maven-improvements

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/2357.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2357


commit 4d2d30cc9dcbe2b929ed8ed124d6719430ec6fae
Author: Prashant Sharma 
Date:   2014-09-11T08:09:46Z

Supported new scala install plugin. Which can let us cross build for scala.

commit 609dd98b7de77397dfc490c1c0a12bb9349830e5
Author: Prashant Sharma 
Date:   2014-09-11T10:00:10Z

Changed to newly updated with cross build support branch.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[ScrapCodes]

Github user ScrapCodes closed the pull request at:

https://github.com/apache/spark/pull/2357


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[ScrapCodes]

Github user ScrapCodes commented on the pull request:

https://github.com/apache/spark/pull/2357#issuecomment-57126589
  
Turns out this is not very useful either.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[ScrapCodes]

Github user ScrapCodes closed the pull request at:

https://github.com/apache/spark/pull/2318


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[srowen]

Github user srowen commented on the pull request:

https://github.com/apache/spark/pull/2357#issuecomment-55572928
  
I see, I thought you mentioned above that running a plugin before `install` 
would work. It sounds like there is some internal state of the plugin you need 
to modify, OK. I don't know, maybe it's useful to elaborate this and see if 
anyone else can see a workaround.

It is worth keeping track of alternatives when evaluating just how far to 
hack this. For example: Scala 2.11 support could be bound up with Spark 2.x 
support. Or it can be in a branch that is maintained over a few minor versions, 
which is not such a big deal if the delta is just flipping "2.10" -> "2.11" in 
many places.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[ScrapCodes]

Github user ScrapCodes commented on the pull request:

https://github.com/apache/spark/pull/2357#issuecomment-55571483
  
Hi @srowen, like I said. We can run a plugin before maven install - no 
question about that. But since maven install gets a copy of STATE(somehoe via 
guice). Altering it in another maven plugin does not help much. So as of now, I 
do not see a way around not modifying maven install plugin.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[srowen]

Github user srowen commented on the pull request:

https://github.com/apache/spark/pull/2357#issuecomment-55570721
  
- If you bind a plugin to `install` phase and declare it before 
`maven-install-plugin`, will it happen to respect the ordering?
- This is arguably something that can happen before `install`, in `package` 
phase? There's a `prepare-package` phase as well


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[ScrapCodes]

Github user ScrapCodes commented on the pull request:

https://github.com/apache/spark/pull/2357#issuecomment-9135
  
I am anyway trying more options to not need to modify maven-install-pluing. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[ScrapCodes]

Github user ScrapCodes commented on the pull request:

https://github.com/apache/spark/pull/2357#issuecomment-6738
  
you are right about that, but we are forking it because we want an install 
plugin modified. If it was possible to run a plugin just before install and 
install plugin magically does the job correctly. That would have been nicer. 
But that isn't possible unless I try to hack things using reflection. Its like 
maven keeps its copy of objects before letting plugins use it.  


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[pwendell]

Github user pwendell commented on the pull request:

https://github.com/apache/spark/pull/2357#issuecomment-55465992
  
Hey @ScrapCodes rather than forking the existing install plug-in, could we 
write our own plug-in that runs before the install plug-in? Forking seems like 
it will be difficult of e.g. the maven install plug-in updates.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[ScrapCodes]

Github user ScrapCodes commented on the pull request:

https://github.com/apache/spark/pull/2357#issuecomment-55362622
  
It will not conflict with that, it works the same way as maven install 
plugin. (accidentally deleted my previous post. :/)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[ScrapCodes]

Github user ScrapCodes commented on the pull request:

https://github.com/apache/spark/pull/2357#issuecomment-55362572
  
I will not conflict with that. One can simply replace maven-install-plugin 
with this one and works the exact same way.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[vanzin]

Github user vanzin commented on the pull request:

https://github.com/apache/spark/pull/2357#issuecomment-55342092
  
This sort of conflicts with parts of #2329. Would it be possible to add a 
flag to the new plugin so that it can automatically skip projects where a 
certain property is set? That way you just set "scala.install.skip" instead of 
copying a whole lot of XML for each module you don't want to install.

No idea if that's possible, and I'm aware that's not very maven-approved, 
but just a suggestion.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/2357#issuecomment-55258220
  
  [QA tests have 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20149/consoleFull)
 for   PR 2357 at commit 
[`609dd98`](https://github.com/apache/spark/commit/609dd98b7de77397dfc490c1c0a12bb9349830e5).
 * This patch **passes** unit tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/2357#issuecomment-55248291
  
  [QA tests have 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20149/consoleFull)
 for   PR 2357 at commit 
[`609dd98`](https://github.com/apache/spark/commit/609dd98b7de77397dfc490c1c0a12bb9349830e5).
 * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[ScrapCodes]

Github user ScrapCodes commented on the pull request:

https://github.com/apache/spark/pull/2318#issuecomment-55244394
  
#2357 Does this with the help of a custom maven plugin. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[ScrapCodes]

GitHub user ScrapCodes opened a pull request:

https://github.com/apache/spark/pull/2357

[SPARK-3437][BUILD] Support crossbuilding in maven. With new 
scala-install-plugin.

Since this plugin is not deployed anywhere, for anyone trying this patch 
has to publish it locally by cloning the following repo 
https://github.com/ScrapCodes/scala-install-plugin.  And then running `mvn 
install`. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ScrapCodes/spark-1 maven-improvements

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/2357.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2357


commit 4d2d30cc9dcbe2b929ed8ed124d6719430ec6fae
Author: Prashant Sharma 
Date:   2014-09-11T08:09:46Z

Supported new scala install plugin. Which can let us cross build for scala.

commit 609dd98b7de77397dfc490c1c0a12bb9349830e5
Author: Prashant Sharma 
Date:   2014-09-11T10:00:10Z

Changed to newly updated with cross build support branch.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[vanzin]

Github user vanzin commented on the pull request:

https://github.com/apache/spark/pull/2318#issuecomment-55138035
  
If there's no way for the maven-install-plugin to fix the existing pom 
(instead of generating a minimal pom with no dependencies), then I guess I 
don't see any other solution. (If only you could use properties in the 
artifactId declaration...)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[ScrapCodes]

Github user ScrapCodes commented on the pull request:

https://github.com/apache/spark/pull/2318#issuecomment-55075420
  
Hey All,
Please correct me if I am wrong, my conclusion is that this also will not 
help because pom published is not appropriate(Since it does not have dependency 
info, which is I guess needed by downstream projects). So IMO writing a plugin 
is the only option now.  I have started writing one here 
https://github.com/ScrapCodes/scala-install-plugin. It is cool in the sense, it 
publishes effective pom that is: if artifact was build using a profile its 
effective pom of dependencies(will have correct properties values set in etc..) 
will be different than one published with a different profile. I am not expert 
here and totally confused, my only source of information is seeing what others 
have done.   


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[srowen]

Github user srowen commented on the pull request:

https://github.com/apache/spark/pull/2318#issuecomment-54932687
  
@vanzin So yeah, it doesn't have a `skip` configuration, darn. How about 
this? 

- Declare plugin in `` in the parent with all 
``
- _Omit_ `` in this parent POM declaration
- In children, repeat the `` declaration in ``, without 
``, but with an `` with the same `` that adds the 
``

The idea is that you can't avoid the plugin being active but can wait to 
connect the `install-file` goal to a phase until the children. It might be 
sufficient to just omit `` until the children.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[ScrapCodes]

Github user ScrapCodes commented on a diff in the pull request:

https://github.com/apache/spark/pull/2318#discussion_r17282239
  
--- Diff: assembly/pom.xml ---
@@ -26,7 +26,7 @@
   
 
   org.apache.spark
-  spark-assembly_2.10
+  spark-assembly
--- End diff --

I just checked the published pom had, just this inside it.
```xml

http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd"; 
xmlns="http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";>
  4.0.0
  org.apache.spark
  spark-core_2.10
  1.2.0-SNAPSHOT
  POM was created from install:install-file

```

No dependency information, I am not sure if we can live without that. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Re: [GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[Sean Owen]

Dang, I suppose because install-file does not bind to a lifecycle by
default so you wouldn't generally need to skip it - you would just not bind
it.  Maybe there is some other way to make it a no-op.

Let me look into at least refactoring the configuration if not the
execution declaration. There has to be something that can be made common.
On Sep 8, 2014 11:57 PM, "vanzin"  wrote:

> Github user vanzin commented on the pull request:
>
> https://github.com/apache/spark/pull/2318#issuecomment-54899897
>
> I don't see a "skip" config option for the install-file goal, but if
> it exists / works, sure, that's an option.
>
>
> ---
> If your project is set up for it, you can reply to this email and have your
> reply appear on GitHub as well. If your project does not have this feature
> enabled and wishes so, or if the feature is enabled but not working, please
> contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
> with INFRA.
> ---
>
> -
> To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
> For additional commands, e-mail: reviews-h...@spark.apache.org
>
>

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[vanzin]

Github user vanzin commented on the pull request:

https://github.com/apache/spark/pull/2318#issuecomment-54899897
  
I don't see a "skip" config option for the install-file goal, but if it 
exists / works, sure, that's an option.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[srowen]

Github user srowen commented on the pull request:

https://github.com/apache/spark/pull/2318#issuecomment-54899349
  
pluginManagement by itself never activates a plugin. But crumbs I suppose
the install plugin is inherited as an active plugin from Maven's own
default POM. Aha, well, configure it in the parent, set config skip=true
there, and set it to false in children? That has worked for me elsewhere.
On Sep 8, 2014 11:47 PM, "Marcelo Vanzin"  wrote:

> I'm not suggesting putting it in but just configuring it in
>
> That's what I meant. If you do that, it will be executed for the
> "spark_parent" pom and will fail. (BTW that's the first thing I tried.)
>
> —
> Reply to this email directly or view it on GitHub
> .
>


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[vanzin]

Github user vanzin commented on the pull request:

https://github.com/apache/spark/pull/2318#issuecomment-54898831
  
> I'm not suggesting putting it in  but just configuring it in 


That's what I meant. If you do that, it will be executed for the 
"spark_parent" pom and will fail. (BTW that's the first thing I tried.)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[srowen]

Github user srowen commented on the pull request:

https://github.com/apache/spark/pull/2318#issuecomment-54890607
  
@vanzin I'm not suggesting putting it in `` but just configuring 
it in `` in the parent. That's where it's configured now. It 
would not make it execute on the parent. You still need to enable it in each 
submodule in ``, but not copy config in each submodule. At least that 
works fine in general and hope it works here.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[vanzin]

Github user vanzin commented on the pull request:

https://github.com/apache/spark/pull/2318#issuecomment-54890315
  
@srowen the problem is that maven is stup... err, peculiar.

If you declare the plugin in the parent pom's "pluginManagement" section, 
it will try to execute the plugin when building the parent pom itself. Since 
the parent pom has packaging "pom", it will fail, since there are no jars. So 
you need a way to only add the plugin to the build in certain situations.

The logical thing would have the plugin only be activated for children, but 
there's no option I can find to do that. The next would be to only bind the 
plugin to a certain packaging type, but again, no dice.

So you have to resort to profiles. You could have a profile activated by a 
property, but then that doesn't work when the property is set in the child pom 
(instead of the command line). So you're left with checking for some file on 
disk, like my approach. I checked for "src/main/scala", but if that's too 
generic, we can use a tag file (and create that file in all projects where we 
want the plugin to be activated).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[srowen]

Github user srowen commented on the pull request:

https://github.com/apache/spark/pull/2318#issuecomment-54889448
  
@vanzin I have a feeling I'm missing something, but does this config need 
to be repeated more than once to begin with, instead of being defined in the 
parent once? Is it not going to work in this situation given what the plugin is 
trying to do with the artifact ID?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[vanzin]

Github user vanzin commented on the pull request:

https://github.com/apache/spark/pull/2318#issuecomment-54889080
  
@ScrapCodes the amount of copy & paste that Maven sometimes seem to enforce 
on people always boggles my mind... have you tried alternative approaches?

I played a little bit with a couple of things and this approach seems to 
have made some progress: https://gist.github.com/vanzin/1ada891e573b89ff8c45

Namely, I saw the executions of the maven-install-plugin when building 
core/, and projects dependending on core were picking up the right dependency 
(with the scala version appended to the artifactId).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[srowen]

Github user srowen commented on a diff in the pull request:

https://github.com/apache/spark/pull/2318#discussion_r17261480
  
--- Diff: assembly/pom.xml ---
@@ -26,7 +26,7 @@
   
 
   org.apache.spark
-  spark-assembly_2.10
+  spark-assembly
--- End diff --

I see now. That makes sense then, and leaves open the possibility of 
publishing parallel builds for Scala 2.10 / 2.11 under the current naming 
convention. Although maybe you could argue for using `classifier` instead, 
there's no point changing conventions if they're not already being changed. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[pwendell]

Github user pwendell commented on the pull request:

https://github.com/apache/spark/pull/2318#issuecomment-54874836
  
I like this approach better than the previous one of modifying the maven 
build with bash - but I haven't tested it locally yet. I do wonder a bit about 
how this works in sbt (is this inherited for an `sbt/sbt publish-local`?). 
Also, it would be really good to see if this could be done in a way that 
doesn't require adding this big build definition in every project.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[pwendell]

Github user pwendell commented on a diff in the pull request:

https://github.com/apache/spark/pull/2318#discussion_r17261129
  
--- Diff: assembly/pom.xml ---
@@ -26,7 +26,7 @@
   
 
   org.apache.spark
-  spark-assembly_2.10
+  spark-assembly
--- End diff --

Sean I think this approach actually modifies the published pom's to look 
different than the poms which are our own build. I.e. it appends the 2.11 or 
2.10 suffix to the artifact in the published poms. Other than being 
horrendously verbose... this does seem to be a nicer approach IMO.

Since donstream projects only consume spark via the published poms, it 
seems okay to me. What is the specific case you're concerned about?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/2318#issuecomment-54810834
  
  [QA tests have 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19973/consoleFull)
 for   PR 2318 at commit 
[`ed315ef`](https://github.com/apache/spark/commit/ed315ef0ec1d8e87009e81972c2a4009df06a726).
 * This patch **passes** unit tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[srowen]

Github user srowen commented on a diff in the pull request:

https://github.com/apache/spark/pull/2318#discussion_r17232335
  
--- Diff: bagel/pom.xml ---
@@ -60,6 +60,53 @@
 
target/scala-${scala.binary.version}/test-classes
 
   
+org.apache.maven.plugins
+maven-install-plugin
+2.5.2
--- End diff --

You should specify this version once in parent pom in `pluginManagement`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[srowen]

Github user srowen commented on a diff in the pull request:

https://github.com/apache/spark/pull/2318#discussion_r17232323
  
--- Diff: assembly/pom.xml ---
@@ -26,7 +26,7 @@
   
 
   org.apache.spark
-  spark-assembly_2.10
+  spark-assembly
--- End diff --

It goes without saying, but this introduces a sort of 'incompatibility' at 
the Maven level. Downstream projects have to change artifact _and_ version when 
selecting between this and a future version with different artifact names. This 
doesn't automatically harmonize the Spark build with the version of Scala 
you're using, and Maven can't resolve dependency conflicts across differently 
named artifacts. These are kind of inevitable.

If an end goal is to publicly support Scala 2.10 and 2.11 simultaneously, 
then we still have to have several published artifacts at once right? this is 
what `classifier` could be used for. But I suppose it's worth asking if that's 
the intent? or is this going to may be happen all at once for Spark 2.x?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[srowen]

Github user srowen commented on a diff in the pull request:

https://github.com/apache/spark/pull/2318#discussion_r17232343
  
--- Diff: bagel/pom.xml ---
@@ -60,6 +60,53 @@
 
target/scala-${scala.binary.version}/test-classes
 
   
+org.apache.maven.plugins
+maven-install-plugin
+2.5.2
+
+  
+1
+
+  install-file
+
+install
+
+  
${project.build.directory}/${project.build.finalName}.jar
+  
${project.artifactId}_${scala.binary.version}
+
+  
+  
+2
+
+  install-file
+
+install
+
+  false
+  pom.xml
+  
${project.build.directory}/${project.build.finalName}-sources.jar
+  
${project.artifactId}_${scala.binary.version}
+  sources
+
+  
+  
+3
+
+  install-file
+
+install
+
+  false
+  pom.xml
--- End diff --

In fact it looks like all of this config can be common in the parent since 
it looks like it's written generically in terms of properties?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/2318#issuecomment-54801998
  
  [QA tests have 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19973/consoleFull)
 for   PR 2318 at commit 
[`ed315ef`](https://github.com/apache/spark/commit/ed315ef0ec1d8e87009e81972c2a4009df06a726).
 * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/2318#issuecomment-54796972
  
  [QA tests have 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19970/consoleFull)
 for   PR 2318 at commit 
[`6c8f9f6`](https://github.com/apache/spark/commit/6c8f9f6fe2c1c151294add2ca1c7820975872df9).
 * This patch **fails** unit tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/2318#issuecomment-54796779
  
  [QA tests have 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19970/consoleFull)
 for   PR 2318 at commit 
[`6c8f9f6`](https://github.com/apache/spark/commit/6c8f9f6fe2c1c151294add2ca1c7820975872df9).
 * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org