Re: Maven and DevOps
Hi Lennart,
bintray is about social aspects (rating and reviews of maven artifacts,
etc.).
Artifactory promotion could simplify your release process, but this may
depend on your setup (do you already have artifactory in place or
something else nexus/OSSRH?).
If you do not want to discuss visionary thoughts but simplify your build
and deployment today, have a look at your current continuous
integration. Do you have a dedicated jenkins or so? Simply automate your
deployment there using according jobs. So far this is the way to go.
Cheers
Jörg
Am 14.05.2016 um 17:33 schrieb Lennart Jörelid:
Hello all,
If I read things correctly, JFrog supplies free accounts for OSS projects
(typically Apache- or MojoHaus-based projects) for distribution on Bintray
central. Do you feel that this approach would complicate or simplify
automated release processes, Jörg? I am not certain I interpret your
thoughts in the last entry correctly.
2016-05-14 17:21 GMT+02:00 Hohwiller, Jörg :
Hi Stian,
thanks for the hint:
https://www.jfrog.com/blog/search-based-promotion/
I also found bintray when digging there:
https://bintray.com/
That is actually part of the social story I was thinking about. JFrog
should only lower barriers (e.g. OAuth login via Github, Google, etc.
instead of forcing bintray account).
Great that I found this. IMHO all maven artifact consumers should have a
look...
Best regards
Jörg
Am 03.05.2016 um 13:29 schrieb Stian Soiland-Reyes:
What you are describing is basically using "continuous" SNAPSHOT
dependencies.
Semantic Versioning is important for understanding API changes (and to
prevent such changes when not necessary). This could of course be computed
automatically, but there are also non-interface changes that a human needs
to indicate (e.g. change of .equals() javadoc)
It is very easy to set up Jenkins to build SNAPSHOT on any commit (e.g.
merge of Pull Request) and to deploy to the snapshot repository only if
the
build and tests succeeded.
Approaches such as Nexus staging repositories and JFrog Artifactory's
release promotion can be used to add quality stamps ("stable versions") to
a separate repository.
On 1 May 2016 9:18 p.m., "Hohwiller, Jörg" wrote:
Hi there,
I wanted to share some thoughs I had recently. Maven introduced a
revolution to the Java world and made a really big step for dependency
and
build management. Open-Source projects are more productive with maven.
However, in the last years DevOps showed up and projects start to
continuously build releases in some cases with a fully automated build
pipeline.
When I look at open-source development around I see that we have great
infrastructure with github and pull-requests, etc.
But as a downside I also see slow and over-complex processes to get
something released (see e.g.
http://www.mojohaus.org/development/performing-a-release.html - wow that
is not really lean).
In order not to fingerpoint someone I will pick myself: I got a
pull-request from someone for servicedocgen-maven-plugin that I maintain
at
mojohaus. I reviewed the PR and merged it. Unfortunately, I was very busy
then and did not create a release for two month now. It might not take
that
much, but still too much. I want to question why do we need all this
stuff
and the votings, etc.
So assume the following future vision for a maven project:
* When a pull request passed (travis, coveralls, etc.) and gets merged a
CI system automatically builds a release (no need to get PGP keys per
developer just one setup once for the project CI). The release simply
gets
a timestamp as version-identifier (MMdd-hhmmss).
* Now besides the project being responsible for quality (by having good
tests and only accepting PRs after reasoable review) the community
(artifact users) could also help and do additional quality assurance.
Assume maven-central would become a collaborative platform where the
users
of artifacts could vote and label artifact releases. Add comments, link
CVE
or bug reports, etc. Vote +1/-1 on quality or security...
* Still the project releasing the artifacts could label releases and
associate minor/major release numbers to branches.
In such case however dependencies would not point to a version like
(4.2.1.RELEASE) but instead to 20160501-235901. In order to pick the
right
technical version you would lookup the collaborative meta data.
I do not expect everybody to shout hurray to this rought idea. But I
would
be happy if people can think about it and may combine it with other ideas
so we get even better in the future some day.
See also
https://dzone.com/articles/continuous-releasing-maven
https://devopsnet.com/2012/02/21/continuous-delivery-using-maven/
I would love to see that maven better supports flexible handling of the
version for the development view while it could simply stay as it is for
the consumer view. When I use variables in versions of POMs (and I do
that
in every project today e.g. in combination with flatten-maven-plugin) I
Re: Maven and DevOps
Hello all,
If I read things correctly, JFrog supplies free accounts for OSS projects
(typically Apache- or MojoHaus-based projects) for distribution on Bintray
central. Do you feel that this approach would complicate or simplify
automated release processes, Jörg? I am not certain I interpret your
thoughts in the last entry correctly.
2016-05-14 17:21 GMT+02:00 Hohwiller, Jörg :
> Hi Stian,
>
> thanks for the hint:
> https://www.jfrog.com/blog/search-based-promotion/
>
> I also found bintray when digging there:
> https://bintray.com/
>
> That is actually part of the social story I was thinking about. JFrog
> should only lower barriers (e.g. OAuth login via Github, Google, etc.
> instead of forcing bintray account).
>
> Great that I found this. IMHO all maven artifact consumers should have a
> look...
>
> Best regards
>
> Jörg
>
>
>
> Am 03.05.2016 um 13:29 schrieb Stian Soiland-Reyes:
>
>> What you are describing is basically using "continuous" SNAPSHOT
>> dependencies.
>>
>> Semantic Versioning is important for understanding API changes (and to
>> prevent such changes when not necessary). This could of course be computed
>> automatically, but there are also non-interface changes that a human needs
>> to indicate (e.g. change of .equals() javadoc)
>>
>> It is very easy to set up Jenkins to build SNAPSHOT on any commit (e.g.
>> merge of Pull Request) and to deploy to the snapshot repository only if
>> the
>> build and tests succeeded.
>>
>> Approaches such as Nexus staging repositories and JFrog Artifactory's
>> release promotion can be used to add quality stamps ("stable versions") to
>> a separate repository.
>>
>> On 1 May 2016 9:18 p.m., "Hohwiller, Jörg" wrote:
>>
>> Hi there,
>>>
>>> I wanted to share some thoughs I had recently. Maven introduced a
>>> revolution to the Java world and made a really big step for dependency
>>> and
>>> build management. Open-Source projects are more productive with maven.
>>> However, in the last years DevOps showed up and projects start to
>>> continuously build releases in some cases with a fully automated build
>>> pipeline.
>>> When I look at open-source development around I see that we have great
>>> infrastructure with github and pull-requests, etc.
>>> But as a downside I also see slow and over-complex processes to get
>>> something released (see e.g.
>>> http://www.mojohaus.org/development/performing-a-release.html - wow that
>>> is not really lean).
>>> In order not to fingerpoint someone I will pick myself: I got a
>>> pull-request from someone for servicedocgen-maven-plugin that I maintain
>>> at
>>> mojohaus. I reviewed the PR and merged it. Unfortunately, I was very busy
>>> then and did not create a release for two month now. It might not take
>>> that
>>> much, but still too much. I want to question why do we need all this
>>> stuff
>>> and the votings, etc.
>>>
>>> So assume the following future vision for a maven project:
>>>
>>> * When a pull request passed (travis, coveralls, etc.) and gets merged a
>>> CI system automatically builds a release (no need to get PGP keys per
>>> developer just one setup once for the project CI). The release simply
>>> gets
>>> a timestamp as version-identifier (MMdd-hhmmss).
>>>
>>> * Now besides the project being responsible for quality (by having good
>>> tests and only accepting PRs after reasoable review) the community
>>> (artifact users) could also help and do additional quality assurance.
>>> Assume maven-central would become a collaborative platform where the
>>> users
>>> of artifacts could vote and label artifact releases. Add comments, link
>>> CVE
>>> or bug reports, etc. Vote +1/-1 on quality or security...
>>>
>>> * Still the project releasing the artifacts could label releases and
>>> associate minor/major release numbers to branches.
>>>
>>> In such case however dependencies would not point to a version like
>>> (4.2.1.RELEASE) but instead to 20160501-235901. In order to pick the
>>> right
>>> technical version you would lookup the collaborative meta data.
>>> I do not expect everybody to shout hurray to this rought idea. But I
>>> would
>>> be happy if people can think about it and may combine it with other ideas
>>> so we get even better in the future some day.
>>>
>>> See also
>>> https://dzone.com/articles/continuous-releasing-maven
>>> https://devopsnet.com/2012/02/21/continuous-delivery-using-maven/
>>>
>>> I would love to see that maven better supports flexible handling of the
>>> version for the development view while it could simply stay as it is for
>>> the consumer view. When I use variables in versions of POMs (and I do
>>> that
>>> in every project today e.g. in combination with flatten-maven-plugin) I
>>> always get warnings from Maven saying:
>>> [WARNING] Some problems were encountered while building the effective
>>> model for ...
>>> [WARNING] 'version' contains an expression but should be a constant.
>>>
>>> Still I see no clear picture how maven will adress these needs that
>>
Re: Maven and DevOps
Hi Stian,
thanks for the hint:
https://www.jfrog.com/blog/search-based-promotion/
I also found bintray when digging there:
https://bintray.com/
That is actually part of the social story I was thinking about. JFrog
should only lower barriers (e.g. OAuth login via Github, Google, etc.
instead of forcing bintray account).
Great that I found this. IMHO all maven artifact consumers should have a
look...
Best regards
Jörg
Am 03.05.2016 um 13:29 schrieb Stian Soiland-Reyes:
What you are describing is basically using "continuous" SNAPSHOT
dependencies.
Semantic Versioning is important for understanding API changes (and to
prevent such changes when not necessary). This could of course be computed
automatically, but there are also non-interface changes that a human needs
to indicate (e.g. change of .equals() javadoc)
It is very easy to set up Jenkins to build SNAPSHOT on any commit (e.g.
merge of Pull Request) and to deploy to the snapshot repository only if the
build and tests succeeded.
Approaches such as Nexus staging repositories and JFrog Artifactory's
release promotion can be used to add quality stamps ("stable versions") to
a separate repository.
On 1 May 2016 9:18 p.m., "Hohwiller, Jörg" wrote:
Hi there,
I wanted to share some thoughs I had recently. Maven introduced a
revolution to the Java world and made a really big step for dependency and
build management. Open-Source projects are more productive with maven.
However, in the last years DevOps showed up and projects start to
continuously build releases in some cases with a fully automated build
pipeline.
When I look at open-source development around I see that we have great
infrastructure with github and pull-requests, etc.
But as a downside I also see slow and over-complex processes to get
something released (see e.g.
http://www.mojohaus.org/development/performing-a-release.html - wow that
is not really lean).
In order not to fingerpoint someone I will pick myself: I got a
pull-request from someone for servicedocgen-maven-plugin that I maintain at
mojohaus. I reviewed the PR and merged it. Unfortunately, I was very busy
then and did not create a release for two month now. It might not take that
much, but still too much. I want to question why do we need all this stuff
and the votings, etc.
So assume the following future vision for a maven project:
* When a pull request passed (travis, coveralls, etc.) and gets merged a
CI system automatically builds a release (no need to get PGP keys per
developer just one setup once for the project CI). The release simply gets
a timestamp as version-identifier (MMdd-hhmmss).
* Now besides the project being responsible for quality (by having good
tests and only accepting PRs after reasoable review) the community
(artifact users) could also help and do additional quality assurance.
Assume maven-central would become a collaborative platform where the users
of artifacts could vote and label artifact releases. Add comments, link CVE
or bug reports, etc. Vote +1/-1 on quality or security...
* Still the project releasing the artifacts could label releases and
associate minor/major release numbers to branches.
In such case however dependencies would not point to a version like
(4.2.1.RELEASE) but instead to 20160501-235901. In order to pick the right
technical version you would lookup the collaborative meta data.
I do not expect everybody to shout hurray to this rought idea. But I would
be happy if people can think about it and may combine it with other ideas
so we get even better in the future some day.
See also
https://dzone.com/articles/continuous-releasing-maven
https://devopsnet.com/2012/02/21/continuous-delivery-using-maven/
I would love to see that maven better supports flexible handling of the
version for the development view while it could simply stay as it is for
the consumer view. When I use variables in versions of POMs (and I do that
in every project today e.g. in combination with flatten-maven-plugin) I
always get warnings from Maven saying:
[WARNING] Some problems were encountered while building the effective
model for ...
[WARNING] 'version' contains an expression but should be a constant.
Still I see no clear picture how maven will adress these needs that
obviously many developer seem to have.
Issues like MNG-4161, MNG-624 and others were simply closed and not fixed.
After just seeing that and levaing an angry comment, I was about to cancel
this mail. However, I just decided to still send it but have little
expectation that it will make any sense...
Best regards
Jörg
-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org
Re: Maven and DevOps
Hi there, thanks for all your feedback. Just to clarify: * I do not want to really discuss the processes of mojohaus or maven community projects with this thread. If you get inspired to simplify by this discussion like Tamás (thanks) even better - but mojohaus process discussion should of course take place on mojohaus dev at googlegroups and even for maven dev out of this thread. * I do not want to blame anybody or say the community did not already do a lot of automation. Thanks for your effort so far and all that has already been archieved. * I understand that what I was actually proposing is partially covered by releasing SNAPSHOT versions to an internal but pulically available repo. However, see all the discussions on the web that imply that people would like a feature to "relabel" an already build release (e.g. some RC5) as the final release without rebuilding (as nothing has to change and all qualitiy rounds have passed). What I want to do is challenge you to think further and show some potential that could be archived with some improvements in the maven tool ecosystem. What I wanted to say is that I see still a lot of room for improvement in simplification of deployment: * Instead of creating a PGP key, registering that key, getting permissions and passwords for deployment repository, logging into nexus after deployment, searching for the staging repository and closing it, voting, waiting, refreshing, publishing, etc. this could all be completely automated by a smart CI chain (what all the DevOps deployment pipelines are about). Deployers would then only require permission to a release branch (or to accept PRs) and the rest would be covered by CI. And this can already be done today with all the tools available. Still traditional versioning, (automatic) tagging, etc. could take place and good old voting processes (on PRs instead) could be kept. This is not what maven or mojohaus seems to aim at but possible. In this context you should also be aware that some releases have been messed up because platform differences (EOL-style, etc.) and a lot of other things can go wrong on the machine of committers causing support effort for the community (still the maven site update fails all the time for me at mojohaus if I build on windows, also for maven-release-plugin I can only build releases with strange workarounds, etc. but all off-topic). * What I was actually requesting to think about for the tooling is probably more a matter of maven repo tools such as nexus or artifactory. So I guess also the wrong place here. I have seen all the great potential in social and open collaboration. For this reason I would like to see an advanced toolchain that publishes the release of every successful CI build with a technical "version" build from timestamp and potentially the revision of the code repo. That repository should then have social features allowing everybody to give feedback to any artefact. Then the actual releasing could be just a "relabelling" of the technical version to a human readable version. I was thinking of a human readable version just as an alias or "symbolic link" to a technial version. * Assume maven would habe been designed from the scratch in a way that the "version" is always a timestamp+revision and the human readable version (4.1.0.RELEASE) is just a link to that technical version (just similar to what is done with multiple SNAPSHOTs of the same version today)... I personally think this would solve all the relabelling version issue I stated above. * My entire vision was to be able to mark an artifact with valuable metadata such as taggig it as bad (insecure, flawed, or whatever) so I could get warnings whenever I use that artifact in my project (instead of OWAPS dependency-check that quadruples the build time). Also a repository could collect a lot of meta-information (how many downloads of each artifact have been done, the maven client could also send an anonymized hash of the groupId of the project requesting the dependency so you collect statistics that can tell how many projects are using a particular artifact, etc.). Assume you could directly see how many projects are using an open source artifact - just one? Or 100? 1000? Quite valuable infos... Still I hope I could put seeds to some inspiration... I do not expect anyone here to change maven or nexus codebase due to my comments. :) However, in my business project I leveraged a great potential with an entirely automated deployment pipleline (that also installs the release on test and staging environment after integration tests passed). Many years ago the Open Source teams were sometimes way ahead of what was done in business IT projects but this has IMHO inverted now... Cheers Jörg Am 02.05.2016 um 19:06 schrieb Hervé BOUTEMY: I don't really understand yes, http://www.mojohaus.org/development/performing-a-release.html is heavy But if you look at the
Re: [VOTE] Release Apache Maven JAR Plugin version 3.0.0 (Take 3)
Hi Robert, I'm diving more into this... I have tested another round with Windows XP (Maven 3.0.5, 3.1.1, 3.2.5, 3.3.9) , Windows 8.1 (Java 1.8) Maven 3.0.5, 3.1.1, 3.2.5, 3.3.9 no issue found... So the question is: Can you reproduce this? On your machine ? Have you seen this only in that single IT (manifest-content)? Or have you observed that in others as well ? I will check this later on a Windows 10 system as well... Kind regards Karl Heinz Marbaise On 5/7/16 11:06 PM, Robert Scholte wrote: I'm using Win10 The funny thing: Manifest-Version: 1.0 Implementation-Vendor: The Apache Software Foundation Implementation-Title: Apache Maven JAR Plugin Implementation-Version: 3.0.0 Implementation-Vendor-Id: org.apache.maven.plugins Built-By: Robert Scholte Build-Jdk: 1.7.0_55 Specification-Vendor: The Apache Software Foundation Specification-Title: Apache Maven JAR Plugin Created-By: Apache Maven 3.3.9 Specification-Version: 3.0.0 Archiver-Version: Plexus Archiver So the version is set correctly in the plugin jar. At least one IT-project should have failed because of the missing version, so it seems there was no additional IT written to check the value of Created-By: Where should this maven.version come from? Robert On Sat, 07 May 2016 22:54:34 +0200, Karl Heinz Marbaise wrote: Hi Robert, On 5/7/16 10:24 PM, Robert Scholte wrote: This is my content of maven-jar-plugin-3.0.0\target\it\manifest-content\target\manifest-content-1.0-SNAPSHOT.jar!/META-INF/MANIFEST.MF Manifest-Version: 1.0 Implementation-Vendor: jar plugin it Implementation-Title: manifest-content-it Implementation-Version: 1.0-SNAPSHOT Implementation-Vendor-Id: org.apache.maven.plugins Built-By: Robert Scholte Build-Jdk: 1.7.0_55 Specification-Vendor: jar plugin it Specification-Title: manifest-content-it Created-By: Apache Maven Specification-Version: 1.0 Main-Class: myproject.HelloWorld Archiver-Version: Plexus Archiver build.log show [DEBUG] Populating class realm plugin>org.apache.maven.plugins:maven-jar-plugin:3.0.0 [DEBUG] Included: org.apache.maven.plugins:maven-jar-plugin:jar:3.0.0 [DEBUG] Included: org.sonatype.sisu:sisu-inject-bean:jar:1.4.2 [DEBUG] Included: org.sonatype.sisu:sisu-guice:jar:noaop:2.1.7 [DEBUG] Included: org.sonatype.aether:aether-util:jar:1.7 [DEBUG] Included: org.codehaus.plexus:plexus-interpolation:jar:1.14 [DEBUG] Included: org.codehaus.plexus:plexus-component-annotations:jar:1.5.5 [DEBUG] Included: org.sonatype.plexus:plexus-sec-dispatcher:jar:1.3 [DEBUG] Included: org.sonatype.plexus:plexus-cipher:jar:1.4 [DEBUG] Included: org.apache.maven:maven-archiver:jar:3.0.2 [DEBUG] Included: org.apache.maven.shared:maven-shared-utils:jar:3.0.0 [DEBUG] Included: commons-io:commons-io:jar:2.4 [DEBUG] Included: com.google.code.findbugs:jsr305:jar:2.0.1 [DEBUG] Included: org.codehaus.plexus:plexus-archiver:jar:3.1.1 [DEBUG] Included: org.codehaus.plexus:plexus-io:jar:2.7.1 [DEBUG] Included: org.apache.commons:commons-compress:jar:1.10 [DEBUG] Included: org.iq80.snappy:snappy:jar:0.4 [DEBUG] Included: org.codehaus.plexus:plexus-utils:jar:3.0.22 Hm...the list is exactly the same i have here on Mac OS so the components are there...so only the behaviour in the components can make the difference... So i have tested on Windows XP with Maven 3.2.1 and 3.3.9 and it works correctly... Manifest-Version: 1.0 Implementation-Vendor: jar plugin it Implementation-Title: manifest-content-it Implementation-Version: 1.0-SNAPSHOT Implementation-Vendor-Id: org.apache.maven.plugins Built-By: kama Build-Jdk: 1.7.0_55 Specification-Vendor: jar plugin it Specification-Title: manifest-content-it Created-By: Apache Maven 3.3.9 Specification-Version: 1.0 Main-Class: myproject.HelloWorld Archiver-Version: Plexus Archiver Hm... I haven't installed the plugin in my repository. Could it be a Windows issue? Robert On Sat, 07 May 2016 21:04:19 +0200, Karl Heinz Marbaise wrote: Hi Robert, On 5/7/16 8:35 PM, Robert Scholte wrote: https://issues.apache.org/jira/browse/MJAR-218 is this really fixed? I still see a versionless Apache Maven in the MANIFEST.MF Hm...I tested with Maven 3.0.5 and 3.3.9 via mvn -Prun-its clean verify and checked the generated JAR files which are Ok Furthermore i run Maven 3.3.9 and also: mvn -Prun-its clean verify -Dinvoker.mavenHome=/usr/share/java/apache-maven-3.0.5 which produces correct artifacts Unfortunately i can not reproduce this at the moment...But it if is not..So it means making a 3.0.1 release very soon... Can you give some examples of files which do not contain the correct: "Created-By: " entry? Kind regards Karl Heinz Marbaise Executed as d:\apache-maven-3.3.9\bin\mvn clean verify -Prun-its -Dinvoker.mavenHome=d:\apache-maven-3.0 thanks, Robert On Wed, 04 May 2016 20:16:20 +0200, Karl Heinz Marbaise wrote: Hi, We solved 21 issues: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12317526&version=1
