Re: Skill set to maintain a enterprise build system using Maven
Hi Thank you for the great advice, I can come to a conclusion to train a team internally from ground up -Dan On Sun, May 31, 2015 at 11:21 PM, Mirko Friedenhagen < mfriedenha...@gmail.com> wrote: > Hello Dan, > > I started as a "normal" Python developer, then built and led a team of test > automation engineers while the management decided to switch to Java 10 > years back and inherited CruiseControl, which we dropped for Hudson. I had > some small scale operating experience as well (email server/router). One of > my nowadays colleagues has been test automation engineer as well, the other > one was always into building completely. > > I think because release processes are often very company/product specific > (at least when you got a huge legacy baggage), building engineers are > easiest trained or formed internally. However, without input from the > outside, you might miss fresh ideas. > > Regards > Mirko > -- > Sent from my mobile > Am 01.06.2015 02:59 schrieb "Dan Tran" : > > > > > > > > > > I'm sure I've said it before, but part of Maven's problem is that this > is > > > all magically taken care of behind the scenes and less people need to > > know > > > how it works to make it work. > > > The downside is that there are then less people who can fix things when > > > they need fixing. > > > > > > > Exactly, it is hard to learn magical thing > > > > I used to train a QA turned dev with java j2ee server testing backround, > > and > > it took a couple of years > > > > is it norm here by slowly training internally from start ( ie java)? > > > > Thanks > > > > -Dan > > > > BTW, i full understand that this type of discussion my be touchy, so I > also > > very appreciated to any one who is able to share with private email > > >
Re: Skill set to maintain a enterprise build system using Maven
Hello Dan, I started as a "normal" Python developer, then built and led a team of test automation engineers while the management decided to switch to Java 10 years back and inherited CruiseControl, which we dropped for Hudson. I had some small scale operating experience as well (email server/router). One of my nowadays colleagues has been test automation engineer as well, the other one was always into building completely. I think because release processes are often very company/product specific (at least when you got a huge legacy baggage), building engineers are easiest trained or formed internally. However, without input from the outside, you might miss fresh ideas. Regards Mirko -- Sent from my mobile Am 01.06.2015 02:59 schrieb "Dan Tran" : > > > > > > I'm sure I've said it before, but part of Maven's problem is that this is > > all magically taken care of behind the scenes and less people need to > know > > how it works to make it work. > > The downside is that there are then less people who can fix things when > > they need fixing. > > > > Exactly, it is hard to learn magical thing > > I used to train a QA turned dev with java j2ee server testing backround, > and > it took a couple of years > > is it norm here by slowly training internally from start ( ie java)? > > Thanks > > -Dan > > BTW, i full understand that this type of discussion my be touchy, so I also > very appreciated to any one who is able to share with private email >
Re: Skill set to maintain a enterprise build system using Maven
> > > I'm sure I've said it before, but part of Maven's problem is that this is > all magically taken care of behind the scenes and less people need to know > how it works to make it work. > The downside is that there are then less people who can fix things when > they need fixing. > Exactly, it is hard to learn magical thing I used to train a QA turned dev with java j2ee server testing backround, and it took a couple of years is it norm here by slowly training internally from start ( ie java)? Thanks -Dan BTW, i full understand that this type of discussion my be touchy, so I also very appreciated to any one who is able to share with private email
Re: Skill set to maintain a enterprise build system using Maven
On 1 June 2015 at 09:40, Dan Tran wrote: > For my case, I am very fortunate to involve with the the product from early > day (a year ba ck) and Maven is embraced to the max where plugins are > developed to solve every build use case in a full dev/qa/releng integration > pipeline. The developments are multi-sites and heavily depending Maven > Central like repository cluster and Maven release plugin. > > The odd thing to me is it is hard to find Java talent with a passion for > build/ci with Maven, and it is also hard to switch perl/python devops into > java to maintain Maven build. am I wrong here? The odd thing to me is it is hard to find *any* talent with a passion for build/ci with *anything*. FTFY. Its a niche market, and one that lots of people still dont see the value in. Getting them to be excited about something tangential to development is difficult - its like eating healthy and excerise, people know they should be doing it but they aint. I *still* see people attempting to develop software with no version control, and hand crafted outputs from IDEs on their local machines. We even get contractors coming in wanting to use VMs in dev that they "promote" into production *shudder*. They best you can do is to hope you can train your people internally about why this is all good. I'm sure I've said it before, but part of Maven's problem is that this is all magically taken care of behind the scenes and less people need to know how it works to make it work. The downside is that there are then less people who can fix things when they need fixing.
Re: Skill set to maintain a enterprise build system using Maven
For my case, I am very fortunate to involve with the the product from early day (a year ba ck) and Maven is embraced to the max where plugins are developed to solve every build use case in a full dev/qa/releng integration pipeline. The developments are multi-sites and heavily depending Maven Central like repository cluster and Maven release plugin. The odd thing to me is it is hard to find Java talent with a passion for build/ci with Maven, and it is also hard to switch perl/python devops into java to maintain Maven build. am I wrong here? Thanks -Dan On Sun, May 31, 2015 at 3:16 PM, Benson Margulies wrote: > First, treating build as a separate discipline from code is, in my > experience, a recipe for trouble. The poms or build.xml or whatever > files are just as much part of the source code as the java. Someone > may own Jenkins or whatever, but the devs should own the building of > the code they write. > > Second, the environmental impact of Maven depends critically on the > nature of the code. If your entire build is composed of compiling Java > code and delivering jar files, the poms are trivial and the Maven cost > is near-zero. If, on the other extreme, you insist on fighting with > Maven and having policies for dependencies, or releases, or whatever, > that require extensive 'negotiation', it's a very different picture. > In between, if you have significant custom build activity (e.g. code > generators), you might need some of your own plugins. > > - > To unsubscribe, e-mail: users-unsubscr...@maven.apache.org > For additional commands, e-mail: users-h...@maven.apache.org > >
Re: Skill set to maintain a enterprise build system using Maven
First, treating build as a separate discipline from code is, in my experience, a recipe for trouble. The poms or build.xml or whatever files are just as much part of the source code as the java. Someone may own Jenkins or whatever, but the devs should own the building of the code they write. Second, the environmental impact of Maven depends critically on the nature of the code. If your entire build is composed of compiling Java code and delivering jar files, the poms are trivial and the Maven cost is near-zero. If, on the other extreme, you insist on fighting with Maven and having policies for dependencies, or releases, or whatever, that require extensive 'negotiation', it's a very different picture. In between, if you have significant custom build activity (e.g. code generators), you might need some of your own plugins. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Skill set to maintain a enterprise build system using Maven
On 30/05/2015 10:44 PM, Dan Tran wrote: Hi Ron One person may not be desirable since he/she may win a lottery :-) I sell Learning Management Systems that include Talent and Succession Planning so I will just say that you should be able to find a backup in your pool of potential successors for any key position.:-) The actual change to a POM is not a high skill job once you have the project's policies and initial POMs in place. Developers should not change the POM who would? RelEng? It should be set up by a senior architect or architectural team that is making and supporting the team's policies for dependency selection and release strategy. We had pretty good success with simple POMs that individual developers were not permitted (not much enforcement required) to alter. If a developer has a need for a new dependency or a change in version, it has to be a team decision with the project manager's OK. Ron Thanks -Dan On Sat, May 30, 2015 at 3:22 PM, Ron Wheeler wrote: On 30/05/2015 1:29 AM, Dan Tran wrote: Hi I would like to ask if the community can share with me what it takes to maintain an enterprise build system with continuous integration of 100+ developer + QA and growing using Maven. The build system contains many components with their own release cycle and they do integrate together. - is java skill set to develop plugin a must? a) Most projects should not need to develop plug-ins. b) This is a one-time need if it exists and there are companies and consultants available to do this for you - do you have a team or just a few of deep understanding of Maven developers? One personwill do. - will a none java RelEng able to perform Maven release? Sure - does your RelEng maintains the pom or developers? developers should not touch POMs - what are your challenges? a) adjusting your project structure and methodology to incorporate "the maven way" b) resisting the impulse to try to impose an existing mindset about development and build processes on Maven Thanks -Dan -- Ron Wheeler President Artifact Software Inc email: rwhee...@artifact-software.com skype: ronaldmwheeler phone: 866-970-2435, ext 102 - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org -- Ron Wheeler President Artifact Software Inc email: rwhee...@artifact-software.com skype: ronaldmwheeler phone: 866-970-2435, ext 102 - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Skill set to maintain a enterprise build system using Maven
Hi Curtis,
It is an awesome info and process, I have a long way to catch up
Thanks
-Dan
On Sun, May 31, 2015 at 2:36 AM, Curtis Rueden wrote:
> Hi Dan,
>
> > PS. Would love to hear other experiences from community rather me
> > sucking out Mirko's :-)
>
> Not sure how relevant my scenario is, but here goes:
>
> My group consists of an international collaboration of OSS developers at
> universities etc., rather than a company. But a lot of our needs are the
> same.
>
> - We have 2-3 core "DevOps" who maintain our Maven architecture
> ("corporate" POM [1] etc.), with several others in the community who have
> been trained enough on how things work to understand and occasionally
> contribute to it.
>
> - The projects encompass several GitHub orgs, each of which has its own
> primary maintainer(s) who at least understand how their Maven POM hierarchy
> works.
>
> - We have Jenkins jobs for everything. http://jenkins.imagej.net/
>
> - We use Nexus to manage our Maven artifacts. http://maven.imagej.net/
>
> > - is java skill set to develop plugin a must?
>
> No, although it is occasionally useful. I think knowing Java can make you a
> better Maven community member though because you can contribute bug-fixes
> to the plugins you find useful. E.g., we use license-maven-plugin, but it
> wouldn't have behaved the way we needed if I hadn't helped it get patched
> upstream.
>
> > - do you have a team or just a few of deep understanding of Maven
> developers?
>
> 2-3 with "deep" understanding and several others worldwide with "moderate"
> understanding has been sufficient for us. And a whole community with next
> to no understanding of Maven, for whom things "just work" in their IDE of
> choice.
>
> > - will a none java RelEng able to perform Maven release?
>
> I don't see why not, although in my scenario everyone who knows about Maven
> also knows about Java. So for us it is the converse: lots of Java
> developers who know little about Maven but want to do their own releases.
> And the beauty of it is: they can! http://imagej.net/Releases
>
> > - does your RelEng maintains the pom or developers?
>
> My group has no dedicated RelEng—just two core developers who also wear the
> RelEng hat among many others. In practice, for the most part, the same
> people who ultimately maintain the most root-level POMs also end up doing
> the releases of the most root-level software projects.
>
> > - what are your challenges?
>
> Maven makes it possible for a small number of developers to maintain 300+
> Git repositories. This is a two edged sword, though: Maven only solves the
> project management side of things, not the other aspects (software
> design/architecture, coding of new features, community support, etc.).
>
> It is important to note that there are many project management challenges
> that Maven makes easier to solve but does not itself fully solve in a
> turn-key way. In particular, continuous integration (thank you Jenkins!)
> and versioning (we use SemVer and carefully managed BOMs) spring to mind.
>
> Since everything we do is OSS, if you care you can read about it yourself:
>
> http://imagej.net/Architecture
> http://imagej.net/Governance
>
> Regards,
> Curtis
>
> On Sun, May 31, 2015 at 4:06 AM, Dan Tran wrote:
>
> > Hi Mirko,
> >
> > Looks like the topic getting very interesting here, but I would like to
> > refocus on the skillset
> >
> > How did you form up your team?, did you all pickup Java before joining
> this
> > devops team?
> >
> > Thanks for all the sharing
> >
> > -Dan
> >
> > PS. Would love to hear other experiences from community rather me
> sucking
> > out Mirko's :-)
> >
> >
> > On Sun, May 31, 2015 at 1:03 AM, Mirko Friedenhagen <
> > mfriedenha...@gmail.com
> > > wrote:
> >
> > > Hello Dan,
> > >
> > > we treat tooling like software as well. Ticket creation is an
> automated 2
> > > click process and the package qa will not take more than 5 minutes for
> > > small changes.
> > >
> > > External libraries from central may be used at free will, but we
> > recommend
> > > stuff in a so called toolbox, these dependencies are managed in the
> > > department pom.
> > >
> > > The (programming) architects and we help discovering alternatives in
> our
> > > toolbox, stuff from repositories outside central is mostly put in a
> > > third-party repo in Artifactory.
> > >
> > > Regards
> > > Mirko
> > > --
> > > Sent from my mobile
> > > Am 31.05.2015 00:06 schrieb "Dan Tran" :
> > >
> > > > Hi Mirko
> > > >
> > > > Looks like you have Artifactory to store all of release artifacts and
> > > > another 'release' repo to store the final approved release
> > > >
> > > > Is internal tooling, thirdparty upload going thru the same release
> > > > process?
> > > >
> > > > Thanks
> > > >
> > > > -Dan
> > > >
> > > > On Sat, May 30, 2015 at 2:41 PM, Mirko Friedenhagen <
> > > > mfriedenha...@gmail.com
> > > > > wrote:
> > > >
> > > > > Hello Dan,
> > > > >
> > > > > - Every developer may deploy SNAPSHOTs, howe
Re: Skill set to maintain a enterprise build system using Maven
Hi Dan,
> PS. Would love to hear other experiences from community rather me
> sucking out Mirko's :-)
Not sure how relevant my scenario is, but here goes:
My group consists of an international collaboration of OSS developers at
universities etc., rather than a company. But a lot of our needs are the
same.
- We have 2-3 core "DevOps" who maintain our Maven architecture
("corporate" POM [1] etc.), with several others in the community who have
been trained enough on how things work to understand and occasionally
contribute to it.
- The projects encompass several GitHub orgs, each of which has its own
primary maintainer(s) who at least understand how their Maven POM hierarchy
works.
- We have Jenkins jobs for everything. http://jenkins.imagej.net/
- We use Nexus to manage our Maven artifacts. http://maven.imagej.net/
> - is java skill set to develop plugin a must?
No, although it is occasionally useful. I think knowing Java can make you a
better Maven community member though because you can contribute bug-fixes
to the plugins you find useful. E.g., we use license-maven-plugin, but it
wouldn't have behaved the way we needed if I hadn't helped it get patched
upstream.
> - do you have a team or just a few of deep understanding of Maven
developers?
2-3 with "deep" understanding and several others worldwide with "moderate"
understanding has been sufficient for us. And a whole community with next
to no understanding of Maven, for whom things "just work" in their IDE of
choice.
> - will a none java RelEng able to perform Maven release?
I don't see why not, although in my scenario everyone who knows about Maven
also knows about Java. So for us it is the converse: lots of Java
developers who know little about Maven but want to do their own releases.
And the beauty of it is: they can! http://imagej.net/Releases
> - does your RelEng maintains the pom or developers?
My group has no dedicated RelEng—just two core developers who also wear the
RelEng hat among many others. In practice, for the most part, the same
people who ultimately maintain the most root-level POMs also end up doing
the releases of the most root-level software projects.
> - what are your challenges?
Maven makes it possible for a small number of developers to maintain 300+
Git repositories. This is a two edged sword, though: Maven only solves the
project management side of things, not the other aspects (software
design/architecture, coding of new features, community support, etc.).
It is important to note that there are many project management challenges
that Maven makes easier to solve but does not itself fully solve in a
turn-key way. In particular, continuous integration (thank you Jenkins!)
and versioning (we use SemVer and carefully managed BOMs) spring to mind.
Since everything we do is OSS, if you care you can read about it yourself:
http://imagej.net/Architecture
http://imagej.net/Governance
Regards,
Curtis
On Sun, May 31, 2015 at 4:06 AM, Dan Tran wrote:
> Hi Mirko,
>
> Looks like the topic getting very interesting here, but I would like to
> refocus on the skillset
>
> How did you form up your team?, did you all pickup Java before joining this
> devops team?
>
> Thanks for all the sharing
>
> -Dan
>
> PS. Would love to hear other experiences from community rather me sucking
> out Mirko's :-)
>
>
> On Sun, May 31, 2015 at 1:03 AM, Mirko Friedenhagen <
> mfriedenha...@gmail.com
> > wrote:
>
> > Hello Dan,
> >
> > we treat tooling like software as well. Ticket creation is an automated 2
> > click process and the package qa will not take more than 5 minutes for
> > small changes.
> >
> > External libraries from central may be used at free will, but we
> recommend
> > stuff in a so called toolbox, these dependencies are managed in the
> > department pom.
> >
> > The (programming) architects and we help discovering alternatives in our
> > toolbox, stuff from repositories outside central is mostly put in a
> > third-party repo in Artifactory.
> >
> > Regards
> > Mirko
> > --
> > Sent from my mobile
> > Am 31.05.2015 00:06 schrieb "Dan Tran" :
> >
> > > Hi Mirko
> > >
> > > Looks like you have Artifactory to store all of release artifacts and
> > > another 'release' repo to store the final approved release
> > >
> > > Is internal tooling, thirdparty upload going thru the same release
> > > process?
> > >
> > > Thanks
> > >
> > > -Dan
> > >
> > > On Sat, May 30, 2015 at 2:41 PM, Mirko Friedenhagen <
> > > mfriedenha...@gmail.com
> > > > wrote:
> > >
> > > > Hello Dan,
> > > >
> > > > - Every developer may deploy SNAPSHOTs, however this is normally done
> > > > by Jenkins.
> > > > - We do not enforce staging from Jenkins, however almost all projects
> > > > do this. We do not enforce this, so Jenkins outages do not inhibit
> > > > releasing hot fixes.
> > > > - Releases are deployed to a staging repository in Artifactory and we
> > > > have a process called package-qa where for every staged release a
> > > > corresponding JIR
Re: Skill set to maintain a enterprise build system using Maven
Hi Mirko, Looks like the topic getting very interesting here, but I would like to refocus on the skillset How did you form up your team?, did you all pickup Java before joining this devops team? Thanks for all the sharing -Dan PS. Would love to hear other experiences from community rather me sucking out Mirko's :-) On Sun, May 31, 2015 at 1:03 AM, Mirko Friedenhagen wrote: > Hello Dan, > > we treat tooling like software as well. Ticket creation is an automated 2 > click process and the package qa will not take more than 5 minutes for > small changes. > > External libraries from central may be used at free will, but we recommend > stuff in a so called toolbox, these dependencies are managed in the > department pom. > > The (programming) architects and we help discovering alternatives in our > toolbox, stuff from repositories outside central is mostly put in a > third-party repo in Artifactory. > > Regards > Mirko > -- > Sent from my mobile > Am 31.05.2015 00:06 schrieb "Dan Tran" : > > > Hi Mirko > > > > Looks like you have Artifactory to store all of release artifacts and > > another 'release' repo to store the final approved release > > > > Is internal tooling, thirdparty upload going thru the same release > > process? > > > > Thanks > > > > -Dan > > > > On Sat, May 30, 2015 at 2:41 PM, Mirko Friedenhagen < > > mfriedenha...@gmail.com > > > wrote: > > > > > Hello Dan, > > > > > > - Every developer may deploy SNAPSHOTs, however this is normally done > > > by Jenkins. > > > - We do not enforce staging from Jenkins, however almost all projects > > > do this. We do not enforce this, so Jenkins outages do not inhibit > > > releasing hot fixes. > > > - Releases are deployed to a staging repository in Artifactory and we > > > have a process called package-qa where for every staged release a > > > corresponding JIRA ticket has to be created with information (changes, > > > Wiki page, diff link since last release, ticket queue). This is a > > > central place where you may see all releases in one place. > > > - This ticket is parsed by a script from Jenkins which procures > > > artifacts from staging to a releases repository and adds general > > > quality information from SonarQube and Jenkins as well as the SHA1 > > > sums to the ticket so we have a second record which may be used to > > > detect forgery. Additionally the script checks for the existence of > > > the SCM tag and retrieves the number of changed lines between > > > releases. > > > - No blocker or critical and no new major issues are allowed in > > > SonarQube, otherwise procurement will fail. > > > - The reporter and the "mover" have to be different persons to enforce > > > a "four eyes" principle. > > > - The "mover" (sometimes someone from development QA, most of the > > > times nowadays another developer) has to check some things and must > > > inspect the diff to detect whether all changes are explained. > > > - Our operations teams will only pick production releases from the > > > final releases repository, other stages may pick up artifacts from the > > > staging repository. > > > - We do not sign artifacts. > > > > > > Regards > > > Mirko > > > Regards Mirko > > > -- > > > http://illegalstateexception.blogspot.com/ > > > https://github.com/mfriedenhagen/ (http://osrc.dfm.io/mfriedenhagen) > > > https://bitbucket.org/mfriedenhagen/ > > > > > > > > > On Sat, May 30, 2015 at 7:31 PM, Dan Tran wrote: > > > > Thanks Mirko > > > > > > > > * What about snapshot and release policy, do developers/qa have > > access > > > to > > > > deploy snapshot and release artifacts? > > > > * do you use artifact signing similar to Maven Central? > > > > > > > > > > > > Thanks again > > > > > > > > -Dan > > > > > > > > On Sat, May 30, 2015 at 9:21 AM, Mirko Friedenhagen < > > > mfriedenha...@gmail.com > > > >> wrote: > > > > > > > >> What I forgot: > > > >> > > > >> patience, social skills and remembering that not every application > > > >> developer needs to be a build specialist are important as well :-) > > > >> > > > >> Regards > > > >> Mirko > > > >> -- > > > >> Sent from my mobile > > > >> Am 30.05.2015 07:29 schrieb "Dan Tran" : > > > >> > > > >> > Hi > > > >> > > > > >> > > > > >> > I would like to ask if the community can share with me what it > > takes > > > to > > > >> > maintain an enterprise build system with continuous integration of > > > 100+ > > > >> > developer + QA and growing using Maven. The build system contains > > > many > > > >> > components with their own release cycle and they do integrate > > > together. > > > >> > > > > >> > > > > >> >- is java skill set to develop plugin a must? > > > >> >- do you have a team or just a few of deep understanding of > Maven > > > >> > developers? > > > >> >- will a none java RelEng able to perform Maven release? > > > >> >- does your RelEng maintains the pom or developers? > > > >> >- what are your challenges? > > > >> > > > > >> > Thanks > > > >> > > > > >> > -Dan >
Re: Skill set to maintain a enterprise build system using Maven
Hello Dan, we treat tooling like software as well. Ticket creation is an automated 2 click process and the package qa will not take more than 5 minutes for small changes. External libraries from central may be used at free will, but we recommend stuff in a so called toolbox, these dependencies are managed in the department pom. The (programming) architects and we help discovering alternatives in our toolbox, stuff from repositories outside central is mostly put in a third-party repo in Artifactory. Regards Mirko -- Sent from my mobile Am 31.05.2015 00:06 schrieb "Dan Tran" : > Hi Mirko > > Looks like you have Artifactory to store all of release artifacts and > another 'release' repo to store the final approved release > > Is internal tooling, thirdparty upload going thru the same release > process? > > Thanks > > -Dan > > On Sat, May 30, 2015 at 2:41 PM, Mirko Friedenhagen < > mfriedenha...@gmail.com > > wrote: > > > Hello Dan, > > > > - Every developer may deploy SNAPSHOTs, however this is normally done > > by Jenkins. > > - We do not enforce staging from Jenkins, however almost all projects > > do this. We do not enforce this, so Jenkins outages do not inhibit > > releasing hot fixes. > > - Releases are deployed to a staging repository in Artifactory and we > > have a process called package-qa where for every staged release a > > corresponding JIRA ticket has to be created with information (changes, > > Wiki page, diff link since last release, ticket queue). This is a > > central place where you may see all releases in one place. > > - This ticket is parsed by a script from Jenkins which procures > > artifacts from staging to a releases repository and adds general > > quality information from SonarQube and Jenkins as well as the SHA1 > > sums to the ticket so we have a second record which may be used to > > detect forgery. Additionally the script checks for the existence of > > the SCM tag and retrieves the number of changed lines between > > releases. > > - No blocker or critical and no new major issues are allowed in > > SonarQube, otherwise procurement will fail. > > - The reporter and the "mover" have to be different persons to enforce > > a "four eyes" principle. > > - The "mover" (sometimes someone from development QA, most of the > > times nowadays another developer) has to check some things and must > > inspect the diff to detect whether all changes are explained. > > - Our operations teams will only pick production releases from the > > final releases repository, other stages may pick up artifacts from the > > staging repository. > > - We do not sign artifacts. > > > > Regards > > Mirko > > Regards Mirko > > -- > > http://illegalstateexception.blogspot.com/ > > https://github.com/mfriedenhagen/ (http://osrc.dfm.io/mfriedenhagen) > > https://bitbucket.org/mfriedenhagen/ > > > > > > On Sat, May 30, 2015 at 7:31 PM, Dan Tran wrote: > > > Thanks Mirko > > > > > > * What about snapshot and release policy, do developers/qa have > access > > to > > > deploy snapshot and release artifacts? > > > * do you use artifact signing similar to Maven Central? > > > > > > > > > Thanks again > > > > > > -Dan > > > > > > On Sat, May 30, 2015 at 9:21 AM, Mirko Friedenhagen < > > mfriedenha...@gmail.com > > >> wrote: > > > > > >> What I forgot: > > >> > > >> patience, social skills and remembering that not every application > > >> developer needs to be a build specialist are important as well :-) > > >> > > >> Regards > > >> Mirko > > >> -- > > >> Sent from my mobile > > >> Am 30.05.2015 07:29 schrieb "Dan Tran" : > > >> > > >> > Hi > > >> > > > >> > > > >> > I would like to ask if the community can share with me what it > takes > > to > > >> > maintain an enterprise build system with continuous integration of > > 100+ > > >> > developer + QA and growing using Maven. The build system contains > > many > > >> > components with their own release cycle and they do integrate > > together. > > >> > > > >> > > > >> >- is java skill set to develop plugin a must? > > >> >- do you have a team or just a few of deep understanding of Maven > > >> > developers? > > >> >- will a none java RelEng able to perform Maven release? > > >> >- does your RelEng maintains the pom or developers? > > >> >- what are your challenges? > > >> > > > >> > Thanks > > >> > > > >> > -Dan > > >> > > > >> > > > > - > > To unsubscribe, e-mail: users-unsubscr...@maven.apache.org > > For additional commands, e-mail: users-h...@maven.apache.org > > > > >
Re: Skill set to maintain a enterprise build system using Maven
Hi Ron One person may not be desirable since he/she may win a lottery :-) Developers should not change the POM who would? RelEng? Thanks -Dan On Sat, May 30, 2015 at 3:22 PM, Ron Wheeler wrote: > On 30/05/2015 1:29 AM, Dan Tran wrote: > >> Hi >> >> >> I would like to ask if the community can share with me what it takes to >> maintain an enterprise build system with continuous integration of 100+ >> developer + QA and growing using Maven. The build system contains many >> components with their own release cycle and they do integrate together. >> >> >> - is java skill set to develop plugin a must? >> > a) Most projects should not need to develop plug-ins. > b) This is a one-time need if it exists and there are companies and > consultants available to do this for you > >> - do you have a team or just a few of deep understanding of Maven >> developers? >> > One personwill do. > >> - will a none java RelEng able to perform Maven release? >> > Sure > >> - does your RelEng maintains the pom or developers? >> > developers should not touch POMs > >> - what are your challenges? >> > a) adjusting your project structure and methodology to incorporate "the > maven way" > b) resisting the impulse to try to impose an existing mindset about > development and build processes on Maven > > Thanks >> >> -Dan >> >> > > -- > Ron Wheeler > President > Artifact Software Inc > email: rwhee...@artifact-software.com > skype: ronaldmwheeler > phone: 866-970-2435, ext 102 > > > > - > To unsubscribe, e-mail: users-unsubscr...@maven.apache.org > For additional commands, e-mail: users-h...@maven.apache.org > >
Re: Skill set to maintain a enterprise build system using Maven
On 30/05/2015 1:29 AM, Dan Tran wrote: Hi I would like to ask if the community can share with me what it takes to maintain an enterprise build system with continuous integration of 100+ developer + QA and growing using Maven. The build system contains many components with their own release cycle and they do integrate together. - is java skill set to develop plugin a must? a) Most projects should not need to develop plug-ins. b) This is a one-time need if it exists and there are companies and consultants available to do this for you - do you have a team or just a few of deep understanding of Maven developers? One personwill do. - will a none java RelEng able to perform Maven release? Sure - does your RelEng maintains the pom or developers? developers should not touch POMs - what are your challenges? a) adjusting your project structure and methodology to incorporate "the maven way" b) resisting the impulse to try to impose an existing mindset about development and build processes on Maven Thanks -Dan -- Ron Wheeler President Artifact Software Inc email: rwhee...@artifact-software.com skype: ronaldmwheeler phone: 866-970-2435, ext 102 - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Skill set to maintain a enterprise build system using Maven
Hi Mirko Looks like you have Artifactory to store all of release artifacts and another 'release' repo to store the final approved release Is internal tooling, thirdparty upload going thru the same release process? Thanks -Dan On Sat, May 30, 2015 at 2:41 PM, Mirko Friedenhagen wrote: > Hello Dan, > > - Every developer may deploy SNAPSHOTs, however this is normally done > by Jenkins. > - We do not enforce staging from Jenkins, however almost all projects > do this. We do not enforce this, so Jenkins outages do not inhibit > releasing hot fixes. > - Releases are deployed to a staging repository in Artifactory and we > have a process called package-qa where for every staged release a > corresponding JIRA ticket has to be created with information (changes, > Wiki page, diff link since last release, ticket queue). This is a > central place where you may see all releases in one place. > - This ticket is parsed by a script from Jenkins which procures > artifacts from staging to a releases repository and adds general > quality information from SonarQube and Jenkins as well as the SHA1 > sums to the ticket so we have a second record which may be used to > detect forgery. Additionally the script checks for the existence of > the SCM tag and retrieves the number of changed lines between > releases. > - No blocker or critical and no new major issues are allowed in > SonarQube, otherwise procurement will fail. > - The reporter and the "mover" have to be different persons to enforce > a "four eyes" principle. > - The "mover" (sometimes someone from development QA, most of the > times nowadays another developer) has to check some things and must > inspect the diff to detect whether all changes are explained. > - Our operations teams will only pick production releases from the > final releases repository, other stages may pick up artifacts from the > staging repository. > - We do not sign artifacts. > > Regards > Mirko > Regards Mirko > -- > http://illegalstateexception.blogspot.com/ > https://github.com/mfriedenhagen/ (http://osrc.dfm.io/mfriedenhagen) > https://bitbucket.org/mfriedenhagen/ > > > On Sat, May 30, 2015 at 7:31 PM, Dan Tran wrote: > > Thanks Mirko > > > > * What about snapshot and release policy, do developers/qa have access > to > > deploy snapshot and release artifacts? > > * do you use artifact signing similar to Maven Central? > > > > > > Thanks again > > > > -Dan > > > > On Sat, May 30, 2015 at 9:21 AM, Mirko Friedenhagen < > mfriedenha...@gmail.com > >> wrote: > > > >> What I forgot: > >> > >> patience, social skills and remembering that not every application > >> developer needs to be a build specialist are important as well :-) > >> > >> Regards > >> Mirko > >> -- > >> Sent from my mobile > >> Am 30.05.2015 07:29 schrieb "Dan Tran" : > >> > >> > Hi > >> > > >> > > >> > I would like to ask if the community can share with me what it takes > to > >> > maintain an enterprise build system with continuous integration of > 100+ > >> > developer + QA and growing using Maven. The build system contains > many > >> > components with their own release cycle and they do integrate > together. > >> > > >> > > >> >- is java skill set to develop plugin a must? > >> >- do you have a team or just a few of deep understanding of Maven > >> > developers? > >> >- will a none java RelEng able to perform Maven release? > >> >- does your RelEng maintains the pom or developers? > >> >- what are your challenges? > >> > > >> > Thanks > >> > > >> > -Dan > >> > > >> > > - > To unsubscribe, e-mail: users-unsubscr...@maven.apache.org > For additional commands, e-mail: users-h...@maven.apache.org > >
Re: Skill set to maintain a enterprise build system using Maven
Hello Dan, - Every developer may deploy SNAPSHOTs, however this is normally done by Jenkins. - We do not enforce staging from Jenkins, however almost all projects do this. We do not enforce this, so Jenkins outages do not inhibit releasing hot fixes. - Releases are deployed to a staging repository in Artifactory and we have a process called package-qa where for every staged release a corresponding JIRA ticket has to be created with information (changes, Wiki page, diff link since last release, ticket queue). This is a central place where you may see all releases in one place. - This ticket is parsed by a script from Jenkins which procures artifacts from staging to a releases repository and adds general quality information from SonarQube and Jenkins as well as the SHA1 sums to the ticket so we have a second record which may be used to detect forgery. Additionally the script checks for the existence of the SCM tag and retrieves the number of changed lines between releases. - No blocker or critical and no new major issues are allowed in SonarQube, otherwise procurement will fail. - The reporter and the "mover" have to be different persons to enforce a "four eyes" principle. - The "mover" (sometimes someone from development QA, most of the times nowadays another developer) has to check some things and must inspect the diff to detect whether all changes are explained. - Our operations teams will only pick production releases from the final releases repository, other stages may pick up artifacts from the staging repository. - We do not sign artifacts. Regards Mirko Regards Mirko -- http://illegalstateexception.blogspot.com/ https://github.com/mfriedenhagen/ (http://osrc.dfm.io/mfriedenhagen) https://bitbucket.org/mfriedenhagen/ On Sat, May 30, 2015 at 7:31 PM, Dan Tran wrote: > Thanks Mirko > > * What about snapshot and release policy, do developers/qa have access to > deploy snapshot and release artifacts? > * do you use artifact signing similar to Maven Central? > > > Thanks again > > -Dan > > On Sat, May 30, 2015 at 9:21 AM, Mirko Friedenhagen > wrote: > >> What I forgot: >> >> patience, social skills and remembering that not every application >> developer needs to be a build specialist are important as well :-) >> >> Regards >> Mirko >> -- >> Sent from my mobile >> Am 30.05.2015 07:29 schrieb "Dan Tran" : >> >> > Hi >> > >> > >> > I would like to ask if the community can share with me what it takes to >> > maintain an enterprise build system with continuous integration of 100+ >> > developer + QA and growing using Maven. The build system contains many >> > components with their own release cycle and they do integrate together. >> > >> > >> >- is java skill set to develop plugin a must? >> >- do you have a team or just a few of deep understanding of Maven >> > developers? >> >- will a none java RelEng able to perform Maven release? >> >- does your RelEng maintains the pom or developers? >> >- what are your challenges? >> > >> > Thanks >> > >> > -Dan >> > >> - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Skill set to maintain a enterprise build system using Maven
Thanks Mirko * What about snapshot and release policy, do developers/qa have access to deploy snapshot and release artifacts? * do you use artifact signing similar to Maven Central? Thanks again -Dan On Sat, May 30, 2015 at 9:21 AM, Mirko Friedenhagen wrote: > What I forgot: > > patience, social skills and remembering that not every application > developer needs to be a build specialist are important as well :-) > > Regards > Mirko > -- > Sent from my mobile > Am 30.05.2015 07:29 schrieb "Dan Tran" : > > > Hi > > > > > > I would like to ask if the community can share with me what it takes to > > maintain an enterprise build system with continuous integration of 100+ > > developer + QA and growing using Maven. The build system contains many > > components with their own release cycle and they do integrate together. > > > > > >- is java skill set to develop plugin a must? > >- do you have a team or just a few of deep understanding of Maven > > developers? > >- will a none java RelEng able to perform Maven release? > >- does your RelEng maintains the pom or developers? > >- what are your challenges? > > > > Thanks > > > > -Dan > > >
Re: Skill set to maintain a enterprise build system using Maven
What I forgot: patience, social skills and remembering that not every application developer needs to be a build specialist are important as well :-) Regards Mirko -- Sent from my mobile Am 30.05.2015 07:29 schrieb "Dan Tran" : > Hi > > > I would like to ask if the community can share with me what it takes to > maintain an enterprise build system with continuous integration of 100+ > developer + QA and growing using Maven. The build system contains many > components with their own release cycle and they do integrate together. > > >- is java skill set to develop plugin a must? >- do you have a team or just a few of deep understanding of Maven > developers? >- will a none java RelEng able to perform Maven release? >- does your RelEng maintains the pom or developers? >- what are your challenges? > > Thanks > > -Dan >
Re: Skill set to maintain a enterprise build system using Maven
Hello Dan, currently I have a team of two (with me three) devops guys. We provide 30 Jenkins masters with 20 build nodes and about 1800 jobs which we provision ourselves as well (Debian is already installed and we are able to login via ssh). We provide the department pom, some Maven plugins, Artifactory, Gitlab, several SonarQube instances, several Testlink instances and some other minor systems as well. The department has about 200 developers, 150 of them use Java/Maven, the others objective-c, c++, Scala, Python, PHP, JavaScript and Ruby. Most teams do have one developer who is more interested in build tooling and we support them with diagnosis of Jenkins or Maven related problems (nowadays we try to answer questions for SBT and npm as well..). We all have a sound knowledge of Java, Python and Shell. Now: while you may write Maven plugins in Jython, Ant or any other JVM language, most of the documentation is directed to writing these in Java. Releasing with Maven is trivial once the project is setup properly, so no Java knowhow is needed here, IMO. The poms are maintained by the teams/developers, we help when they encounter problems (which might to restructuring the project as well). The challenges are mostly with the non-Maven projects, IMO having a soundly setup department pom *and* settings will help, Maven is a very good build tool for company setups. Because of the many systems (we have to arrange for firewall settings from build nodes to qa systems as well) we get about 4 support requests per day, so one of us is ticketeer of the day. Regards Mirko -- Sent from my mobile Am 30.05.2015 07:29 schrieb "Dan Tran" : > Hi > > > I would like to ask if the community can share with me what it takes to > maintain an enterprise build system with continuous integration of 100+ > developer + QA and growing using Maven. The build system contains many > components with their own release cycle and they do integrate together. > > >- is java skill set to develop plugin a must? >- do you have a team or just a few of deep understanding of Maven > developers? >- will a none java RelEng able to perform Maven release? >- does your RelEng maintains the pom or developers? >- what are your challenges? > > Thanks > > -Dan >
