Hi Hervé,
I could imagine that it could still work. As far as I understand all the
gitpubsub and svnpubsub, what generally happens, is that the HTTPD server
serving a projects website is just a directory. And the pubsubs just pull
changes from a source to the static files directory of the projects HTTPD
instance. Infra sets up jobs to pull from a VCS source and push to a directory.
So it should be possible to mix at the target. If however the current setup is
for multiple builds to push to a central website VCS, then I agree that with
this setup it would be difficult, but eventually worth changing (Eventually
even setting up a maven-components svn as target for all of this)
The reason for splitting up, was that in our case we wanted unit-test results,
code-coverage etc. to be part of the site, this requires us to execute the
unit- and integration-tests ... but executing the unit-test on the git-website
node would needlessly block resources on these nodes. That's why with this
setup we only use the ubuntu machines for publishing to nexus, the git-website
nodes for updating the site asf-site branch and use our project VM build node
for all other steps (We do low level network packet manipulation stuff there
... that's why we run our tests on a dedicated machine).
Well I don't have a strong opinion on SVN or GIT as site-repo, I prefer GIT
over SVN in general and I like having all things together. But besides that, I
just thought I'd point out other options.
Chris
Am 11.05.18, 10:32 schrieb "Hervé BOUTEMY" <herve.bout...@free.fr>:
Hi Christofer,
Thanks for the sharing: the idea of building on any node, could be
interesting
in the future if we have performance issues with site commit-enabled nodes.
Normally, these nodes are only used for site, which build is quite fast, so
I'm not convinced there will be any issues in the near future. But at
least,
we already have a solution, to keep in mind.
On GitPubSub vs SvnPubSub:
We absolutely need SvnPubSub for components:
see https://maven.apache.org/developers/website/ and have a look at the
content of components directory
Then I chose to let everything in svnpubsub: I could have done / from
GitPubSub and /components from SvnPubSub, but the mix would IMHO have been
more complex to understand. And I don't see much benefit to GitPubSub over
SvnPubSub: the commit duration is a little bit faster, but not to a degree
that I find useful enough to do the change.
Are there other benefits you see from GitPubSub over SvnPubSub?
Regards,
Hervé
On the idea of switching to GitPubSub
Le vendredi 11 mai 2018, 09:34:03 CEST Christofer Dutz a écrit :
> Hi guys,
>
> just stumbled over this thread ... sorry for not noticing it earlier.
>
> As I did the same for several other projects here (Apache PLC4X probably
> being the most advanced site), I thought I might be able to help.
> When updating to GIT, why not also switch to GitPubSub?
>
> Also I just finished finetuning plc4x's Jenkinsfile ... perhaps it's worth
> having a look at. The important parts are:
- Build the site on any node
> able to do that (the "git-websites" labeled nodes shouldn't be used for
the
> heavy testing ... we only have 1 or 2) - Jenkins-Stash the generated site
> - Unstash the site on a "git-websites" labeled node
> - push the changes to the "asf-site" branch of the current projects repo
> - check what's committed to the "asf-site" branch and reduce the changes
to
> an absolute minimum (I removed most timestamp-stuff as the pub-sub-systems
> do tend to choke on large changes)
> Some remarks from several months of working with this setup:
> - As mentioned above: gitpubsub tends to choke on large changes. Whenever
it
> choked no changes were published at all. Usually I had to beg Infra to
> re-sync the repos, a few weeks ago they added a "resync" to the
> self-service portal at https://selfserve.apache.org/ (The "synchronize git
> repositories" Button). Don't know if I begged so often, that they added
the
> button to get rid of my begging ... but in the end now things work nicely.
> I did reduce the amount of choking by inspecting the commits done to the
> asf-site branch and refactored things to omit unnecessary timestamps and
> invisible things.
> And if you run into problems ... I'm glad to help ...
>
> Chris
>
> PLC4X's Jenkinsfile:
> https://github.com/apache/incubator-plc4x/blob/master/Jenkinsfile
PLC4X's
> Documentation on Website generation:
> http://plc4x.apache.org/developers/website.html
>
>
>
> Am 11.05.18, 01:45 schrieb "Stephen Connolly"
> <stephen.alan.conno...@gmail.com>:
> On Thu 10 May 2018 at 23:51, Hervé BOUTEMY <herve.bout...@free.fr>
> wrote:
>
> > Le jeudi 10 mai 2018, 10:52:01 CEST Stephen Connolly a écrit :
> >
> > >
> >
> >
https://builds.apache.org/view/M-R/view/Maven/job/maven-box/job/maven-> >
site/j
>
> > > ob/master/1/console looks good. @Hervé can you delete
> > > https://builds.apache.org/view/M-R/view/Maven/job/maven-site/ if
you
> > > are
> > > happy with it?
> >
> > great job, even with multi-branch: I'll copy the Jenkinsfile to
Doxia
> > site
> >
> >
> >
> > How can we consult the local site build result? I don't see on
> > master:
> > would
> > something appear in a branch?
>
>
>
> Ok I am having a hard time parsing that request. Could you try to
> rephrase?
>
>
> >
> >
> >
> > >
> > >
> > > On 10 May 2018 at 09:37, Stephen Connolly <
> >
> > stephen.alan.conno...@gmail.com>
> >
> > >
> > >
> > > wrote:
> > >
> > > > On 10 May 2018 at 07:25, Hervé BOUTEMY <herve.bout...@free.fr>
> > > > wrote:
> > > >
> > > >> Le mercredi 9 mai 2018, 19:40:54 CEST Karl Heinz Marbaise a
écrit
> > > >> :
> > > >>
> > > >> > Hi,
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> > This means if you do a commit in the gitbox repository[1] the
> >
> > job[2] on
> >
> > > >> > our build server will start and deploy directly to production
> >
> > site[3]
> >
> > > >> > without the need to public via cms[4].
> > > >>
> > > >>
> > > >>
> > > >> yes, you get it
> > > >>
> > > >>
> > > >>
> > > >> > Great work...
> > > >>
> > > >>
> > > >>
> > > >> thank you
> > > >>
> > > >>
> > > >>
> > > >> > BTW: @Hervé can you explain what is meant to write the
> > > >> > Jenkinsfile
> >
> > to
> >
> > > >> > make a mvn site-deploy ?
> > > >>
> > > >>
> > > >>
> > > >> currently, the Jenkins job that builds and deploy the website
is
> >
> > manually
> >
> > > >> configured
> > > >> If there was a Jenkinsfile in the git repo, the handcrafted
> > > >> Jenkins
> >
> > job
> >
> > > >> could
> > > >> be replaced by the equivalent maven-box generated one
> > > >> https://builds.apache.org/job/maven-box/
> > > >>
> > > >>
> > > >>
> > > >> My personal knowledge of Jenkinsfile and how to test it before
> >
> > committing
> >
> > > >> is
> > > >> null, then we're staying for the moment with the handcrafted
job
> > > >>
> > > >>
> > > >>
> > > >> Initially, I thought that only the Jenkinsfile would bring us
> > > >> the
> > > >> immediate
> > > >> rebuild on Git commit: but it seems the Git pubsub feature is
> >
> > available
> >
> > > >> also
> > > >> on handcrafted job.
> > > >
> > > >
> > > >
> > > > Tsk Tsk did you think that little of me that I would implement
> > > > pub-sub
> >
> > for
> >
> > > > multibranch and leave regular in the cold just to try and drive
> > > > multibranch
> > > > adoption :-(
> >
> > I don't know Jenkins sufficiently to know if Git pubsub is really a
> > separate
> > feature from Apache Hosted Git plugin: glad to see it works for any
> > type
> > of
> > job
> >
> >
> >
> >
> > > >
> > > >
> > > >> Then the only benefit we'll have is to remove the handmade
> > > >> job: nice to have, but nothing critical...
> > > >>
> > > >>
> > > >>
> > > >> Regards,
> > > >>
> > > >>
> > > >>
> > > >> Hervé
> > > >>
> > > >>
> > > >>
> > > >> > Kind regards
> > > >> > Karl Heinz Marbaise
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> > [1]: https://gitbox.apache.org/repos/asf/maven-site.git
> > > >> > [2]:
> > > >> > https://builds.apache.org/view/M-R/view/Maven/job/maven-site/
> > > >> > [3]: https://maven.apache.org/
> > > >> > [4]: https://cms.apache.org/maven/
> > > >> >
> > > >> >
> > > >> >
> > > >> > On 30/04/18 22:49, Hervé BOUTEMY wrote:
> > > >> >
> > > >> > > sites sources migration to Git at GitBox done: now you can
> > > >> > > edit
> >
> > Maven
> >
> > > >>
> > > >>
> > > >> and
> > > >>
> > > >>
> > > >>
> > > >> > > Doxia sites directly from GitHub through the edit link
> > > >>
> > > >>
> > > >>
> > > >> > > The site build and publish Jenkins jobs are still the
> > > >> > > previous
> >
> > ones:
> >
> > > >> if
> > > >>
> > > >>
> > > >>
> > > >> > > someone creates a Jenkinsfile, we'll have immediate build
in
> > > >> > > maven-box
> > > >> > > job.
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > Next step will be the move of components svnpubsub location
> > > >> > > from
> > > >> > > infra
> > > >> > > location [1] to ASF [2] this will require a change in
> > > >> > > distributionManagement/site/url in a new parent POM and
> > > >> > > every
> >
> > child
> >
> > > >>
> > > >>
> > > >> POMs:
> > > >>
> > > >> > > if you intend to do a release, please tell and we'll check
> > > >> > > what
> >
> > to do
> >
> > > >> > >
> > > >> > >
> > > >> > > Regards,
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > Hervé
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > [1]
> > > >> > >
https://svn.apache.org/repos/infra/websites/production/maven
> > > >>
> > > >>
> > > >>
> > > >> /components/
> > > >>
> > > >>
> > > >>
> > > >> > > +
> > > >> > >
https://svn.apache.org/repos/infra/websites/production/maven
> > > >>
> > > >>
> > > >>
> > > >> -doxia/compon
> > > >>
> > > >>
> > > >>
> > > >> > > ents/
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > [2]
> > > >> > > https://svn.apache.org/repos/asf/maven/website/components/
> > > >> > > +
> >
> > https://svn.apache.org/repos/asf/maven/doxia/website/components/
> >
> > > >> > >
> > > >> > >
> > > >> > > Le dimanche 15 avril 2018, 15:25:47 CEST Hervé BOUTEMY a
> > > >> > > écrit :
> > > >> > >
> > > >> > >> Jira issue for source migration to Git opened:
> > > >> > >> https://issues.apache.org/jira/browse/INFRA-16361
> > > >> > >>
> > > >> > >>
> > > >> > >>
> > > >> > >> Regards,
> > > >> > >>
> > > >> > >>
> > > >> > >>
> > > >> > >> Hervé
> > > >> > >>
> > > >> > >>
> > > >> > >>
> > > >> > >> Le dimanche 15 avril 2018, 01:23:06 CEST Hervé BOUTEMY a
> > > >> > >> écrit :
> > > >> > >>
> > > >> > >>> here it is, now Maven site and Doxia sub-site are
published
> > > >> > >>> by
> > > >>
> > > >>
> > > >>
> > > >> Jenkins
> > > >>
> > > >>
> > > >>
> > > >> > >>> CMS is not used any more
> > > >> > >>> We'll just need to move components reference
documentation
> > > >> > >>> out
> >
> > of
> >
> > > >>
> > > >>
> > > >> the
> > > >>
> > > >>
> > > >>
> > > >> > >>> CMS
> > > >> > >>> svn space: I'll do it later with infra
> > > >> > >>>
> > > >> > >>>
> > > >> > >>>
> > > >> > >>> now, I'll work on GitBox migration
> > > >> > >>>
> > > >> > >>>
> > > >> > >>>
> > > >> > >>> If someone can write a Jenkinsfile, this would be great:
> > > >> > >>> the job
> > > >>
> > > >>
> > > >>
> > > >> has to
> > > >>
> > > >>
> > > >>
> > > >> > >>> be
> > > >> > >>> tied to websites1 node, and the command is simply "mvn
> >
> > site-deploy"
> >
> > > >> > >>>
> > > >> > >>>
> > > >> > >>> Regards,
> > > >> > >>>
> > > >> > >>>
> > > >> > >>>
> > > >> > >>> Hervé
> > > >> > >>>
> > > >> > >>>
> > > >> > >>>
> > > >> > >>> Le samedi 14 avril 2018, 22:07:04 CEST Hervé BOUTEMY a
> > > >> > >>> écrit :
> > > >> > >>>
> > > >> > >>>> FYI, switch is happening currently, I'm on HipChat with
> > > >> > >>>> infra
> > > >> > >>>>
> > > >> > >>>>
> > > >> > >>>>
> > > >> > >>>> then CMS does not have any effect now: this will be
> > > >> > >>>> Jenkins job
> > > >> > >>>> (or
> > > >> > >>>> manual
> > > >> > >>>> build)
> > > >> > >>>>
> > > >> > >>>>
> > > >> > >>>>
> > > >> > >>>> please don't try to publish anything for now: I'll tell
> > > >> > >>>> once
> > > >>
> > > >>
> > > >>
> > > >> everything
> > > >>
> > > >>
> > > >>
> > > >> > >>>> is
> > > >> > >>>> stabilized
> > > >> > >>>>
> > > >> > >>>>
> > > >> > >>>>
> > > >> > >>>> Regards,
> > > >> > >>>>
> > > >> > >>>>
> > > >> > >>>>
> > > >> > >>>> Hervé
> > > >> > >>>>
> > > >> > >>>>
> > > >> > >>>>
> > > >> > >>>> Le vendredi 13 avril 2018, 18:15:21 CEST Karl Heinz
> > > >> > >>>> Marbaise a
> > > >>
> > > >>
> > > >>
> > > >> écrit :
> > > >>
> > > >> > >>>>> Hi Hervé,
> > > >> > >>>>>
> > > >> > >>>>>
> > > >> > >>>>>
> > > >> > >>>>> On 12/04/18 08:17, Hervé BOUTEMY wrote:
> > > >> > >>>>>
> > > >> > >>>>>> Hi,
> > > >> > >>>>>>
> > > >> > >>>>>>
> > > >> > >>>>>>
> > > >> > >>>>>> I'm working with infra for a good time on Git
migration
> > > >> > >>>>>> for
> >
> > site
> >
> > > >> > >>>>>> source,
> > > >> > >>>>>> which will de-facto mean that we drop Apache CMS (that
> > > >> > >>>>>> only
> > > >>
> > > >>
> > > >>
> > > >> supports
> > > >>
> > > >>
> > > >>
> > > >> > >>>>>> svn
> > > >> > >>>>>> for source): see INFRA-16088 [1]
> > > >> > >>>>>>
> > > >> > >>>>>>
> > > >> > >>>>>>
> > > >> > >>>>>> Today, we got a first visible result: Jenkins was able
> > > >> > >>>>>> to
> >
> > build
> >
> > > >>
> > > >>
> > > >> and
> > > >>
> > > >>
> > > >>
> > > >> > >>>>>> publish
> > > >> > >>>>>> html output of the Maven main website to svn [2]
> > > >> > >>>>>>
> > > >> > >>>>>>
> > > >> > >>>>>>
> > > >> > >>>>>> We're now ready to switch to the new Jenkins build,
then
> > > >> > >>>>>> drop
> > > >>
> > > >>
> > > >>
> > > >> CMS:
> > > >>
> > > >> > >>>>>> any
> > > >> > >>>>>> objection to do it?
> > > >> > >>>>>>
> > > >> > >>>>>>
> > > >> > >>>>>>
> > > >> > >>>>>> The plan will be then to switch source to GitBox, to
> > > >> > >>>>>> benefit
> > > >> > >>>>>> from
> > > >> > >>>>>> immediate
> > > >> > >>>>>> commit detection (instead of svn hourly polling): at
> > > >> > >>>>>> this
> >
> > step,
> >
> > > >> > >>>>>> you'll
> > > >> > >>>>>> be
> > > >> > >>>>>> able to do easy site source updates through GitHub,
> > > >> > >>>>>> with
> > > >>
> > > >>
> > > >>
> > > >> immediate
> > > >>
> > > >>
> > > >>
> > > >> > >>>>>> publish. I don't see the value of adding a staging
step,
> > > >> > >>>>>> but
> >
> > if
> >
> > > >> > >>>>>> someone
> > > >> > >>>>>> wants staging, it will be a matter of Jenkinsfile
> > > >> > >>>>>> tweaking.
> > > >> > >>>>>>
> > > >> > >>>>>>
> > > >> > >>>>>>
> > > >> > >>>>>> In the temporary time with Jenkins polling on source
> > > >> > >>>>>> from
> >
> > svn ,
> >
> > > >> > >>>>>> either
> > > >> > >>>>>> you'll wait for less than 1 hour or you'll manually
> > > >> > >>>>>> launch
> >
> > the
> >
> > > >> > >>>>>> Jenkins
> > > >> > >>>>>> job
> > > >> > >>>>>>
> > > >> > >>>>>>
> > > >> > >>>>>>
> > > >> > >>>>>> any question?
> > > >> > >>>>>
> > > >> > >>>>>
> > > >> > >>>>>
> > > >> > >>>>> Jenkins will build the Site (as done before
> > > >> > >>>>> ci.apache.org/builders/maven-site-staging) and deploys
it
> > > >> > >>>>> to
> >
> > the
> >
> > > >>
> > > >>
> > > >> site
> > > >>
> > > >>
> > > >>
> > > >> > >>>>> (maven.staging.apache.org?)...but it's gets the source
of
> > > >> > >>>>> the
> > > >>
> > > >>
> > > >>
> > > >> site
> > > >>
> > > >>
> > > >>
> > > >> > >>>>> from
> > > >> > >>>>> svn at the moment ?.
> > > >> > >>>>>
> > > >> > >>>>>
> > > >> > >>>>>
> > > >> > >>>>> If I wan't to publish it to live I have to go via
> > > >> > >>>>> cms.apache.org/maven/publish as before (for now)...
> > > >> > >>>>>
> > > >> > >>>>>
> > > >> > >>>>>
> > > >> > >>>>>
> > > >> > >>>>>
> > > >> > >>>>> Kind regards
> > > >> > >>>>> Karl Heinz Marbaise
> > > >> > >>>>
> > > >> > >>>>
> > > >> > >>>>
> > > >> > >>>>
----------------------------------------------------------
> > > >> > >>>> --
> > > >>
> > > >>
> > > >>
> > > >> ---------
> > > >>
> > > >>
> > > >>
> > > >> > >>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > > >> > >>>> For additional commands, e-mail:
> > > >> > >>>> dev-h...@maven.apache.org
> > > >> > >>>
> > > >> > >>>
> > > >> > >>>
> > > >> > >>>
-----------------------------------------------------------
> > > >> > >>> -
> > > >>
> > > >>
> > > >>
> > > >> ---------
> > > >>
> > > >>
> > > >>
> > > >> > >>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > > >> > >>> For additional commands, e-mail:
dev-h...@maven.apache.org
> > > >> > >>
> > > >> > >>
> > > >> > >>
> > > >> > >>
------------------------------------------------------------
> > > >> > >>
> > > >>
> > > >>
> > > >>
> > > >> ---------
> > > >>
> > > >>
> > > >>
> > > >> > >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > > >> > >> For additional commands, e-mail: dev-h...@maven.apache.org
> > > >> > >
> > > >> > >
> > > >> > >
> >
> >
---------------------------------------------------------------------
> >
> > > >> > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > > >> > > For additional commands, e-mail: dev-h...@maven.apache.org
> > > >> >
> > > >> >
> > > >> >
> > > >> > Mit freundlichem Gruß
> > > >> > Karl-Heinz Marbaise
> > > >>
> > > >>
> > > >>
> > > >>
-----------------------------------------------------------------
> > > >> ----
> > > >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > > >> For additional commands, e-mail: dev-h...@maven.apache.org
> >
> >
> >
> >
> >
> >
---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > For additional commands, e-mail: dev-h...@maven.apache.org
> >
> >
> >
> > --
>
> Sent from my phone
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org
