Perhaps a better mechanism for things like javadocs, would be to support some javadoc viewer which serves javadocs directly from javadoc jars? That way, you just upload the jars you build with maven or ant, which are already compressed. That should work well enough with svn or the CMS external build feature. (And would also have the added benefit of only needing to update the viewer when there's a javadoc security vulnerability, rather than require every project to patch their javadocs, as I seem to remember was recently the case.)
-- Christopher L Tubbs II http://gravatar.com/ctubbsii On Thu, Jan 8, 2015 at 4:28 AM, Robert Metzger <rmetz...@apache.org> wrote: > I think there is a performance difference between git and svn because with > git, you are syncing repositories, not files. Git is usually compressing > the repository before sending it over the network. > I did a little test with our website directory and pushed it to github: > > git add : 7 seconds (17k files) > git repo size: 58 MB (du -sh in ".git" dir) > git push to github : 35 seconds (using the same internet connection as with > the svn) > > I think we can achieve the same performance at apache using git for the > websites. > > I agree with you that I should probably take a closer look into the CMS > external build feature. Using svn or git for 17k files is certainly wrong. > In particular because we add generated files (javadocs, documentation) to > the VCS which doesn't have any value. > > I'll get in touch with Infra on a separate thread to resolve our website > issues. > > > > On Thu, Jan 8, 2015 at 10:01 AM, Daniel Gruno <humbed...@apache.org> > wrote: > > > > > On 2015-01-08 09:55, Robert Metzger wrote: > > > >> Hi, > >> > >> If there is sufficient demand, however, that could change - the code > that > >>> > >> would make this possible does exist. > >> > >> I would like to express demand from the Flink project. svn is a pain to > >> use > >> (since we host javadocs and our documentation on our website, the upload > >> usually runs for 6+ hours. Probably because its so many files). > >> > > How is that in any way related to svn? svn or git, if you have that many > > files, it's going to take ages to upload regardless of which repository > > system you use. > > > > If it makes it easier, you might peruse http://www.apache.org/dev/ > > cmsref.html#external-build and see if you can't use an external build job > > for you site - that way, you just commit the changes to your template/raw > > docs, and the CMS system builds the javadoc for you. > > > > > > > >> Almost all incoming incubator projects are using git nowadays. Many of > >> them > >> use GitHub and the "gh-pages" hosting feature. I guess a lot of them > would > >> appreciate a similar feature from the ASF infra. > >> > >> > >> > >> On Wed, Jan 7, 2015 at 9:42 PM, Daniel Gruno <humbed...@apache.org> > >> wrote: > >> > >> Hi Jay, > >>> It sounds like you have the CMS setup that is quite common in the ASF. > >>> If you would rather have straight-up svnpubsub, you can ask for it. > That > >>> way, whatever static content you push to svn will instantly be shown on > >>> your web site without having to publish through the UI first. > >>> > >>> The main reason we do not support git in this workflow is that git does > >>> not enable single-file checkouts, and that we haven't properly tested > >>> gitwcsub (a git version of svnwcsub which is the frontend for > svnpubsub) > >>> for web sites yet. If there is sufficient demand, however, that could > >>> change - the code that would make this possible does exist. > >>> > >>> With regards, > >>> Daniel. > >>> > >>> > >>> On 2015-01-07 21:29, jay vyas wrote: > >>> > >>> thanks daniel... > >>>> > >>>> here at bigtop we are 100% git based. so having an svn account , > >>>> just > >>>> to > >>>> push changes to a site, seems to slow us down alot. > >>>> > >>>> is SVN required ? or is there another way? > >>>> > >>>> right now we have a system that uses maven, followed by svn and then > we > >>>> have to approve the changes in the web ui. > >>>> > >>>> would rather just push static html pages to our git repo , the way we > >>>> push > >>>> everything else. > >>>> > >>>> are all apache projects using SVN or do some folks have an easier > >>>> workflow ? > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> On Wed, Jan 7, 2015 at 3:12 PM, Daniel Gruno <humbed...@apache.org> > >>>> wrote: > >>>> > >>>> Essentially, github uses the same method as we do with svnpubsub. > >>>> > >>>>> Files are pushed to a repository and then from there pushed directly > to > >>>>> the web site. > >>>>> > >>>>> Is there anything specific about the github model that you think > differ > >>>>> from how we do things? > >>>>> Apart from it being git and not subversion, obviously. > >>>>> > >>>>> With regards, > >>>>> Daniel. > >>>>> > >>>>> On 2015-01-07 21:06, jay vyas wrote: > >>>>> > >>>>> Hi apache ! > >>>>> > >>>>>> Whats the simplest way to maintain the xyz.apache.org site? Right > >>>>>> now > >>>>>> we > >>>>>> push to SVN, but would be great to use something like the github.io > >>>>>> model, > >>>>>> where the static pages are just hosted directly. > >>>>>> > >>>>>> > >>>>>> On Wed, Jan 7, 2015 at 2:56 PM, <dev-h...@community.apache.org> > >>>>>> wrote: > >>>>>> > >>>>>> Hi! This is the ezmlm program. I'm managing the > >>>>>> > >>>>>> dev@community.apache.org mailing list. > >>>>>>> > >>>>>>> I'm working for my owner, who can be reached > >>>>>>> at dev-ow...@community.apache.org. > >>>>>>> > >>>>>>> Acknowledgment: I have added the address > >>>>>>> > >>>>>>> jayunit100.apa...@gmail.com > >>>>>>> > >>>>>>> to the dev mailing list. > >>>>>>> > >>>>>>> Welcome to dev@community.apache.org! > >>>>>>> > >>>>>>> Please save this message so that you know the address you are > >>>>>>> subscribed under, in case you later want to unsubscribe or change > >>>>>>> your > >>>>>>> subscription address. > >>>>>>> > >>>>>>> > >>>>>>> --- Administrative commands for the dev list --- > >>>>>>> > >>>>>>> I can handle administrative requests automatically. Please > >>>>>>> do not send them to the list address! Instead, send > >>>>>>> your message to the correct command address: > >>>>>>> > >>>>>>> To subscribe to the list, send a message to: > >>>>>>> <dev-subscr...@community.apache.org> > >>>>>>> > >>>>>>> To remove your address from the list, send a message to: > >>>>>>> <dev-unsubscr...@community.apache.org> > >>>>>>> > >>>>>>> Send mail to the following for info and FAQ for this list: > >>>>>>> <dev-i...@community.apache.org> > >>>>>>> <dev-...@community.apache.org> > >>>>>>> > >>>>>>> Similar addresses exist for the digest list: > >>>>>>> <dev-digest-subscr...@community.apache.org> > >>>>>>> <dev-digest-unsubscr...@community.apache.org> > >>>>>>> > >>>>>>> To get messages 123 through 145 (a maximum of 100 per request), > mail: > >>>>>>> <dev-get.123_...@community.apache.org> > >>>>>>> > >>>>>>> To get an index with subject and author for messages 123-456 , > mail: > >>>>>>> <dev-index.123_...@community.apache.org> > >>>>>>> > >>>>>>> They are always returned as sets of 100, max 2000 per request, > >>>>>>> so you'll actually get 100-499. > >>>>>>> > >>>>>>> To receive all messages with the same subject as message 12345, > >>>>>>> send a short message to: > >>>>>>> <dev-thread.12...@community.apache.org> > >>>>>>> > >>>>>>> The messages should contain one line or word of text to avoid being > >>>>>>> treated as sp@m, but I will ignore their content. > >>>>>>> Only the ADDRESS you send to is important. > >>>>>>> > >>>>>>> You can start a subscription for an alternate address, > >>>>>>> for example "john@host.domain", just add a hyphen and your > >>>>>>> address (with '=' instead of '@') after the command word: > >>>>>>> <dev-subscribe-john=host.dom...@community.apache.org> > >>>>>>> > >>>>>>> To stop subscription for this address, mail: > >>>>>>> <dev-unsubscribe-john=host.dom...@community.apache.org> > >>>>>>> > >>>>>>> In both cases, I'll send a confirmation message to that address. > When > >>>>>>> you receive it, simply reply to it to complete your subscription. > >>>>>>> > >>>>>>> If despite following these instructions, you do not get the > >>>>>>> desired results, please contact my owner at > >>>>>>> dev-ow...@community.apache.org. Please be patient, my owner is a > >>>>>>> lot slower than I am ;-) > >>>>>>> > >>>>>>> --- Enclosed is a copy of the request I received. > >>>>>>> > >>>>>>> Return-Path: <jayunit100.apa...@gmail.com> > >>>>>>> Received: (qmail 14748 invoked by uid 99); 7 Jan 2015 19:56:45 > -0000 > >>>>>>> Received: from athena.apache.org (HELO athena.apache.org) > >>>>>>> (140.211.11.136) > >>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 07 Jan 2015 > >>>>>>> 19:56:45 > >>>>>>> +0000 > >>>>>>> X-ASF-Spam-Status: No, hits=1.5 required=5.0 > >>>>>>> tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS > >>>>>>> X-Spam-Check-By: apache.org > >>>>>>> Received-SPF: pass (athena.apache.org: domain of > >>>>>>> jayunit100.apa...@gmail.com designates 74.125.82.171 as permitted > >>>>>>> sender) > >>>>>>> Received: from [74.125.82.171] (HELO mail-we0-f171.google.com) > >>>>>>> (74.125.82.171) > >>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 07 Jan 2015 > >>>>>>> 19:56:39 > >>>>>>> +0000 > >>>>>>> Received: by mail-we0-f171.google.com with SMTP id > >>>>>>> u56so1794125wes.30 > >>>>>>> for <dev-sc.1420660420.dnaojabgibffbmnboifj- > >>>>>>> jayunit100.apache= > >>>>>>> gmail....@community.apache.org>; Wed, 07 Jan 2015 11:56:18 -0800 > >>>>>>> (PST) > >>>>>>> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; > >>>>>>> d=gmail.com; s=20120113; > >>>>>>> > >>>>>>> h=mime-version:in-reply-to:references:date:message-id: > >>>>>>> subject:from:to > >>>>>>> :content-type; > >>>>>>> bh=4OB4KLyw2fmFpQl3M1XmOX18z5O8zy+yvZghLrCCQH0=; > >>>>>>> > >>>>>>> > b=XSc/5mc3l7ZxTLvKQwhwblcJ9JBklYotsWNQjg2deDzdYhR4p1icsaTjQrklu+hpk2 > >>>>>>> > >>>>>>> W0Y80Uk/PcDTvjO5JSDU0IxtUsSrjV5QK6KU2H > >>>>>>> 9w7H39EytqRv/lFn7QL1Yw4UYib6zo > >>>>>>> > >>>>>>> 0hNaCgxt84skf1IQm7yRWP5pxIaQuDyI1r2W8WL5Bt6d34ps80thba1ILoPz > >>>>>>> /b1htNVT > >>>>>>> > >>>>>>> huI4b35Fqf2Q8mf2rKWYL1ppgyelNXheWOwWfqerEILddc3En+ > >>>>>>> r2PKqzTY6Jh5R8aoVn > >>>>>>> > >>>>>>> BwI0pN6WhAjm59qb7WZ0vb4Aorj3G2pMIOSn4R75jTvmFRY3DlSd8q582L+ > >>>>>>> JFWGC5tWd > >>>>>>> KASw== > >>>>>>> MIME-Version: 1.0 > >>>>>>> X-Received: by 10.194.88.131 with SMTP id bg3mr9420545wjb.99. > >>>>>>> 1420660578606; > >>>>>>> Wed, 07 Jan 2015 11:56:18 -0800 (PST) > >>>>>>> Received: by 10.27.144.214 with HTTP; Wed, 7 Jan 2015 11:56:18 > -0800 > >>>>>>> (PST) > >>>>>>> In-Reply-To: <1420660420.2944.ez...@community.apache.org> > >>>>>>> References: <1420660420.2944.ez...@community.apache.org> > >>>>>>> Date: Wed, 7 Jan 2015 14:56:18 -0500 > >>>>>>> Message-ID: <CACVCA=fV3HDyzkHn= > >>>>>>> tehwkosmky2vxtyicacstnhq4n+eer...@mail.gmail.com> > >>>>>>> Subject: Re: confirm subscribe to dev@community.apache.org > >>>>>>> From: jay vyas <jayunit100.apa...@gmail.com> > >>>>>>> To: dev-sc.1420660420.dnaojabgibffbmnboifj-jayunit100.apache= > >>>>>>> gmail....@community.apache.org > >>>>>>> Content-Type: multipart/alternative; boundary= > >>>>>>> 089e010d849e226317050c15515d > >>>>>>> X-Virus-Checked: Checked by ClamAV on apache.org > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > > >
