Re: [OTish] Version Control?
perl Makefile.PL /dev/null works for us. We encapsulate it in a macro (see below). Now why didn't I think of that? :). This works nicely. We still ahve to patch some of the individual Makefile.PL's, but that is acceptable (some of them have exit; at the end of them for example which causes the Makefile generation to stop :)). We use a vendor branch to import the new versions of modules as we update, and CVS handles this fairly well. Mike
Re: [OTish] Version Control? (Project Idea)
I find not only amazing that so many of us are doing the exact same thing, but also that the project I'm working on now is about these same ideas we have. So, why not make it GPL? I'm developing an intranet to organize a web development team and there is a lot of CVS involved in it. Right now all everyone, from designers to programmers, is using CVS and the next step being developed is making the correct versions of the site to be published in the right servers, for the client to approve, for developers to test and benchmark, etc. Any thoughts about this? []s Luis [EMAIL PROTECTED] wrote: Who needs network guys, reverse pop the ssh tunnel ;) I find it amazing that so many of us are doing the exact same thing in terms of managing our large site installs yet its nowhere to be found in any FAQ, knowledge base or general forum. I know on our side we developed these solutions over months and months on our own, dealing with problems at every step. Had there been a nice little FAQ on it, well, it would have saved us a ton of time :) I sense some writing.. John- On Wed, 30 Oct 2002 15:40:45 -0700 Ken Miller [EMAIL PROTECTED] wrote: doubt if I could have enticed the network folks to open the firewall to let CVS traffic through in any case Cheers! -klm.
Re: [OTish] Version Control?
Michael Schout writes:
example, one time we upgraded Apache::Filter between releases.
Unfortunately, the old version was not compatible with the new version,
so a single machine could run either the current release branch, or the
development branch, but not both simultaneously (because Apache::Filter
was incomptaible and was installed under /usr/lib/perl5).
We are transitioning (slowly) between perl 5.005 and 5.6.1. Our trick
is to have separate 5.005 and 5.6.1 build/test (and sometimes dev)
machines. I'm not sure this solves your problem.
1) some Makefile.PL's refuse to generate a Makefile if PREREQ_PM's are
not satisfied (if we haven't built them yet)
If we have to bootstrap, we do a regular CPAN install on the build
machine and then install over it with the RPM build. Also, we use
Red Hat which has many CPAN modules already installed (see uninstall
instructions below), so bootstrapping is rarely an issue.
2) some Makefile.PL's are INTERACTIVE, and you cant turn it off (e.g.:
Apache::Filter requires you to hit Return a number of times at a MINIMUM.
perl Makefile.PL /dev/null works for us. We encapsulate it in a
macro (see below).
So we resorted to a set of overly-complicated GNUmakefiles that would
generate Makefile's from Makefile.PL's, and these would set PERL5LIB to
find the dependencies (e.g.: DBD-Pg would put ../DBI/blib into
PERL5LIB).
Here's our spec file:
Name: perl.modules
Summary: Perl Modules not in stock RH7
Group: Perl/Modules
Provides: perl.modules perl-libwww-perl
Requires: perl
Version: 5.6
BuildRoot: install
%define modules BSD-Resource IO-stringy Digest-MD5 Digest-HMAC Digest-SHA1 MD5
Crypt-IDEA Crypt-DES Crypt-Blowfish Crypt-CBC DBI DBD-Pg DBD-Oracle DBD-Sybase
DBD-mysql TimeDate MailTools MIME-tools Devel-Symdump Image-Size Compress-Zlib
Archive-Zip File-MMagic TermReadKey Crypt-SSLeay libwww-perl Parse-RecDescent
Mail-Field-Received POP3Client Mail-2IMAPClient Test-Simple Time-HiRes Digest-Nilsimsa
razor-agents Mail-Audit Mail-SpamAssassin XML-XPath httpmail
%description
Perl Modules not in stock RH7 or newer CPAN versions.
To remove RedHat standard installs, do:
rpm -e --nodeps $(rpm -qa | egrep 'perl-(DBD|DBI|libwww)')
If you want to use Sybase (SQL Server), you need:
b-release install freetds-0.53-1.i386.rpm
And to compile this, you need:
b-release install freetds-devel-0.53-1.i386.rpm
%prep
%{cvs} external/perl-modules-5.6
%build
cd external/perl-modules-5.6
unset PERL_MM_OPT
for f in %{modules}; do
(
if test $f = 'Crypt-IDEA'; then
export PERL_MM_OPT='POLLUTE=1'
elif test $f = 'DBD-Sybase'; then
export SYBASE=/usr
elif test $f = 'DBD-Pg'; then
export POSTGRES_LIB=/usr/lib POSTGRES_INCLUDE=/usr/include/pgsql
fi
cd $f
%{perl_make}
)
done
%install
cd external/perl-modules-5.6
for f in %{modules}; do
(set -e; cd $f; %{perl_make_install})
done
cd $RPM_BUILD_ROOT
%{allfiles} ../files
%files -f files
%clean
[ $RPM_BUILD_ROOT != / ] rm -rf $RPM_BUILD_ROOT
%pre
# Perl must be setup properly
perl -e 'require syscall.ph' 2 /dev/null || (
umask 022
cd /usr/include
h2ph -r -l . /dev/null
)
The macros perl_make_install and perl_make are defined below. We run
a program (Bivio::Util::Release mentioned in another post) which
generates the actual spec file and calls rpm. (%{allfiles} and %{cvs}
are trivial and defined there, too.) This program also builds a
separate directory and defines topdir, etc. correctly so you can build
everything as any user.
sub _perl_make {
return
'%define perl_make umask 022 perl Makefile.PL /dev/null '
. make POD2MAN=true\n
. '%define perl_make_install umask 022; make '
. join(' ', map {
uc($_) . '=$RPM_BUILD_ROOT' . $Config::Config{$_};
} grep($_ =~ /^install(?!style)/
$Config::Config{$_} $Config::Config{$_} =~ m!^/!,
sort(keys(%Config::Config
. ' POD2MAN=true pure_install '
. ' find $RPM_BUILD_ROOT%{_libdir}/perl? -name *.bs '
. -o -name .packlist -o -name perllocal.pod | xargs rm -f\n;
}
[Uh oh, there's that nasty map function. ;-]
Note that we don't install man pages. This slows down the
build/install, and perldoc is just as easy to type as man. :-)
We use this same function for all our perl apps. Indeed, to build a
new app, our specfile looks like:
Copyright: Logistics R Us, Inc.
Requires: Bivio-bOP apache ImageMagick-perl
%define perl_root LogisticalNightmare
%define perl_exe_prefix ln
_b_release_include('perl-app.include');
perl-app.include knows how to read our tree structure, which is
consistent across projects, and it installs all perl, programs,
images, views, etc.
How does everyone else cope with this (managing trees of CPAN modules /
CPAN module tree build environments)? Maybe we are sort of unique in
that we use so many 3rd
Re: [OTish] Version Control?
From: Steven Lembark [EMAIL PROTECTED] One quick way to deal with testing things is to tag the releases with something meaningful and pass the tag to Q/A. They do an update -d -P -r yourtag and test it. If things work then the update gets run again in production and life goes on. Since nothing but tagged releases go into production they can also back off easily if a problem is found; ditto Q/A. Yes, you always need to tag everything at least at the 'known-working' points. In addition to assigning unique tags that you have to remember, you can 'float' existing tags to specific points which makes it easier to script actions and keep track of the 'release-1' and 'release-2' versions. For example, one script can pull several developers' work together from the head of the repository into a test area. Then when it all works, another script can float the 'release-n' tags up one position, tag the testbed instance as the current release, then update using the release tag to the production machine or a staging area where you rsync to production. This allows new work to continue at the head of the repository without getting pulled into a production update before testing and gives you a known tag to go back to the previous tested/working system. I've never actually backed out a whole release that way but its one of those things that gives a warm-fuzzy feeling knowing that you can and the ability to see all the changes between versions from cvsweb has made it much easier to find and fix problems that pop up after an update into production. --- Les Mikesell [EMAIL PROTECTED]
Re: [OTish] Version Control?
Okay, I'll chime in on this one. I work on a medium sized mod_perl project (approximately 50,000 lines of perl code). This project is managed similarly to to the setups that have been described so far. We store all of our CPAN module sources in CVS, and currently we distribute the modules to the servers via RPM. Its critical that we maintian sources to the CPAN modules in CVS because some modules have to be patched to make the modules work for this particular project (and some modules just plain have bugs :)). The modules get installed in the main perl directory (e.g.: /usr/lib/perl5). We use cvs branches to manage releases and development versions of the web site and perl code. When we get to a realease point we make a branch (e.g.: RELEASE-1_0), and html developers continue working on that branch for changes to the currently released site. The web servers can be updated with simply cvs update this way. On the main trunk perl/web development continues until the next release happens. However, due to the fact that the CPAN modules are distributed via RPM, there are various incompatibility problems that come up. For example, one time we upgraded Apache::Filter between releases. Unfortunately, the old version was not compatible with the new version, so a single machine could run either the current release branch, or the development branch, but not both simultaneously (because Apache::Filter was incomptaible and was installed under /usr/lib/perl5). So currently we are looking into also installing CPAN modules under the project directory and managing the binary CPAN modules CVS, which would solve this problem. Developers could check out a sandbox of the release branch and they would get the modules that belong to that release (binary compatibility is okay because our development machines match the production machines). The management nightmare to all of this is that automating the build process for lots of CPAN modules (rougly 110 of them) is painful. Our CPAN source tree looks something like this: project-support/ src/ Makefile Apache-Filter/ CGI/ Digest-MD5/ ... First we tried making a Makefile.PL in src that used the DIR attribute of WriteMakefile to specify the subdirs to build all of the modules. THis didnt work so well. The problems with this are: 1) some Makefile.PL's refuse to generate a Makefile if PREREQ_PM's are not satisfied (if we haven't built them yet) 2) some Makefile.PL's are INTERACTIVE, and you cant turn it off (e.g.: Apache::Filter requires you to hit Return a number of times at a MINIMUM. So we resorted to a set of overly-complicated GNUmakefiles that would generate Makefile's from Makefile.PL's, and these would set PERL5LIB to find the dependencies (e.g.: DBD-Pg would put ../DBI/blib into PERL5LIB). I'm not convinced this is the best way to go about it, but it sort of works. How does everyone else cope with this (managing trees of CPAN modules / CPAN module tree build environments)? Maybe we are sort of unique in that we use so many 3rd part CPAN modules, but because we use so many of them its pretty critical that we manage them in CVS. Mike
Re: [OTish] Version Control?
Rob Nagler wrote: Another approach which allows easy sharing between projects is: ~/src/perl/ + Project1/ + Project2/ + Project3/ where Project[123] are root package names. Set PERLLIB=~/src/perl and you can get access to any *.pm in the system, each has a globally unique name. This makes it easy to implement cross-project refactorings. How do you cope with the problem that perl has of running different versions of modules? We have a similiar situation, in that we're running several projects with different sets of perl libraries. We have common code between them. The trouble starts when we're running several sites on the same virtual server. At that point, there's only one copy of the canonical code running, rather than each vhost getting its own copy. I realise that this is usually what's wanted, but it can make life very difficult when you're trying to upgrade one vhost, which needs a new version of one of the shared modules because you might break one of the others by accident. I haven't found a good solution to this yet... -Dom -- | Semantico: creators of major online resources | | URL: http://www.semantico.com/ | | Tel: +44 (1273) 72 | | Address: 33 Bond St., Brighton, Sussex, BN1 1RD, UK. |
RE: [OTish] Version Control?
Hi Dom -- How do you cope with the problem that perl has of running different versions of modules? We have a similiar situation, in that we're running several projects with different sets of perl libraries. We have common code between them. The trouble starts when we're running several sites on the same virtual server. At that point, there's only one copy of the canonical code running, rather than each vhost getting its own copy. The short answer: PERL5LIB Read my earlier messages in this thread. In one I describe the layout of my site architecture which fully accommodates this reality. In a nutshell, each site has its own modules/ directory in which ALL the modules for a project, including CPAN libraries, are stored. The environment variable PERL5LIB can be set separately for each virtual host (Apache directive SetEnv and PerlSetEnv for mod_perl users), thereby allowing each site to have its OWN versions of libraries. If you isolate your web site from the host operating system you will gain significant freedom and ease of management. Warmest regards, -Jesse- -- Jesse Erlbaum The Erlbaum Group [EMAIL PROTECTED] Phone: 212-684-6161 Fax: 212-684-6226
RE: [OTish] Version Control?
At 8:27 AM -0500 11/1/02, Jesse Erlbaum wrote: Hi Dom -- How do you cope with the problem that perl has of running different versions of modules? We have a similiar situation, in that we're running several projects with different sets of perl libraries. We have common code between them. The trouble starts when we're running several sites on the same virtual server. At that point, there's only one copy of the canonical code running, rather than each vhost getting its own copy. The short answer: PERL5LIB Read my earlier messages in this thread. In one I describe the layout of my site architecture which fully accommodates this reality. In a nutshell, each site has its own modules/ directory in which ALL the modules for a project, including CPAN libraries, are stored. The environment variable PERL5LIB can be set separately for each virtual host (Apache directive SetEnv and PerlSetEnv for mod_perl users), thereby allowing each site to have its OWN versions of libraries. Um ... somebody's missing something here ... Jesse are you saying that you have a single apache server running multiple vhosts with different versions of the same module loaded for different vhosts? Is that actually possible? I thought that whenever you first require ModuleX it will use the value of PERL5LIB in that context to load it, but the next time a require happens, even if PERL5LIB is different, it will see that the ModuleX is loaded and won't even try to load the one from a different directory. Right? What I do (with a very limited number of vhosts) is have each vhost proxy to a different backend server which does have its own version of (nearly) everything. -- Ray Zimmerman / e-mail: [EMAIL PROTECTED] / 428-B Phillips Hall Sr Research / phone: (607) 255-9645 / Cornell University Associate / FAX: (815) 377-3932 / Ithaca, NY 14853
RE: [OTish] Version Control?
Hi Ray -- Jesse are you saying that you have a single apache server running multiple vhosts with different versions of the same module loaded for different vhosts? Is that actually possible? I thought that whenever you first require ModuleX it will use the value of PERL5LIB in that context to load it, but the next time a require happens, even if PERL5LIB is different, it will see that the ModuleX is loaded and won't even try to load the one from a different directory. Right? What I do (with a very limited number of vhosts) is have each vhost proxy to a different backend server which does have its own version of (nearly) everything. That's very nearly exactly what I do. I prefer to have each site have its own Apache instance because there is some contention in the way mod_perl caches code. Your vhost/reverse-proxy solution is a good one, particularly for places where there is a shortage of IP addresses. TTYL, -Jesse-
Re: [OTish] Version Control?
Dominic Mitchell writes: How do you cope with the problem that perl has of running different versions of modules? Actually, we have two problems. One problem is developing with multiple versions and the other is what you mention, running production systems. Sometimes I might be in the middle of some big refactoring, and a customer calls with a problem. I then do: cd mkdir src_bla cd src_bla cvs checkout perl/Bla perl/Bivio where Bivio is our shared code. Then I set PERLLIB to ~/src_bla. We've got a bash command that allows me to switch the configuration and PERLLIB as well. It's very easy to do. Oh, and we *never* (almost :-) put code in programs. The programs invoke a *.pm file's main so we can say bla-some-command and always get the right version. We solve the second problem by buying cheap machines which run Linux just fine. (I just bought 4 x Dell 2300, 2 x Dell 1300, and 2 x white box for $1800. $-) It just isn't worth my time trying to make two sites work on the same machine, although we do this in a couple of cases (e.g. www.bivio.biz and petshop.bivio.biz). When two or more sites do share the same machine, we always run the same version of the infrastructure. This avoids many problems, e.g. running into defects twice and managing multiple versions. We don't tag our CVS. We can backout changes with RPM. We do several releases a week on active applications, and one release a week on applications in maintenance mode. One final reason to avoid multiple versions is with schema changes. The more different database versions you have, the more confusing. On bivio.com we upgraded the schema about 250 times in about two years. It would have been impossible to keep the development, test, and production if these three diverged too much. Rob
RE: [OTish] Version Control?
Hey Rob -- Oh, and we *never* (almost :-) put code in programs. The programs invoke a *.pm file's main so we can say bla-some-command and always get the right version. This is how CGI::Application works, too. You might want to check it out! For the benefit of those who are not yet putting all their code in modules, it really does give you a profound advantage in ease of development and management. TTYL, -Jesse- -- Jesse Erlbaum The Erlbaum Group [EMAIL PROTECTED] Phone: 212-684-6161 Fax: 212-684-6226
Re: [OTish] Version Control?
Ged Haywood wrote: Hi all, On Thu, 31 Oct 2002, Iain 'Spoon' Truskett wrote: experimenting with perforce [which, so far, appears nicer than CVS] ) Yikes! Josh, you got anything to say about that? Well I have used perforce much more than CVS now, so that is where I'm most comfortable. My basic feeling is that perforce is harder to set up, but more idiot proof while using it, especially for more advance operations like branching integration. CVS however is a breeze to set up, but I feel like its branching tagging is more of a hack ( note I am just starting to use it recently. ) I really like how I do not have to explicitly cvs edit files like I do in perforce before I start working on them. Oh, CVS is free which is always a good thing. :) As for scaling, perforce probably has a better chance to scale with its dbm based storage locking, however at this one company that uses it they get db corruption when developers hammer its with huge media checkins which I suspect CVS never would get. Regards, Josh Josh Chamas, Founder phone:925-552-0128 Chamas Enterprises Inc.http://www.chamas.com NodeWorks Link Checkinghttp://www.nodeworks.com
Re: [OTish] Version Control?
Oh, CVS is free which is always a good thing. :) subversion (http://subversion.tigris.org/) has been gaining lots of momentum lately, and I just saw this link today: http://svnbook.red-bean.com/ some of you might know the folks at red-bean from their other projects, like http://cvsbook.red-bean.com/ and cvs2cl.pl (http://www.red-bean.com/~kfogel/cvs2cl.shtml) HTH --Geoff
Re: [OTish] Version Control?
Hi all, On Thu, 31 Oct 2002, Iain 'Spoon' Truskett wrote: experimenting with perforce [which, so far, appears nicer than CVS] ) Yikes! Josh, you got anything to say about that? 73, Ged.
Re: [OTish] Version Control?
Another approach which allows easy sharing between projects is: ~/src/perl/ + Project1/ + Project2/ + Project3/ where Project[123] are root package names. Set PERLLIB=~/src/perl and you can get access to any *.pm in the system, each has a globally unique name. This makes it easy to implement cross-project refactorings. We use CVS for source management, and we use RPMs for deployment. RPM allows you to ask what release of ProjectN is on the system. RPM also allows you to manage permissions and ownership easily. Our RPM spec builder/installer can be found at: http://petshop.bivio.biz/src?s=Bivio::Util::Release Rob
Re: [OTish] Version Control?
On Wed, 30 Oct 2002 [EMAIL PROTECTED] wrote: I don't believe in transfering _any_ binaries around, every binary recompiles on its new platform at install time. All modules, apache, external software etc. This eliminates those pesky little problems that pop up when you start pushing binaries. Uhmn, if your systems are well managed you don't get any of those pesky little problems. Do you recompile the base system on each server too? In my experience, then as soon as you have more than a few handfuls of servers you get more trouble trying to coordinate recompiling binaries than doing it once and distributing them (in tarballs, rpms, or with your revisioning system). - ask -- ask bjoern hansen, http://www.askbjoernhansen.com/ !try; do();
RE: [OTish] Version Control?
On Wed, 30 Oct 2002, Bill Moseley wrote: At 04:47 PM 10/30/02 -0500, Jesse Erlbaum wrote: Web development projects can map very nicely into CVS. We have a very mature layout for all web projects. In a nutshell, it boils down to this: project/ + apache/ + bin/ That requires binary compatibility, though. I have a similar setup, but the perl and Apache are built separately on the target machine since my machines are linux and the production machine is Solaris. No, you don't need binary compatibility. You just need to design your system right and use the right tools. :-) With Perforce - http://www.perforce.com/ - you can map different directories in the repository to directories on the client with client views. I have used that (Graham Barr came up with it there) so we could keep the binaries version controlled too. It's great. There we had all of our application software (including perl, apache, etc) distributed with perforce. With CVS or Subversion you could do something similar by setting up branches right or having a few symlinks. We had separate installations around the world serving different clients. Before we migrated from CVS to perforce we had a cvs tag for each installation, so rolling a new release was just o) make a new release number (r345), tag the files with that o) move the installation tag (LosAngeles_Production or QA or whatever) to the new release tag (r345). o) tell the servers to upgrade and restart the application. Perforce provides some features that makes it a bit neater, so the implementation was different but the concept the same. It worked (works actually) great. At another place we have a script to tag the files with a build number and then roll some tar balls. The tar balls then go to QA server and then to the production servers. There all the code is living on shared NFS servers, so rolling it out is just putting the code there and restarting the servers. Perl, Apache, etc are installed with rpms there. For a big project I like installing everything in /home/[project]/ - including perl, apache, mod_perl, ... - and having the whole directory version controlled. On the perl.org we have a bunch of smaller projects. There we use a shared /home/perl/ installation that gets distributed to the servers we use for production and for development. It has perl 5.8, various versions of Apache and other goodies. Each project (in /home/[project]/) then just reference that installation. Making it easy to setup a development environment on your own box is really really nice. On bigger projects I often even have several development environments installed at the same time, so I easily can work on different sub project or test different branches at the same time. [...] Is anyone using cvs to manage updates made with web-based forms? The Twiki system and the Faq-O-Matic are using RCS on the backend to version control entries. - ask -- ask bjoern hansen, http://www.askbjoernhansen.com/ !try; do();
Re: [OTish] Version Control?
I dont really understand where you are coming from, its easy enough for the script to recompile its external dependancies on install. This way I don't really care what core version of linux or solaris or BSD is installed on each platform since we do not control them all. Precisely _because_ they are not well managed is why its important for our environment to build itself cleanly. No, not the core os etc but netpbm, imagemagick, ripmime and other tools get rebuilt on a 'major' config. I guess in your book we suck either way eh? Life goes on :) John On Thu, 31 Oct 2002 13:02:20 -0800 (PST) Ask Bjoern Hansen [EMAIL PROTECTED] wrote: On Wed, 30 Oct 2002 [EMAIL PROTECTED] wrote: I don't believe in transfering _any_ binaries around, every binary recompiles on its new platform at install time. All modules, apache, external software etc. This eliminates those pesky little problems that pop up when you start pushing binaries. Uhmn, if your systems are well managed you don't get any of those pesky little problems. Do you recompile the base system on each server too? In my experience, then as soon as you have more than a few handfuls of servers you get more trouble trying to coordinate recompiling binaries than doing it once and distributing them (in tarballs, rpms, or with your revisioning system). - ask -- ask bjoern hansen, http://www.askbjoernhansen.com/ !try; do();
Re: [OTish] Version Control?
On Thu, 31 Oct 2002 [EMAIL PROTECTED] wrote: [...] I guess in your book we suck either way eh? Life goes on :) You'll change your ways when you get more servers. :-) It's not easy enough for the script to do anything if the base system installations are not very similar. If they are; then it's much easier to distribute binaries. If they are not then that's the first thing you'll change as you add servers above what is managable otherwise. You are right, that if your software runs on many uncontrolled small groups of servers, then it's likely easier to recompile for each box. - ask -- ask bjoern hansen, http://www.askbjoernhansen.com/ !try; do();
[OTish] Version Control?
Does anyone in the list use any kind of version control (e.g. CVS) for the perl/template codebase of their website? Now that my code base is growing I feel the increasing need to provide better version/backup control than my current hourly crontab tar. I don't however feel that the organizational logic of a websites code base fits well into the CVS paradigm. Am I being to short sighted in this assumption? Does anyone have any recommended method? I don't use version numbers at all? Does anyone? Richard.
Re: [OTish] Version Control?
We felt the same way but once we went to CVS we never looked back and can not imagine going with out source control. It may seem like the web doesnt fit that paradigm but if you break your modules up properly it works like a champ. We broke out into 'html','components', 'Libs','external_tools','internal_tools','perlinstall'. This gave us good control over each different area. For our development team its more about consistency then versioning. If you go all the way with it like we did you can give each developer a sandbox that they work in and CVS merges for you, it is a huge benefit. Its to the point now where you check out all the modules and run one script. That script builds all the perl dependancies, rebuilds your http daemon, rebuilds the proxies, configures the server for the platform its on based on hostname and installs all the relevant files. Every so often we bundle everything up into a tagged 'Release' and send it on its way to production. This works really well. A case in point was when we did our I18N conversion. We had one version of the code that was being entirely hacked apart to accomodate our changes but we still had to actively support bug fixes on the release. Without CVS[insert favorite source system here] this would have been impossible. So, without good CVS things like our I18N effort, our auto-install systems etc would have not been possible or been a LOT more painful. As it was we busted through it in record time. John- On Wed, 30 Oct 2002 21:09:05 + Richard Clarke [EMAIL PROTECTED] wrote: Does anyone in the list use any kind of version control (e.g. CVS) for the perl/template codebase of their website? Now that my code base is growing I feel the increasing need to provide better version/backup control than my current hourly crontab tar. I don't however feel that the organizational logic of a websites code base fits well into the CVS paradigm. Am I being to short sighted in this assumption? Does anyone have any recommended method? I don't use version numbers at all? Does anyone? Richard.
Re: [OTish] Version Control?
Richard, What's your operation system? Solaris or Linux? I use Solaris with CVS as the version control. It is very useful tool. You can also consider to use Solaris Package that is for installation and also the version control as well. Willy Richard Clarke [EMAIL PROTECTED]To: [EMAIL PROTECTED] cc: Subject: [OTish] Version Control? 10/30/2002 04:09 PM Does anyone in the list use any kind of version control (e.g. CVS) for the perl/template codebase of their website? Now that my code base is growing I feel the increasing need to provide better version/backup control than my current hourly crontab tar. I don't however feel that the organizational logic of a websites code base fits well into the CVS paradigm. Am I being to short sighted in this assumption? Does anyone have any recommended method? I don't use version numbers at all? Does anyone? Richard.
RE: [OTish] Version Control?
CVS is easy to use but confusing at first. Once you get used to it, you should not complain. I don't quite get your saying I don't however feel that the organizational logic of a websites code base fits well into the CVS paradigm. Isn't your files hierarchical? If so, why is CVS not fitting your purpose? Chang-Ping Hsiao -Original Message- From: Richard Clarke [mailto:ric;likewhoa.com] Sent: Wednesday, October 30, 2002 1:09 PM To: [EMAIL PROTECTED] Subject: [OTish] Version Control? Does anyone in the list use any kind of version control (e.g. CVS) for the perl/template codebase of their website? Now that my code base is growing I feel the increasing need to provide better version/backup control than my current hourly crontab tar. I don't however feel that the organizational logic of a websites code base fits well into the CVS paradigm. Am I being to short sighted in this assumption? Does anyone have any recommended method? I don't use version numbers at all? Does anyone? Richard. .
RE: [OTish] Version Control?
I've wondered about this too. Mainly, if you have multiple developers working with the same web server, how would you test your scripts without running into each other? it seems like CVS would work well if everyone was developing on his/her own box. Nate -Original Message- From: Hsiao, Chang-Ping [mailto:CHsiao;corp.untd.com] Sent: Wednesday, October 30, 2002 4:29 PM To: 'Richard Clarke'; [EMAIL PROTECTED] Subject: RE: [OTish] Version Control? CVS is easy to use but confusing at first. Once you get used to it, you should not complain. I don't quite get your saying I don't however feel that the organizational logic of a websites code base fits well into the CVS paradigm. Isn't your files hierarchical? If so, why is CVS not fitting your purpose? Chang-Ping Hsiao -Original Message- From: Richard Clarke [mailto:ric;likewhoa.com] Sent: Wednesday, October 30, 2002 1:09 PM To: [EMAIL PROTECTED] Subject: [OTish] Version Control? Does anyone in the list use any kind of version control (e.g. CVS) for the perl/template codebase of their website? Now that my code base is growing I feel the increasing need to provide better version/backup control than my current hourly crontab tar. I don't however feel that the organizational logic of a websites code base fits well into the CVS paradigm. Am I being to short sighted in this assumption? Does anyone have any recommended method? I don't use version numbers at all? Does anyone? Richard. .
Re: [OTish] Version Control?
Just give each developer their own sandbox. We have gone from : 1 Apache proxy/1 modperl server to 1 apache proxy / 1 modperl per developer running out of their homedirs to each developer gets their own proxy and modperl. If you tune your apache min/max server stuff this is quite doable. Sandboxes are key though, everyone works on their own stuff. John- On Wed, 30 Oct 2002 16:34:24 -0500 Nathan Hardt [EMAIL PROTECTED] wrote: I've wondered about this too. Mainly, if you have multiple developers working with the same web server, how would you test your scripts without running into each other? it seems like CVS would work well if everyone was developing on his/her own box. Nate -Original Message- From: Hsiao, Chang-Ping [mailto:CHsiao;corp.untd.com] Sent: Wednesday, October 30, 2002 4:29 PM To: 'Richard Clarke'; [EMAIL PROTECTED] Subject: RE: [OTish] Version Control? CVS is easy to use but confusing at first. Once you get used to it, you should not complain. I don't quite get your saying I don't however feel that the organizational logic of a websites code base fits well into the CVS paradigm. Isn't your files hierarchical? If so, why is CVS not fitting your purpose? Chang-Ping Hsiao -Original Message- From: Richard Clarke [mailto:ric;likewhoa.com] Sent: Wednesday, October 30, 2002 1:09 PM To: [EMAIL PROTECTED] Subject: [OTish] Version Control? Does anyone in the list use any kind of version control (e.g. CVS) for the perl/template codebase of their website? Now that my code base is growing I feel the increasing need to provide better version/backup control than my current hourly crontab tar. I don't however feel that the organizational logic of a websites code base fits well into the CVS paradigm. Am I being to short sighted in this assumption? Does anyone have any recommended method? I don't use version numbers at all? Does anyone? Richard. .
Re: [OTish] Version Control?
John, [EMAIL PROTECTED] wrote: We felt the same way but once we went to CVS we never looked back and can not imagine going with out source control. It may seem like the web doesnt fit that paradigm but if you break your modules up properly it works like a champ. We broke out into 'html','components', 'Libs','external_tools','internal_tools','perlinstall'. This gave us good control over each different area. For our development team its more about consistency then versioning. If you go all the way with it like we did you can give each developer a sandbox that they work in and CVS merges for you, it is a huge benefit. Its to the point now where you check out all the modules and run one script. That script builds all the perl dependancies, rebuilds your http daemon, rebuilds the proxies, configures the server for the platform its on based on hostname and installs all the relevant files. Every so often we bundle everything up into a tagged 'Release' and send it on its way to production. This works really well. A case in point was when we did our I18N conversion. We had one version of the code that was being entirely hacked apart to accomodate our changes but we still had to actively support bug fixes on the release. Without CVS[insert favorite source system here] this would have been impossible. Do you use CVS checkouts to upgrade the live system or do you this manually. i.e. stop apache, tar and remove old code, untar new code, start apache et voila? So, without good CVS things like our I18N effort, our auto-install systems etc would have not been possible or been a LOT more painful. As it was we busted through it in record time. John- Ric.
RE: [OTish] Version Control?
Hi Richard -- Does anyone in the list use any kind of version control (e.g. CVS) for the perl/template codebase of their website? Every time! My team has been developing web sites with CVS for years now. I can't imagine doing a web project without it! It would be like.. Oh... Skydiving without a parachute? Now that my code base is growing I feel the increasing need to provide better version/backup control than my current hourly crontab tar. I don't however feel that the organizational logic of a websites code base fits well into the CVS paradigm. Am I being to short sighted in this assumption? Does anyone have any recommended method? I don't use version numbers at all? Does anyone? Web development projects can map very nicely into CVS. We have a very mature layout for all web projects. In a nutshell, it boils down to this: project/ + apache/ + bin/ + cron/ + devdocs/ + htdocs/ + modules/ + templates/ The project/ directory is the CVS module, and is an identifier for the project on which we are working. So, if we were hired to re-develop all of Yahoo! in mod_perl (*grin*), the project directory might be yahoo/. A developer might check out a copy of the project with one command: $ cvs co yahoo All the other directories reflect the fact that every web site has meta content beyond htdocs -- Perl modules, template files, etc. Here is a run-down on the purpose of each directory: apache/- This directory contains a compiled Apache binary tree, complete with configuration files, shared libraries, etc. This, combined with CVS' branching and merging, has made upgrading Apache for our client's sites a breeze. bin/ - A directory for executable scripts and binaries related to the project. These may be sys-admin utilities for managing bits of the project or production web site. Every project ends up with a tool box of these. cron/ - Similar to bin/, this contains scripts and executables. Files in this directory are expected to be accessed via cron. I also put a copy of the crontab file here so that it, too, can be version controlled. Very useful. devdocs/ - Documentation! You have some of that, right? :-) Also included, any installation files (RPMs, packages, external source tarballs, etc.) needed by the project. htdocs/- The hypertext documents directory. This is the directory is mapped as the document root in your web server config. modules/ - Project Perl modules. Via the web server configuration, the PERL5LIB environment variable publishes the path to this directory so that your code can find what it needs. Besides the code we write for a given project, we also include *ALL* the CPAN Perl modules used for the project. It has been a long time since I've had to figure out which Perl modules are required for a web site. templates/ - We use HTML::Template, and this path contains all the external template files. (TT users can also use this structure.) Via the web server configuration, the environment variable HTML_TEMPLATE_ROOT publishes the path to this directory so that your code can find what it needs. Our environment is specifically designed to allow every developer, designer, QA specialist, project manager, and client to have their own completely independent, concurrent environment. CVS provides the glue for connecting these environments. Other than the occasional FTP binary/ASCII mode upload snafu, it is rare that developers on my projects step on each others' feet. FYI, we work exclusively on UNIX. Our work is split roughly 80/20, Linux/Solaris. Warmest regards, -Jesse- -- Jesse Erlbaum The Erlbaum Group [EMAIL PROTECTED] Phone: 212-684-6161 Fax: 212-684-6226
Re: [OTish] Version Control?
We use CVS to do in-place upgrades on the live system for smaller updates. For the big stuff we bring the boxes out of their pools one at a time and upgrade them. In both cases, the worse case is that a user might see two versions of the same page in the span of 60 seconds if they catch us in mid-update. John- On Wed, 30 Oct 2002 21:47:01 + Richard Clarke [EMAIL PROTECTED] wrote: John, [EMAIL PROTECTED] wrote: We felt the same way but once we went to CVS we never looked back and can not imagine going with out source control. It may seem like the web doesnt fit that paradigm but if you break your modules up properly it works like a champ. We broke out into 'html','components', 'Libs','external_tools','internal_tools','perlinstall'. This gave us good control over each different area. For our development team its more about consistency then versioning. If you go all the way with it like we did you can give each developer a sandbox that they work in and CVS merges for you, it is a huge benefit. Its to the point now where you check out all the modules and run one script. That script builds all the perl dependancies, rebuilds your http daemon, rebuilds the proxies, configures the server for the platform its on based on hostname and installs all the relevant files. Every so often we bundle everything up into a tagged 'Release' and send it on its way to production. This works really well. A case in point was when we did our I18N conversion. We had one version of the code that was being entirely hacked apart to accomodate our changes but we still had to actively support bug fixes on the release. Without CVS[insert favorite source system here] this would have been impossible. Do you use CVS checkouts to upgrade the live system or do you this manually. i.e. stop apache, tar and remove old code, untar new code, start apache et voila? So, without good CVS things like our I18N effort, our auto-install systems etc would have not been possible or been a LOT more painful. As it was we busted through it in record time. John- Ric.
RE: [OTish] Version Control?
Hi Richard -- Do you use CVS checkouts to upgrade the live system or do you this manually. i.e. stop apache, tar and remove old code, untar new code, start apache et voila? On single-server web sites (as opposed to a site running in a clustered environment) a CVS checkout works just fine. Once you get into a situation where you need to update multiple servers simultaneously you need another system. For sites with a modest amount of traffic and a smallish cluster, NFS works very well. For high-volume sites with a large cluster of servers I would use rsync. FYI, I used to be adverse to having a checked out CVS working directory on a production website, but I changed my tune on that years ago. Using CVS for distribution of changes into production is far easier, less prone to mistakes and downright efficient than distributing static files. CVS over SSH is also encrypted and secure. Just make sure to configure your web server to block access to CVS/ directories! TTYL, -Jesse- -- Jesse Erlbaum The Erlbaum Group [EMAIL PROTECTED] Phone: 212-684-6161 Fax: 212-684-6226
Re: [OTish] Version Control?
Hsiao, Chang-Ping wrote: CVS is easy to use but confusing at first. This could be the root of my reservations. Once you get used to it, you should not complain. I don't quite get your saying I don't however feel that the organizational logic of a websites code base fits well into the CVS paradigm. Isn't your files hierarchical? If so, why is CVS not fitting your purpose? Indeed, I don't get what I'm saying either. My organisation is something like (slightly over-simplified), _lib/Example/Control/section_a.pm _lib/Example/Control/section_b.pm _lib/Example/Control/section_c.pm _lib/Example/Config/section_a.pm _lib/Example/Config/section_b.pm _lib/Example/Config/section_c.pm _lib/Example/Model/section_c.pm _lib/Example/Model/section_c.pm _lib/Example/Model/section_c.pm _tmpl/section_a/view1.html _tmpl/section_a/view2.html _tmpl/section_a/view3.html _tmpl/section_b/view1.html _tmpl/section_b/view2.html _tmpl/section_b/view3.html _tmpl/section_c/view1.html _tmpl/section_c/view2.html _tmpl/section_c/view3.html I though CVS was more suited for something which looks more like _lib/Example/Section_a/Control _lib/Example/Section_a/Model _lib/Example/Section_a/View/1 _lib/Example/Section_a/View/2 _lib/Example/Section_a/View/3 etc i.e. more suited to a procedural orangisation than an object organisation Then again, like i said... perhaps my assumptions were too short sighted. Ric.
Re: [OTish] Version Control?
On Oct 30 Richard Clarke wrote: Does anyone in the list use any kind of version control (e.g. CVS) for the perl/template codebase of their website? Using cvs, I'm not sure of an elegant way to update. I'm worried about CVS subdirectories sitting under htdocs, to be specific. Perhaps I just am not familiar enough with cvs. Perhaps there is an option that will supress the CVS that will ease rollouts. The specific comments given so far are great. On Oct 30 Jesse Erlbaum wrote: SSH is also encrypted and secure. Just make sure to configure your web server to block access to CVS/ directories! Also, Randal Schwartz wrote about cvs is a slightly more general setting: http://www.linux-mag.com/2002-07/perl_01.html Jim
Re: [OTish] Version Control?
Hear hear! We're using AxKit for our development and everyone has a copy of the entire site (many thousand files) in their home directories on the development machine and its been great! No more I didn't touch it but it stopped working problems :) It also allows everyone to experiment with features without messing up what anyone else is working on. Combine that with a series of perl scripts (also in cvs) which generate our bulk content (i.e. massive database dumps into xml files), Every so often we run a cvs update on our production machine to keep things in sync (we treat it just like another client). Go for it...its worth the trouble...especially the first time someone breaks something and you can determine who and when it was broken. Brian Wheeler [EMAIL PROTECTED] On Wed, 2002-10-30 at 16:30, [EMAIL PROTECTED] wrote: We felt the same way but once we went to CVS we never looked back and can not imagine going with out source control. It may seem like the web doesnt fit that paradigm but if you break your modules up properly it works like a champ. We broke out into 'html','components', 'Libs','external_tools','internal_tools','perlinstall'. This gave us good control over each different area. For our development team its more about consistency then versioning. If you go all the way with it like we did you can give each developer a sandbox that they work in and CVS merges for you, it is a huge benefit. Its to the point now where you check out all the modules and run one script. That script builds all the perl dependancies, rebuilds your http daemon, rebuilds the proxies, configures the server for the platform its on based on hostname and installs all the relevant files. Every so often we bundle everything up into a tagged 'Release' and send it on its way to production. This works really well. A case in point was when we did our I18N conversion. We had one version of the code that was being entirely hacked apart to accomodate our changes but we still had to actively support bug fixes on the release. Without CVS[insert favorite source system here] this would have been impossible. So, without good CVS things like our I18N effort, our auto-install systems etc would have not been possible or been a LOT more painful. As it was we busted through it in record time. John- On Wed, 30 Oct 2002 21:09:05 + Richard Clarke [EMAIL PROTECTED] wrote: Does anyone in the list use any kind of version control (e.g. CVS) for the perl/template codebase of their website? Now that my code base is growing I feel the increasing need to provide better version/backup control than my current hourly crontab tar. I don't however feel that the organizational logic of a websites code base fits well into the CVS paradigm. Am I being to short sighted in this assumption? Does anyone have any recommended method? I don't use version numbers at all? Does anyone? Richard.
Re: [OTish] Version Control?
Jim Martinez wrote: On Oct 30 Richard Clarke wrote: Does anyone in the list use any kind of version control (e.g. CVS) for the perl/template codebase of their website? Using cvs, I'm not sure of an elegant way to update. I'm worried about CVS subdirectories sitting under htdocs, to be specific. Perhaps I just am not familiar enough with cvs. Perhaps there is an option that will supress the CVS that will ease rollouts. The specific comments given so far are great. On Oct 30 Jesse Erlbaum wrote: SSH is also encrypted and secure. Just make sure to configure your web server to block access to CVS/ directories! Also, Randal Schwartz wrote about cvs is a slightly more general setting: http://www.linux-mag.com/2002-07/perl_01.html Jim I should have known better. Is there any topic he *hasn't* written an article on? Ric
RE: [OTish] Version Control?
Hey Jim -- Also, Randal Schwartz wrote about cvs is a slightly more general setting: http://www.linux-mag.com/2002-07/perl_01.html The system we use goes a bit beyond even what Randal describes (although, he is on a similar track). In a nutshell, the Apache httpd.conf file is templatized to replace elements such as the IP address, host name and path names with variables. To start the server for a particular environment the developer runs a Perl program which creates a custom httpd.conf
Re: [OTish] Version Control?
Hey Jim -- Also, Randal Schwartz wrote about cvs is a slightly more general setting: http://www.linux-mag.com/2002-07/perl_01.html The system we use goes a bit beyond even what Randal describes (although, he is on a similar track). In a nutshell, the Apache httpd.conf file is templatized to replace elements such as the IP address, host name and path names with variables. To start the server for a particular environment the developer runs a Perl program which creates a custom httpd.conf This is so close to what I recently put together one would have to worry about plagarism :-) I took it one step further, in that my setup script will configure not only the base http config files (for both proxy and app server), but also all the database configuration scripts, and application configurations. I did an install for a fellow developer today in about 5 minutes - complete proxy/app server config, build and populate database tables. I had to edit one file, which in turn was used to create the customized configuration files. I don't keep the base config files in CVS - yet. All source is in CVS, but I haven't got around to moving the config into CVS. Regarding deployment, I wrote a tool which goes into CVS and extracts tagged modules. The extracted code is placed into a tar archive with a few other files which indicate where the code is to be installed, plus a few other goodies. I can then give this deployment archive to our sysadmin who can simply type deploy --install the-file.dar and the code gets installed. We have separate deployment files for the perl libs, the Mason components, images, and a few other odds and ends. It works well, and it's very easy. Why a custom deploy tool? I didn't like the idea of using CVS on the production server, which is in a very tightly controlled environment. I doubt if I could have enticed the network folks to open the firewall to let CVS traffic through in any case Cheers! -klm.
Re: [OTish] Version Control?
Who needs network guys, reverse pop the ssh tunnel ;) I find it amazing that so many of us are doing the exact same thing in terms of managing our large site installs yet its nowhere to be found in any FAQ, knowledge base or general forum. I know on our side we developed these solutions over months and months on our own, dealing with problems at every step. Had there been a nice little FAQ on it, well, it would have saved us a ton of time :) I sense some writing.. John- On Wed, 30 Oct 2002 15:40:45 -0700 Ken Miller [EMAIL PROTECTED] wrote: doubt if I could have enticed the network folks to open the firewall to let CVS traffic through in any case Cheers! -klm.
RE: [OTish] Version Control?
Hi Jesse, I really like your approach, and appreciate your explanation. I'm wondering how you handle the packaging and installation; do you just use a shell script to put the different pieces where they belong, or have you devised something else? Right now I'm also using CVS for my web development, and have the modules built around ExtUtils::ModuleMaker. The server gets updated via the standard perl Makefile.PL make make test make install. My htdocs and templates are in their own dirs but in the same overall hierarchy as what ModuleMaker generates, and they get installed during the 'make install' part through some extra rules in the Makefile.PL files. Devdocs are all in POD, but I don't really have the bin and cron types of things from your system well integrated; they seem too peripheral to include with the 'make install' stuff. Of course another consideration is that I'm developing for an intranet, not deploying to external customers, so I have a lot more control of the production server. Wes Sheldahl Jesse Erlbaum [EMAIL PROTECTED] on 10/30/2002 04:47:41 PM To:[EMAIL PROTECTED] cc: Subject:RE: [OTish] Version Control? small excerpt from original post follows Web development projects can map very nicely into CVS. We have a very mature layout for all web projects. In a nutshell, it boils down to this: project/ + apache/ + bin/ + cron/ + devdocs/ + htdocs/ + modules/ + templates/ Jesse Erlbaum The Erlbaum Group [EMAIL PROTECTED] Phone: 212-684-6161 Fax: 212-684-6226
RE: [OTish] Version Control?
At 04:47 PM 10/30/02 -0500, Jesse Erlbaum wrote: Web development projects can map very nicely into CVS. We have a very mature layout for all web projects. In a nutshell, it boils down to this: project/ + apache/ + bin/ That requires binary compatibility, though. I have a similar setup, but the perl and Apache are built separately on the target machine since my machines are linux and the production machine is Solaris. I only work on single servers, so things are a bit easier. I always cvs co to a new directory on the production machine and start up a second set of servers on high ports. That lets me (and the client) test on the final platform before going live. Then it's apache stop mv live old mv new live apache start kind of thing, which is a fast way to update. I'd love to have the Perl modules in cvs, though. Especially mod_perl modules. It makes me nervous upgrading mod_perl on the live machine's perl library. Should make more use of PREFIX, I suppose. Speaking of cvs, here's a thread branch: I have some client admin features that they update via web forms -- some small amount of content, templates, and text-based config settings. I currently log a history of changes, but it doesn't have all the features of cvs. Is anyone using cvs to manage updates made with web-based forms? -- Bill Moseley mailto:moseley;hank.org
RE: [OTish] Version Control?
Hi Richard -- Thanks for the extensive info, I am curious Assuming your website is made up different sections, would I be correct in assuming that the files this section depends on are finite? e.g. bin/section_a.sh cron/section_a.crontab template/section_a.html htdocs/section_a.pm Does CVS provide any ability to checkout the corresponding file from each section? I figure this would require a certain amount of developer supplied meta data so that CVS knew which files we part of which sections. Is this kind of thing supported. I'm not sure if I understand how you are using the word section here. The way I use CVS, I always check out ALL the files related to a project -- even those not related to the immediate task at hand. There is no advantage (at least not one I can see) for checking out only certain files. Finally, going completely off topic, does your choice of editor reflect your extensive usage of CVS say emacs with a cvs module? I don't think so. I actually use pico. (Don't bother making fun of me -- I've already been roasted by the best of 'em!) A number of my developers are emacs nuts, one uses vi (ouch!), and one uses a Windows-based editor and FTP. I imagine emacs has some sort of special CVS tie-in (it ties in everything else!), but it obviously isn't really important for basic CVS usage. TTYL, -Jesse- -- Jesse Erlbaum The Erlbaum Group [EMAIL PROTECTED] Phone: 212-684-6161 Fax: 212-684-6226
RE: [OTish] Version Control?
This could be the root of my reservations. Why do you think people always talk about re-organize? :-) Indeed, I don't get what I'm saying either. My organisation is something like (slightly over-simplified), _lib/Example/Control/section_a.pm _lib/Example/Control/section_b.pm _lib/Example/Control/section_c.pm _lib/Example/Config/section_a.pm _lib/Example/Config/section_b.pm _lib/Example/Config/section_c.pm _lib/Example/Model/section_c.pm _lib/Example/Model/section_c.pm _lib/Example/Model/section_c.pm _tmpl/section_a/view1.html _tmpl/section_a/view2.html _tmpl/section_a/view3.html _tmpl/section_b/view1.html _tmpl/section_b/view2.html _tmpl/section_b/view3.html _tmpl/section_c/view1.html _tmpl/section_c/view2.html _tmpl/section_c/view3.html I though CVS was more suited for something which looks more like _lib/Example/Section_a/Control _lib/Example/Section_a/Model _lib/Example/Section_a/View/1 _lib/Example/Section_a/View/2 _lib/Example/Section_a/View/3 etc i.e. more suited to a procedural orangisation than an object organisation In fact, CVS doesn't complain about how you organize your files. As long as they are in directories, CVS seems to work fine! Of course your organization needs to fit your needs -- for you to easily find what you want, but that's out of the scope of the CVS discussion. Chang-Ping .
Re: [OTish] Version Control?
We check in all of our perl modules into CVS and its a _MAJOR_ life saver. Keeps everyone on the same path so to speak. I don't believe in transfering _any_ binaries around, every binary recompiles on its new platform at install time. All modules, apache, external software etc. This eliminates those pesky little problems that pop up when you start pushing binaries. John- On Wed, 30 Oct 2002 14:58:01 -0800 Bill Moseley [EMAIL PROTECTED] wrote: At 04:47 PM 10/30/02 -0500, Jesse Erlbaum wrote: Web development projects can map very nicely into CVS. We have a very mature layout for all web projects. In a nutshell, it boils down to this: project/ + apache/ + bin/ That requires binary compatibility, though. I have a similar setup, but the perl and Apache are built separately on the target machine since my machines are linux and the production machine is Solaris. I only work on single servers, so things are a bit easier. I always cvs co to a new directory on the production machine and start up a second set of servers on high ports. That lets me (and the client) test on the final platform before going live. Then it's apache stop mv live old mv new live apache start kind of thing, which is a fast way to update. I'd love to have the Perl modules in cvs, though. Especially mod_perl modules. It makes me nervous upgrading mod_perl on the live machine's perl library. Should make more use of PREFIX, I suppose. Speaking of cvs, here's a thread branch: I have some client admin features that they update via web forms -- some small amount of content, templates, and text-based config settings. I currently log a history of changes, but it doesn't have all the features of cvs. Is anyone using cvs to manage updates made with web-based forms? -- Bill Moseley mailto:moseley;hank.org
RE: [OTish] Version Control?
Hi Wes -- I really like your approach, and appreciate your explanation. I'm wondering how you handle the packaging and installation; do you just use a shell script to put the different pieces where they belong, or have you devised something else? Something else! Files don't get installed all over the system -- they all stay together. We use environment variables (PERL5LIB, HTML_TEMPLATE_ROOT, etc.) set by the web server configuration to point to where the assets are stored. This assures that installing and upgrading doesn't involve managing a spider web of files all over the operating system. FYI, in production we typically create a UNIX user who owns all project files, and put all these files in that user's home directory. This simplifies management enormously. Right now I'm also using CVS for my web development, and have the modules built around ExtUtils::ModuleMaker. The server gets updated via the standard perl Makefile.PL make make test make install. Very cool! If you make conscience use of the built-in mechanism for regression testing for your web apps you will really have a fantastic software development factory. I don't really have the bin and cron types of things from your system well integrated; they seem too peripheral to include with the 'make install' stuff. Of course another consideration is that I'm developing for an intranet, not deploying to external customers, so I have a lot more control of the production server. The only difference here is that your job continues beyond the project, where our job (as contractors) generally scales down. IOW, if something unplanned comes up, you are probably going to be available to answer questions. If you can isolate all project related files to a single UNIX user you will have a tremendously portable and maintainable environment. For example, even the cron-processes on our sites run as the user who owns the project files. TTYL, -Jesse- -- Jesse Erlbaum The Erlbaum Group [EMAIL PROTECTED] Phone: 212-684-6161 Fax: 212-684-6226
Re: [OTish] Version Control?
* [EMAIL PROTECTED] ([EMAIL PROTECTED]) [31 Oct 2002 09:47]: [...] I find it amazing that so many of us are doing the exact same thing in terms of managing our large site installs yet its nowhere to be found in any FAQ, knowledge base or general forum. I suspect a lot of us are so used to having everything in some sort of VCS that we take it for granted =) (RCS at work; CVS at home; both at uni; experimenting with perforce [which, so far, appears nicer than CVS] ) cheers, -- Iain.
RE: [OTish] Version Control?
Hi Bill -- project/ + apache/ + bin/ That requires binary compatibility, though. I have a similar setup, but the perl and Apache are built separately on the target machine since my machines are linux and the production machine is Solaris. Binary incompatibility is not a problem for us because we generally develop on the same platform on which the final code will run. I've been bitten enough times that I don't like to wait until the proverbial last minute to test my code on the target platform. Part of our job is to make sure that *all* the parts of the system work -- even those developed by the Apache Software Foundation. ;-) After all, my client isn't going to accept the excuse that our code is perfect -- it's that OTHER code! At the end of the day, if we recommend Apache (or any other third-party code), we are expected to stand by it. That said, if you have to support multiple platforms for any reason (a Solaris to Linux migration, for instance... Hehe) you are right -- extra work is involved. Perl has a nice structure for dealing with multiple architectures (arch library paths i686/, sun4/, etc.) but you will have to devise something else for application like Apache. I only work on single servers, so things are a bit easier. I always cvs co to a new directory on the production machine and start up a second set of servers on high ports. That lets me (and the client) test on the final platform before going live. Then it's apache stop mv live old mv new live apache start kind of thing, which is a fast way to update. I don't like any system which requires you to make any last minute changes. If you test on port 8080, there is no proof that you won't have problems when you move to port 80. What happens if a developer has clumsily hard-coded :8080 into a link to solve a problem in testing? I've seen it happen a thousand times, even to very smart programmers. If you have root access on the server you would be better off binding an additional IP address and running the stage server there, IMHO. I'd love to have the Perl modules in cvs, though. Especially mod_perl modules. It makes me nervous upgrading mod_perl on the live machine's perl library. Should make more use of PREFIX, I suppose. That is more or less how you do it. I have a module build process which allows me to build ALL modules in user space, isolated from the host operating system. If you put CPAN modules (including mod_perl) in CVS you will even be able to select versions. I've had forward-compatibility problems with CPAN modules in the past. Being able to run an old version of a library for one site, and a different version on the host machine is a really helpful thing. Note that if you want to put mod_perl in CVS you *have* to also put an Apache binary in mod_perl. There is no way around it. I've created an Apache mod_perl build process which automates compiling in user space: http://www.erlbaum.net/Apache-Perl-SSL/ HTH! Speaking of cvs, here's a thread branch: I have some client admin features that they update via web forms -- some small amount of content, templates, and text-based config settings. I currently log a history of changes, but it doesn't have all the features of cvs. Is anyone using cvs to manage updates made with web-based forms? Humm... You're talking about revision control on databases. That's a really difficult thing. In short, it's not possible to revision control a SQL database. If you have data which needs to be revision controlled put it in XML or in a Perl module. Another thing to add: Only files which are directly managed by a developer can be effectively managed in CVS. Files which are changed through a web form or an automated process are not candidates for CVS because the automation can't usually do important things like commit changes and resolve merge conflicts. I actually create a directory hierarchy outside of CVS, generally called EXTERNALS/, to store files which are managed through automation. TTYL, -Jesse- -- Jesse Erlbaum The Erlbaum Group [EMAIL PROTECTED] Phone: 212-684-6161 Fax: 212-684-6226
Re: [OTish] Version Control?
At 03:21 PM 10/30/02 -0800, [EMAIL PROTECTED] wrote: We check in all of our perl modules into CVS and its a _MAJOR_ life saver. Keeps everyone on the same path so to speak. I think I confused two different things: perl module source vs. installed modules. Do you check in the source or the installed modules? I keep the source of my perl modules under cvs, but not the perl library i.e. the files generated from make install, which might include binary components. I use a PREFIX for my own modules, but I tend to install CPAN modules in the main perl library. My own modules get installed in the application directory tree so that there's still a top level directory for the entire application/site. It does worry me that I'll update a CPAN module (or Apache::*) in the main Perl library and break something some day. (Although on things like updating mod_perl I have copied /usr/local/lib/perl5 before make install.) -- Bill Moseley mailto:moseley;hank.org
Re: [OTish] Version Control?
Richard == Richard Clarke [EMAIL PROTECTED] writes: Richard Does anyone in the list use any kind of version control (e.g. CVS) for Richard the perl/template codebase of their website? Yup. Even wrote a column about it: http://www.stonehenge.com/merlyn/LinuxMag/col38.html -- Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095 [EMAIL PROTECTED] URL:http://www.stonehenge.com/merlyn/ Perl/Unix/security consulting, Technical writing, Comedy, etc. etc. See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
