On Sep 28, 2013, at 8:12 AM, Brice Goglin <[email protected]> wrote:
>> Let's go through the use cases of what we want:
>>
>> 1. "make dist" in a developer's git clone. This should make a "hwloc-<git
>> describe>.tar.*.
>
> This is actually the critical point, see below.
>
>> 2. make a nightly snapshot tarball. The more I think about this, the more I
>> think it's the same as #1, right?
>
> Yes, that's why I didn't understand why the create_tarball script also
> modified VERSION to add git describe. These changes should be either
> entirely outside of make dist (if we want git describe for nightly but
> not for make dist) or entirely inside make dist (if we want for both).
Agreed. Let's have distscript.csh be the one that sets the git-describe value
in VERSION (more below).
>> 3. make a release tarball, "hwloc-major.minor.releasegreek.tar.*".
>>
>> Are these the three (or two, if #2 is the same as #1) use cases we want? If
>> so, I can see about making the code simpler -- e.g., I didn't know about
>> overriding the VERSION Makefile macro trick...
>
> Changing VERSION on the command-line doesn't change the lstopo
> --version, so it may not be what we really want. Also, if changing the
> suffix is just a sed on VERSION file before autogen or configure, it's
> fine too.
>
> This all depends on what we really want for (1).
> * If we don't do (1), we can remove tons of lines of code from the
> configury and just have the nightly and release scripts modify VERSION
> before running autogen. Easy.
> * If we do (1), that needs much more work.
I'm not sure I agree. Let me review the 3 scripts and what each does (and why):
1. config/distscript.csh:
- Called by "make dist[check]"
- Does 3 main things:
a) Sets git-describe value in VERSION file (but only if snapshot=1)
b) Generates/copies doxygen output files to the dist tree
--> We wanted to not require users to have doxygen
--> We also wanted to ship final doxygen output in tarballs
--> This created quite a bit of complexity in itself
c) Get new config.guess and config.sub files from git.savannah.gnu.org
--> This code is from when GNU Autotools releases were few and far
between; it may not be necessary any more. We might well be able to just
remove this code and still be fine.
2. contrib/nightly/make_snapshot_tarball:
- Invoked via cron on the build machine
- Very specifically written to interact with the web download tree
- Generally does the following:
a) Gets a new git clone (into a unique directory)
b) Compares output from "git describe ..." to latest_snapshot.txt
c) If they're the same, exit 0
--> If there are no commits since last tarball, no need to do anything
d) Run autogen, configure, make distcheck.
e) Move resulting tarballs to the web download directory
e) Re-generate md5sums.txt/sha1sums.txt
3. contrib/dist/make_dist_tarball:
- Invoked manually by a human when making releases
- Generally does the following:
a) Check versions of GNU Autotools
--> To ensure to use *the same* versions of the Autotools for an entire
release series
b) Update VERSION file with the release date
c) Run autogen + configure, build doxygen docs, build README, run make
distcheck
d) Remove "greek" value from VERSION and run c) again
> I actually don't care much about (1), I am used to tarballs without the
> SVN revision suffix (not sure why I didn't always get that suffix). I
> agree that it's convenient to have the suffix for developer builds (when
> you want to compare several of them, when you distribute that tarball
> for some reason, etc). But maybe the nightly script is enough for these
> cases? Does it work with locally modified trees? Or trees that contain
> additional commits?
No, it gets a fresh clone every night. The intent was that this script runs in
an automated fashion, and sometimes the build fails. So it leaves the failed
clone/build in a place where a human can go post-mortem the tree and figure out
why the build failed.
But to be fair, this hasn't been a big issue for hwloc (it has with Open MPI --
e.g., "make dist" works for developers, but it failed on the build machine. So
it was really helpful to be able to login as mpiteam@build_machine and go look
into the build tree and see WTF happened).
It also specifically interacts with the web download tree. To be clear: this
script is all about deciding whether to make a new snapshot, and if so, run
"make dist" to do so, and then copies to results to the web tree.
The core of this is:
- *all* use cases run "make dist"; config/distscript.csh is used by all of them
- make_dist_tarball is some additional accounting surrounding "make dist"
- make_snapshot_tarball is some additional (different) accounting surrounding
"make dist"
All that being said, I think 2 immediate improvements/simplifications are:
- have make_snapshot_tarball not set "git describe" in VERSION
- remove the config.guess/config.sub fetching from distscript.csh
I'll commit those now.
However, for the other cases, I think that doxygen is our main culprit for
complexity. :-\ Meaning: I'm now not sure how to make them simpler...
Thoughts?
> By the way, maybe move that script back from nightly to dist.
Sure; I don't have much of an opinion here.
>> Do you have a favorite git commit emailing script? We can probably use the
>> generic github "WebHook URLs" hook (at the top of the list) and host the
>> diff-emailing script at IU (or anywhere, actually).
>
> I use the attached one for Open-MX and KNEM. Not sure I tried many of
> them, but this one worked fine so far. It generates messages such as:
>
> http://lists.gforge.inria.fr/pipermail/knem-commits/2013-July/000465.html
> I don't think it truncates the diff yet. We may want some separators
> between commits too. All this shouldn't be hard to configure.
It looks like github sends an HTTP post with the following information in it:
https://help.github.com/articles/post-receive-hooks
Do you have an easy place to host your script to try it out? Or do you want to
wait for IU to host it (i.e., tomorrow)?
--
Jeff Squyres
[email protected]
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/
