Re: [ITP] astrometry.net-0.38-1
The package discussed in this thread is now available via http://astrotortilla.sourceforge.net/cygwin-custom -- jussi
Re: [ITP] astrometry.net-0.38-1
Hello, and pardon me for still pestering you. I've got a custom repo now working with setup.exe -X @ http://rubor.org/astrometrytortilla (won't be the final location). Signatures are more troublesome. I suspect my .sig files are in a wrong format, but can't find detailed info on what the correct format is. Currently, I've just done gpg --output setup.ini.sig --sign setup.ini gpg --output setup.bz2.sig --sign setup.bz2 I also did my own key, gpg --gen-keys (to generate a 1024bit DSA key, no passphrase) gpg --export Jussi Kantola tortilla.gpg intending to run setup.exe -K http://rubor.org/astrometrytortilla/tortilla.gpg However, setup.exe can't verify the signatures. Error is Internal Error: gcrypt library error 163 illegal old tag I looked at setup.exe source, for hints on what the signatures should be like, but got the impression that the full libgcrypt set should be useable. Apparently this impression is wrong -- or something. Help? :-) -- jussi
Re: [ITP] astrometry.net-0.38-1
On Fri, Nov 11, 2011 at 12:46 PM, Jussi Kantola wrote: Hello, and pardon me for still pestering you. Right. I didn't intend this for the mailing-list. Also, I think I just got the signatures working by using --detach-sig instead of --sign. Please excuse me my butter-fingers and all that comes with 'em ... -- jussi
Re: [ITP] astrometry.net-0.38-1
Well, well. I'm grateful for esp. the hours Chuck put into this, and for having at least seen the process and all, but I suppose it's time to end this -- I just can't bear to watch losing all the votes so bravely fought for ;-) AstroTortilla is fine with a custom repo. All we ever wanted was to be able to install astrometry.net with Cygwin's setup.exe. There were two reasons for doing the ITP: 1) Astrometry.net is immensely useful, albeit for a relatively minor userbase (*), and it *will* be part of both Cygwin and all the other major/applicable distros, eventually 2) the thought never occured to me that custom repos are possible ... (*) So-called computerized GOTO telescopes have been sold by the unknown tens if not hundreds of thousands over the past 10-20yrs. ALL of them are essentially fixed by AstroTortilla, which critically relies on Astrometry.net. So it may well be that once the word gets out that there's a GOTO correction program just like the one Hubble Space Telescope uses, available for amateur astronomers and compatible with their trusty old mounts, we'll see some downloads. How many would we need for it to be considered significant enough? Is this document still valid? http://sourceware.org/cygwin-apps/package-server.html Anything else I need to know? Thanks once again for your time and effort! I'm sorry the lessons you gave me will go down the drain if I won't become a package manager ... ;-) -- jussi P.S. If this message doesn't turn out to be the end of story just yet, let it be known that I will have a look at building the package with dynamic linking, if package size is deemed a bigger issue than the superficially miniscule userbase. It's just there's work (as in paycheck) to do, and my family has very recently grown by one, so I'm in no greater hurry with this than before ...
Re: [ITP] astrometry.net-0.38-1
On 11/9/2011 5:00 AM, Jussi Kantola wrote: AstroTortilla is fine with a custom repo. All we ever wanted was to be able to install astrometry.net with Cygwin's setup.exe OK. How many would we need for it to be considered significant enough? No idea. Is this document still valid? http://sourceware.org/cygwin-apps/package-server.html Seems accurate -- but it's missing information about gpg security. I think you want Creating a custom Cygwin package server -- you probably don't want to create or host a full mirror. Anything else I need to know? Here's what I do, locally: top/setup.exe top/genini top/release/foo/foo-1.2.3-1.tar.bz2 top/release/foo/foo-1.2.3-1-src.tar.bz2 top/release/foo/setup.hint $ cd cygwin $ ./genini --recursive release setup.ini $ bzip2 -c setup.ini setup.bz2 Then, upload setup.ini, setup.bz2, the new tarballs and setup.hint to your website, replicating the directory structure (from top/ on down). Now, your users will have to invoke setup.exe with the -X, because otherwise setup.exe will expect the setup.ini/bz2 files to be signed. However, turning the security measures off is a problem, because then your users have no protection against corrupted files on the *main* mirrors, either. So, ideally, you would ALSO sign *your* setup.ini/setup.bz2 files. See http://www.cygwin.com/ml/cygwin-announce/2008-08/msg1.html Now, this still requires your end users to take an explicit action (see item (3i),(3ii),(3iii) in the referenced announcement.) You could enable them to do (3i) or (3iii) via a batch file that you distribute...or... See the cygwin-ports instructions for their users, here: http://sourceware.org/cygwinports/ In that case, the use of 'cygstart' implies that cygwinports would be *added* to an existing cygwin installation; hence a bare-windows installation would require two separate setup.exe runs (*). This is actually a /good/ thing, because it means there's no confusion between the standard cygwin installation on my box and the cygwinports cygwin installation on my box -- your end users would just have one, to which they've added the extra stuff. (*) future update runs of setup would handle both the 'standard' packages and the addons simultaneously. Thanks once again for your time and effort! I'm sorry the lessons you gave me will go down the drain if I won't become a package manager ... ;-) You're still managing a package...it just wouldn't be hosted as an intrinsic part of the cygwin distribution itself. -- Chuck
Re: [ITP] astrometry.net-0.38-1
On 11/8/2011 2:19 AM, Christopher Faylor wrote: Cygwin is supposed to be like a Linux distro. Including packages which come with Linux distros is a no-brainer. Including a large, specialized package which is not commonly found on Linux and which has a small user base is not a no-brainer. [...] Given the fact that the votes needed to trickle in over the course of more than a month, it seems that most people don't feel very strongly about including this package. I understand that the OP wants to have a convenient way of distributing it to the small number of people who need it but I don't think that is necessarily a good enough reason for the package to occupy disk space and bandwidth on sourceware.org and mirrors, for potential package support to show up in the cygwin mailing list, and for someone to take time to upload updates to sourceware.org. I was wondering if anyone else felt like I did about this package. If it hadn't required many weeks to get approval + more weeks to get packaging worked out, it would have gone in without comment from me. Since it did drag on and required multiple pleading messages to keep things moving, it certainly seems like there isn't a lot of enthusiasm for getting this in. The bottom line is that I was trying to ask why people voted +1. If it was just to help the guy out then that's not a really good reason for the package to be in the distro. If it was because people thought this was kinda cool... then that would imply that there was more thought involved. I was the last person to vote +1, and I have to admit that it was mostly to help the guy out. I regret that I didn't give it more thought. Now that I've seen this discussion, I am no longer in favor of including the package in the distro. I think it would make more sense for the OP to set up his own repository. Ken
Re: [ITP] astrometry.net-0.38-1
On 11/8/2011 8:19 AM, Christopher Faylor wrote: I think it's kinda cool for cygwin be one of the first (not THE first; it's already in BSD ports IIUC) to provide these tools, so that's why I voted +1. ^That is actually the type of answer I was looking for. I wanted to know why people thought the package was needed in the distro. cgf Of course there is no need for such package, but I noticed that all distro have some astronomical package and at first glance they are not more used that Astrometry. The time taken to approve is likely linked to such particular usage. I looked at the website and they have a community, a mailing list, a bug tracker and some planning. It has some usage and I can image that porting to windows with cygwin is much simpler than trying another route. For that reason I voted +1. I built it using one of the first trial and I noticed that the build system is crude and every utility is static linked, so personally my preference will be a smaller package using dynamic linking; but it is a personal estetic opinion. If the package size is a problem we can put it as a condition Regards Marco
Re: [ITP] astrometry.net-0.38-1
On Tue, Nov 08, 2011 at 07:01:52PM +0100, Marco Atzeri wrote: On 11/8/2011 8:19 AM, Christopher Faylor wrote: I think it's kinda cool for cygwin be one of the first (not THE first; it's already in BSD ports IIUC) to provide these tools, so that's why I voted +1. ^That is actually the type of answer I was looking for. I wanted to know why people thought the package was needed in the distro. Of course there is no need for such package, but I noticed that all distro have some astronomical package and at first glance they are not more used that Astrometry. Sigh. Ok. Let me rephrase: I wanted to know why people thought the package was useful enough in the distro that they decided to vote for it knowing that standard Linux distros do not contain the package. If all distros have an astronomical package why aren't we looking to get one of those rather than using one that isn't used in other Linux distros? The time taken to approve is likely linked to such particular usage. I can't precisely parse what you're trying to say but you seem to be asserting that everyone else who voted +1 has done the same research as you. We already know that wasn't true. I looked at the website and they have a community, a mailing list, a bug tracker and some planning. It has some usage and I can image that porting to windows with cygwin is much simpler than trying another route. For that reason I voted +1. I appreciate that you did some real research and came to a valid conclusion but porting to Cygwin is a tautology here. It can't be used as part of a reason to include anything in the distro. cgf (I fully expect that at some point in this discussion someone will find that this package is actually included in Debian or something, rendering this whole point moot)
Re: [ITP] astrometry.net-0.38-1
On 8 November 2011 07:44, Ken Brown wrote: I was the last person to vote +1, and I have to admit that it was mostly to help the guy out. I regret that I didn't give it more thought. Now that I've seen this discussion, I am no longer in favor of including the package in the distro. I think it would make more sense for the OP to set up his own repository. Much like Ken I wasn't as thorough as I should have been when voting for the package. I like the concept of the package, but in all likelihood I will likely not use it very often. Following the thread it looks like the package may not quite be ready for the distro. Chris -- Chris Sutcliffe http://emergedesktop.org http://www.google.com/profiles/ir0nh34d
Re: [ITP] astrometry.net-0.38-1
On Fri, Nov 4, 2011 at 11:13 PM, Charles Wilson wrote: You should probably do that, to ensure that the build procedure works on your machine. Also, to test the resuts; I have no idea how to use this stuff. It builds fine, and the resulting installation works fine when I put some sky catalogs in /usr/share/astrometry/data/. The question becomes, would it be better to create a separate package (astrometry.net-data-tycho or such) for the (example/test) catalogs, than to have them in the binary/source packages? Theoretically, and I suppose in eventual actuality as well, there could be many different sets of catalogs, so separate packaging sounds like the way to go ... Provided you can rebuild this package on your machine, AND that it actually works, consider it GTG. With the notice/question above, it worked like a charm -- and I thank you dearly! -- jussi
Re: [ITP] astrometry.net-0.38-1
On 11/7/2011 8:18 AM, Jussi Kantola wrote: On Fri, Nov 4, 2011 at 11:13 PM, Charles Wilson wrote: You should probably do that, to ensure that the build procedure works on your machine. Also, to test the resuts; I have no idea how to use this stuff. It builds fine, and the resulting installation works fine when I put some sky catalogs in /usr/share/astrometry/data/. Good news. Please post *your* rebuilt packages somewhere, so they can be uploaded. The question becomes, would it be better to create a separate package (astrometry.net-data-tycho or such) for the (example/test) catalogs, than to have them in the binary/source packages? Theoretically, and I suppose in eventual actuality as well, there could be many different sets of catalogs, so separate packaging sounds like the way to go ... Definitely separate. However, it may be best not to create any catalog packages at all, and instead provide helper scripts (in /usr/lib/astrometry/scripts/ ?) to d/l and install the individual catalogs. The reason for this suggestion is twofold. First, if you create a cygwin package containing the data from catalog foo, then cygwin will be *redistributing* that data. However, many scientific databases of this sort, while free (gratis) to use, prohibit redistribution -- everybody is required to get them directly from the source. So, for this sort of catalog, a helper script to enable the end-user to do THAT is the only solution. Second, these catalogs are HUGE. 70GB? 25GB? That's 10 to 30 times the size of the entire cygwin distribution, source and all -- for one catalog. Our mirrors probably won't accept that. Provided you can rebuild this package on your machine, AND that it actually works, consider it GTG. With the notice/question above, it worked like a charm -- and I thank you dearly! I have to admit, it was pretty painful. The upstream astrometry guys really should work on making their project more cross-platform- and distro-packaging- friendly. E.g. use the autotools including autoconf, automake, and libtool throughout, and treat all of the helper libraries -- gsl-an, libkd, liban*, etc -- as libtool convenience libraries, link all the utils against the top-level backend lib, ... Oh, but we'd lose all the make-it-really-fast fine-tuned CFLAGS settings in our hand-rolled makefiles -- fine, create a FastMakefile that deduces the platform, and invokes (top-level) configure with the appropriate CFLAGS, CPPFLAGS, and LDFLAGS, and tell users to do make -f FastMakefile make ...Premature optimization is the root of all evil. -- Donald Knuth. E.g. make it work, THEN make it work fast. But that's not your problem...nor mine. :-) Go ahead and post your rebuilt packages, and I'll upload them. Postpone the star catalog pkgs and/or helper scripts for the -3 release (*) -- Chuck (*) howto: drop the scripts in the CYGWIN-PATCHES directory, and add lines like the following to the end of the src_install() function in the .cygport file: exeinto /usr/lib/astrometry/scripts doexe ${C}/my-first-script doexe ${C}/my-second-script Since the scripts would probably use wget or curl to d/l the catalog(s), you'd want to update the requires: line in your setup.hint.
Re: [ITP] astrometry.net-0.38-1
On Mon, Nov 7, 2011 at 4:46 PM, Charles Wilson wrote: Good news. Please post *your* rebuilt packages somewhere, so they can be uploaded. wget \ http://rubor.org/astrometrytortilla/astrometry.net/astrometry.net-0.38-2.tar.bz2 \ http://rubor.org/astrometrytortilla/astrometry.net/astrometry.net-0.38-2-src.tar.bz2 \ http://rubor.org/astrometrytortilla/astrometry.net/setup.hint Definitely separate. However, it may be best not to create any catalog packages at all, and instead provide helper scripts (in /usr/lib/astrometry/scripts/ ?) to d/l and install the individual catalogs. The reason for this suggestion is twofold. I like this even more. Yeah, we'll go the dl-script way; for the initial release, there will be no catalogs included, however, I modified the cygwin-specific README with instructions for downloading the previously included tycho*.fits from my own server. With future updates, we can have a fuller index from Tycho at f.e. AstroTortillas site, or then people will just request the indices from astrometry.net, which is the default procedure anyway. I have to admit, it was pretty painful. The upstream astrometry guys really should work on making their project more cross-platform- and distro-packaging- friendly. Yes, while I'm far too happy to have a free astrometrical solver to actually complain, the codebase sure isn't in the best of shapes for porting purposes. I've been down the where's that coming from -route you described in an earlier post :-) Once more, thanks for helping out! -- jussi
Re: [ITP] astrometry.net-0.38-1
On Mon, Nov 07, 2011 at 09:46:21AM -0500, Charles Wilson wrote: On 11/7/2011 8:18 AM, Jussi Kantola wrote: On Fri, Nov 4, 2011 at 11:13 PM, Charles Wilson wrote: You should probably do that, to ensure that the build procedure works on your machine. Also, to test the resuts; I have no idea how to use this stuff. It builds fine, and the resulting installation works fine when I put some sky catalogs in /usr/share/astrometry/data/. Good news. Please post *your* rebuilt packages somewhere, so they can be uploaded. The question becomes, would it be better to create a separate package (astrometry.net-data-tycho or such) for the (example/test) catalogs, than to have them in the binary/source packages? Theoretically, and I suppose in eventual actuality as well, there could be many different sets of catalogs, so separate packaging sounds like the way to go ... Definitely separate. However, it may be best not to create any catalog packages at all, and instead provide helper scripts (in /usr/lib/astrometry/scripts/ ?) to d/l and install the individual catalogs. The reason for this suggestion is twofold. First, if you create a cygwin package containing the data from catalog foo, then cygwin will be *redistributing* that data. However, many scientific databases of this sort, while free (gratis) to use, prohibit redistribution -- everybody is required to get them directly from the source. So, for this sort of catalog, a helper script to enable the end-user to do THAT is the only solution. Second, these catalogs are HUGE. 70GB? 25GB? That's 10 to 30 times the size of the entire cygwin distribution, source and all -- for one catalog. Our mirrors probably won't accept that. I've been trying not to offer an opinion here but it isn't clear to me why so many people voted +1 for this package. It seems like we're adding a huge package to the distribution just to help out a very miniscule user base. Do we really need this package in the Cygwin distribution? cgf
Re: [ITP] astrometry.net-0.38-1
On 11/7/2011 11:17 AM, Christopher Faylor wrote: I've been trying not to offer an opinion here but it isn't clear to me why so many people voted +1 for this package. It seems like we're adding a huge package Meh, if you exclude the star catalogs (and I think we should; and the OP has agreed to avoid), then bin+src weighs in at 25MB (65MB uncompressed), which is pretty large but not unheard of. Most of that is because it has a ton of exe's, and all but one are linked statically to various support libraries. (Oddly all of those libs get dumped together into the DLL, and that dll is used by only one client. But, conceivably, the other exes could also link to that dll, for a big win: from 45MB uncompressed to approx 2.5MB, based on my seat-of-pants calculation). to the distribution just to help out a very miniscule user base. Meh, without casting aspersions, I doubt the user base of our various specialized math tools -- like singular, octave, fftw3, qhull, etc -- are very large in absolute terms. But...we have maintainers, they volunteered and contributed, so here we are. If they go AWOL, then the package gets slapped with _obsolete. Same deal here. Do we really need this package in the Cygwin distribution? Well, not as such, no. We don't really NEED very much of what's currently part of the distro -- but that's never been the justification for package acceptance. Do we need fortune or robots? I think it's kinda cool for cygwin be one of the first (not THE first; it's already in BSD ports IIUC) to provide these tools, so that's why I voted +1. However, you're still (one of the) benevolent(?) dictators-for-life. Are you exercising a veto? If so, we can teach the OP how to set up an add-on setup.exe repository, like cygwin-ports, which he can host over at astronomytortilla or whatever -- so it's not a disaster if you are vetoing. -- Chuck
Re: [ITP] astrometry.net-0.38-1
On Mon, Nov 07, 2011 at 11:49:45PM -0500, Charles Wilson wrote: On 11/7/2011 11:17 AM, Christopher Faylor wrote: I've been trying not to offer an opinion here but it isn't clear to me why so many people voted +1 for this package. It seems like we're adding a huge package Meh, if you exclude the star catalogs (and I think we should; and the OP has agreed to avoid), then bin+src weighs in at 25MB (65MB uncompressed), which is pretty large but not unheard of. Most of that is because it has a ton of exe's, and all but one are linked statically to various support libraries. (Oddly all of those libs get dumped together into the DLL, and that dll is used by only one client. But, conceivably, the other exes could also link to that dll, for a big win: from 45MB uncompressed to approx 2.5MB, based on my seat-of-pants calculation). to the distribution just to help out a very miniscule user base. Meh, without casting aspersions, I doubt the user base of our various specialized math tools -- like singular, octave, fftw3, qhull, etc -- are very large in absolute terms. But...we have maintainers, they volunteered and contributed, so here we are. If they go AWOL, then the package gets slapped with _obsolete. Same deal here. No. It's not, and please avoid the obnoxious Mehs. There is no need for that affectation unless you are purposely trying to offend. Given this reasoning we might as well do away with votes entirely since it doesn't really matter if anyone will use the package as long as someone is willing to support it. The packages that you are talking about are commonly used Linux packages, found in other distros. If we didn't have something like them people would be asking for them. You certainly know this. Do we really need this package in the Cygwin distribution? Well, not as such, no. We don't really NEED very much of what's currently part of the distro -- but that's never been the justification for package acceptance. Do we need fortune or robots? Cygwin is supposed to be like a Linux distro. Including packages which come with Linux distros is a no-brainer. Including a large, specialized package which is not commonly found on Linux and which has a small user base is not a no-brainer. I really can't believe that I have to explain this or that you really think you're making a valid argument. I think it's kinda cool for cygwin be one of the first (not THE first; it's already in BSD ports IIUC) to provide these tools, so that's why I voted +1. ^That is actually the type of answer I was looking for. I wanted to know why people thought the package was needed in the distro. However, you're still (one of the) benevolent(?) dictators-for-life. Are you exercising a veto? If I was doing that you would have seen the word veto in the message. Given the fact that the votes needed to trickle in over the course of more than a month, it seems that most people don't feel very strongly about including this package. I understand that the OP wants to have a convenient way of distributing it to the small number of people who need it but I don't think that is necessarily a good enough reason for the package to occupy disk space and bandwidth on sourceware.org and mirrors, for potential package support to show up in the cygwin mailing list, and for someone to take time to upload updates to sourceware.org. I was wondering if anyone else felt like I did about this package. If it hadn't required many weeks to get approval + more weeks to get packaging worked out, it would have gone in without comment from me. Since it did drag on and required multiple pleading messages to keep things moving, it certainly seems like there isn't a lot of enthusiasm for getting this in. The bottom line is that I was trying to ask why people voted +1. If it was just to help the guy out then that's not a really good reason for the package to be in the distro. If it was because people thought this was kinda cool... then that would imply that there was more thought involved. cgf
Re: [ITP] astrometry.net-0.38-1
On 10/7/2011 12:18 PM, Jussi Kantola wrote: However I had to modify backend-main.c so that the config file (which defines the location of index files) could be read from cygwin's preferred location, /usr/share/astrometry/etc/backend.cfg. That's a little odd, and I don't think that's exactly what was meant by the documentation http://cygwin.com/setup.html#package_contents. I don't think this is a showstopper for this release, but ordinarily configuration files belong in the top-level /etc directory. However, once installed there, a name like backend.cfg is poorly chosen, since it doesn't really indicate what package it belongs to, and thus might conflict with some other package. /usr/share/pkg/ is usually used for other, more extensive data files and support -- e.g. that's probably where your star catalog files (indexes?) ought to go. Furthermore, if backend.cfg is user-editable, then you would probably want to install it into /etc/defaults/etc/backend.cfg and include a postinstall/preremove scripts here: (a) /etc/postinstall/astrometry.net.sh (b) /etc/preremove/astrometry.net.sh whose job is to (a) copy the default version into /etc on install, IF no such file already exists, and (b) remove the /etc/backend.cfg file on uninstall, if and only if it is identical to the /etc/defaults/ one. See /etc/postinstall/tcp_wrappers.sh [or .done] /etc/preremove/tcp_wrappers.sh for and example. FYI, cygport will handle creating these scripts for you (almost) automatically, via a single command: make_etc_defaults /etc/backend.cfg All this complexity is so that user-modified configuration settings don't get clobbered by package updates, but non-modified files will get updated. However, what is probably a showstopper are various lines like this in the build: # Try building and installing python spherematch module make install-spherematch make[2]: Entering directory `/usr/src/packages/tmp/astrometry.net-0.38-1/libkd' python setup.py build --force --build-base build --build-platlib build/lib running build running build_py creating build creating build/lib copying spherematch.py - build/lib running build_ext building 'spherematch_c' extension creating build/temp.cygwin-1.7.9-i686-2.6 gcc -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/lib/python2.6/site-packages/numpy/core/include/numpy -I../qfits-an/include -I../util -I. -I/usr/include/python2.6 -c pyspherematch.c -o build/temp.cygwin-1.7.9-i686-2.6/pyspherematch.o gcc -shared -Wl,--enable-auto-image-base build/temp.cygwin-1.7.9-i686-2.6/pyspherematch.o libkd.a ../util/libanfiles.a ../util/libanutils.a ../qfits-an/lib/libqfits.a -L/usr/lib/python2.6/config -lpython2.6 -o build/lib/spherematch_c.dll cp build/lib/spherematch_c.so spherematch_c.so cp: cannot stat `build/lib/spherematch_c.so': No such file or directory make[2]: *** [spherematch_c.so] Error 1 IOW, cygwin python's distutils is _doing the right thing_ -- creating the plugin with the name spherematch_c.dll -- but the Makefile in astrometry.net thinks the plugin will always be named spherematch_c.so and reports an error when it tries to install the latter file. Also, when I did 'make install ...' my lib/astrometry/bin directory was empty. I can't help but think this will cause problems for your users... Now, this bit is interesting: - mkdir -p $(INSTALL_DIR)/python/astrometry/libkd + mkdir -p $(INSTALL_DIR)/share/astrometry/python/astrometry/libkd Normally, python extensions go under the real python dir: /usr/lib/python2.6/site-packages/pkgname/... But...this whole build system doesn't really support that; it hardcodes the destination path and then adds that path to sys.path via the application .py files; when it really should use some mechanism to find the entry in $python's sys.path list which contains site-packages. So...accepting that, the python stuff should ALSO go under /lib rather than /share, because (a) the .pyc files are platform specific, and (b) the .dll's which ought to go there are also platform specific. so, not GTG. Also, you missed Corinna's statement: If the binaries are using it (the libbackend library), they should also be linked against [the DLL], rather than being linked statically. Your build still links against libbackend.a, rather than cygbackend.dll. I'm trying to massage your -src package to DTRT. Stay tuned. -- Chuck
Re: [ITP] astrometry.net-0.38-1
On Fri, Nov 4, 2011 at 8:29 AM, Charles Wilson wrote: I don't think this is a showstopper for this release, but ordinarily configuration files belong in the top-level /etc directory. However, once installed there, a name like backend.cfg is poorly chosen, since it doesn't really indicate what package it belongs to, and thus might conflict with some other package. Which is precisely the reason I've tried to keep everything under paths that have astrometry in them; as has already been pointed out (and as you point out below), a.n isn't really (at this point) designed for portability and/or 3rd party vendor distribution. I actually shuddered when I realized there would be a cygbackend.dll, as it sounds ALL wrong :-) /usr/share/pkg/ is usually used for other, more extensive data files and support -- e.g. that's probably where your star catalog files (indexes?) ought to go. Yes; indexes = star catalogs. Also, when I did 'make install ...' my lib/astrometry/bin directory was empty. That's odd, I was building and installing it all night last night and the folder was being updated duly. Looking at the source package, the Makefile seems fine (at least on that regard). so, not GTG. Also, you missed Corinna's statement: If the binaries are using it (the libbackend library), they should also be linked against [the DLL], rather than being linked statically. Oops, and roger. I'm trying to massage your -src package to DTRT. Stay tuned. Thank you very much and sorry for the trouble I'm causing! I'll have another session with the package myself, paying attention to what you've said. Also, I guess it's about time I get in contact with an active a.n developer to get clarification on some suspicions I've had ... but thanks again! -- jussi
Re: [ITP] astrometry.net-0.38-1
On Fri, 2011-11-04 at 02:29 -0400, Charles Wilson wrote: On 10/7/2011 12:18 PM, Jussi Kantola wrote: However I had to modify backend-main.c so that the config file (which defines the location of index files) could be read from cygwin's preferred location, /usr/share/astrometry/etc/backend.cfg. That's a little odd, and I don't think that's exactly what was meant by the documentation http://cygwin.com/setup.html#package_contents. I don't think this is a showstopper for this release, but ordinarily configuration files belong in the top-level /etc directory. However, once installed there, a name like backend.cfg is poorly chosen, since it doesn't really indicate what package it belongs to, and thus might conflict with some other package. Then this should be /etc/astrometry/backend.cfg instead. IOW, cygwin python's distutils is _doing the right thing_ -- creating the plugin with the name spherematch_c.dll -- but the Makefile in astrometry.net thinks the plugin will always be named spherematch_c.so and reports an error when it tries to install the latter file. So patch the Makefile.in. Now, this bit is interesting: - mkdir -p $(INSTALL_DIR)/python/astrometry/libkd + mkdir -p $(INSTALL_DIR)/share/astrometry/python/astrometry/libkd Normally, python extensions go under the real python dir: /usr/lib/python2.6/site-packages/pkgname/... But...this whole build system doesn't really support that; it hardcodes the destination path and then adds that path to sys.path via the application .py files; when it really should use some mechanism to find the entry in $python's sys.path list which contains site-packages. So...accepting that, the python stuff should ALSO go under /lib rather than /share, because (a) the .pyc files are platform specific, and (b) the .dll's which ought to go there are also platform specific. I'm not so sure about (a), but my Linux VMs already use Python 2.7 so I couldn't check. That aside, I have no problem with application-specific pure-Python code in /usr/share/$PN, as they serve no purpose outside the given application, and many packages do this, in fact. Obviously DLL extensions can't go there though. so, not GTG. Also, you missed Corinna's statement: If the binaries are using it (the libbackend library), they should also be linked against [the DLL], rather than being linked statically. Your build still links against libbackend.a, rather than cygbackend.dll. I'm trying to massage your -src package to DTRT. Stay tuned. Good luck, it sounds like this software is quite unpolished. No wonder it's not in the distros yet. Yaakov
Re: [ITP] astrometry.net-0.38-1
On 11/4/2011 11:11 AM, Yaakov (Cygwin/X) wrote: software is quite unpolished. From reading the web page, it appears to be a research project by a couple of grad students -- with goals of supporting amateur and even professional astronomers by automating what is currently a labor-intensive task. So...like all research projects, it appears to suffer from get-it-done-itis focused on Linux and/or BSD, with little attention paid to portability. That's...problematic when going to any flavor of win32, including cygwin. I mean really: hand-editing makefiles? no configure scripts (some subdirs have them, but they are invoked as part of the top-level make, which doesn't). IMO the whole thing needs to be autotoolized, but I'm not going there. OTOH, I just spent a couple hours adding $(EXE) to about a thousand makefile rules, only to watch it blow up because of some built-in rule I can't turn off (where is it COMING from!!!??) that adds foo.exe.o to the dependencies and then looks for foo.exe.c to build it...Aaarrrggh -- Chuck
Re: [ITP] astrometry.net-0.38-1
On 11/4/2011 2:29 AM, Charles Wilson wrote: Your build still links against libbackend.a, rather than cygbackend.dll. I'm trying to massage your -src package to DTRT. Stay tuned. I've posted a revised version of your package here: http://cygwin.cwilson.fastmail.fm/astrometry.net-0.38-2-src.tar.bz2 http://cygwin.cwilson.fastmail.fm/astrometry.net-0.38-2.tar.bz2 Inside the -src tarball is the *original* upstream sources (astrometry.net-0.38.tar.bz2), a few patches, and a .cygport script. to rebuild, unpack the -src and do: $ cygport astrometry.net-0.38-2.cygport all You should probably do that, to ensure that the build procedure works on your machine. Also, to test the resuts; I have no idea how to use this stuff. Finally: the backend.exe program is linked to cygbackend.dll, which are both in /usr/lib/astrometry/bin/. All the python stuff, including three extension DLLs, are in /usr/lib/astrometry/python/*/ I modified the build procedure for cygbackend.dll -- it now generates an import lib, and it also (re)exports all of the symbols from the other (sub)libraries. That means when linking backend.exe, you don't need to list those other dependencies, because cygbackend.dll will satisfy them all. (*) Provided you can rebuild this package on your machine, AND that it actually works, consider it GTG. (*) I did not do this, but because cygbackend now exports all the symbols from (e.g.) libanfiles.a, libutils.a, libkd.a, etc, you COULD modify the link commands for almost all of the /other/ .exe's in the blind/ directory, which currently link against (many) of those static libraries, to instead link solely against cygbackend.dll (actually, libbackend.dll.a). That would make the entire package MUCH smaller. But it's a lot of work. -- Chuck
Re: [ITP] astrometry.net-0.38-1
On Oct 30 23:10, Jussi Kantola wrote: Bump. The package has the 5 votes required for inclusion of a new package, still missing the GTG. If someone knows the package is not GTG, can I please get directions about what's wrong. The project that sort of needs astrometry.net in the Cygwin repos is close to its first publicized release (in fact, we're just waiting for the package approval); check it out at http://sourceforge.net/p/astrotortilla/ Meanwhile, wget \ http://rubor.org/astrometrytortilla/astrometry.net/astrometry.net-0.38-1.tar.bz2 \ http://rubor.org/astrometrytortilla/astrometry.net/astrometry.net-0.38-1-src.tar.bz2 \ http://rubor.org/astrometrytortilla/astrometry.net/setup.hint I hoped that somebody who voted on the package would do the final GTG, but in vain it seems. Anyway, I just had a look. The packaging now looks basically good. One issue I still have with the package is the big number of non-standard binaries in /usr/bin. Is it really necessary to have all the binaries as user-accessible binaries in /usr/bin? Or are many of the binaries just called from another (or other) binaries which serve as the primary UI? What also bugs me are the generic names of the binaries. Plotstuff, merge-index, tablist, tabsort, checktree, ... This all sounds not much like astronomer stuff. So, I would prefer to split the binaries into two groups: - UI binary (or binaries) into /usr/bin - Helper binaries into /usr/share/astrometry/bin There's also still the issue with usr/lib/astrometry/libbackend.so. If that's really a shared lib, it should be called cygbackend.dll and should reside in /usr/bin. If the binaries are using it, they should also be linked against it, rather than being linked statically. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Project Co-Leader cygwin AT cygwin DOT com Red Hat
Re: [ITP] astrometry.net-0.38-1
On 11/3/2011 8:02 AM, Corinna Vinschen wrote: I hoped that somebody who voted on the package would do the final GTG, but in vain it seems. Sorry...I had intended to, but got swamped with other stuff. :-( Anyway, I just had a look. The packaging now looks basically good. One issue I still have with the package is the big number of non-standard binaries in /usr/bin. Is it really necessary to have all the binaries as user-accessible binaries in /usr/bin? Or are many of the binaries just called from another (or other) binaries which serve as the primary UI? What also bugs me are the generic names of the binaries. Plotstuff, merge-index, tablist, tabsort, checktree, ... This all sounds not much like astronomer stuff. So, I would prefer to split the binaries into two groups: - UI binary (or binaries) into /usr/bin - Helper binaries into /usr/share/astrometry/bin FHS says that architecture-dependent files should go under /usr/lib/ not /usr/share...so /usr/lib/astrometry/bin -- Chuck
Re: [ITP] astrometry.net-0.38-1
On Nov 3 08:55, Charles Wilson wrote: On 11/3/2011 8:02 AM, Corinna Vinschen wrote: I hoped that somebody who voted on the package would do the final GTG, but in vain it seems. Sorry...I had intended to, but got swamped with other stuff. :-( Anyway, I just had a look. The packaging now looks basically good. One issue I still have with the package is the big number of non-standard binaries in /usr/bin. Is it really necessary to have all the binaries as user-accessible binaries in /usr/bin? Or are many of the binaries just called from another (or other) binaries which serve as the primary UI? What also bugs me are the generic names of the binaries. Plotstuff, merge-index, tablist, tabsort, checktree, ... This all sounds not much like astronomer stuff. So, I would prefer to split the binaries into two groups: - UI binary (or binaries) into /usr/bin - Helper binaries into /usr/share/astrometry/bin FHS says that architecture-dependent files should go under /usr/lib/ not /usr/share...so /usr/lib/astrometry/bin Uh, yes, indeed. Thanks for spotting that. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Project Co-Leader cygwin AT cygwin DOT com Red Hat
Re: [ITP] astrometry.net-0.38-1
On Thu, Nov 3, 2011 at 2:02 PM, Corinna Vinschen wrote: Is it really necessary to have all the binaries as user-accessible binaries in /usr/bin? Or are many of the binaries just called from another (or other) binaries which serve as the primary UI? The latter was true. I updated the packages as per your instructions: the actual binaries are now in /usr/lib/astrometry/bin, whereas two user interface commands are PATH-setting shell scripts, /usr/bin/solve-field and /usr/bin/wcsinfo. Solve-field is the actual astrometry.net solver, whereas wcsinfo is used to extract World Coordinate System data from the output generated by solve-field. There may be more commands that should be available in the default PATH; I'm sure the astrometry.net community will notify me if this is the case. There's also still the issue with usr/lib/astrometry/libbackend.so. If that's really a shared lib, it should be called cygbackend.dll and should reside in /usr/bin. If the binaries are using it, they should also be linked against it, rather than being linked statically. I dug a little deeper into it, and to my best knowledge the .so was being used only by two Python-scripts, which weren't even installed by the Makefile. So I'm assuming it's either a WIP or obsolete code. I modified the package source (and Makefile) to reflect Cygwin's naming scheme, however, the shared lib is _not_ installed anymore (neither in the binary package, nor by the Makefile in the source package). I haven't noticed anything breaking up due to this. Package filenames are the same, but the contents have been updated. wget \ http://rubor.org/astrometrytortilla/astrometry.net/astrometry.net-0.38-1.tar.bz2 \ http://rubor.org/astrometrytortilla/astrometry.net/astrometry.net-0.38-1-src.tar.bz2 \ http://rubor.org/astrometrytortilla/astrometry.net/setup.hint -- jussi
Re: [ITP] astrometry.net-0.38-1
Bump. The package has the 5 votes required for inclusion of a new package, still missing the GTG. If someone knows the package is not GTG, can I please get directions about what's wrong. The project that sort of needs astrometry.net in the Cygwin repos is close to its first publicized release (in fact, we're just waiting for the package approval); check it out at http://sourceforge.net/p/astrotortilla/ Meanwhile, wget \ http://rubor.org/astrometrytortilla/astrometry.net/astrometry.net-0.38-1.tar.bz2 \ http://rubor.org/astrometrytortilla/astrometry.net/astrometry.net-0.38-1-src.tar.bz2 \ http://rubor.org/astrometrytortilla/astrometry.net/setup.hint -- jussi
Re: [ITP] astrometry.net-0.38-1
Bump, if only to show that I'm serious about maintaining the package. On Tue, Oct 11, 2011 at 1:19 PM, Marco Atzeri wrote: +1 . Same comment about missing package procedure Thanks for the vote. I'm sorry but if there's still an issue about packaging, or procedure, I need it pointed out. AFAIK the package is OK now. This upstream package is not really thought for easy porting, it inglobes a lot of library including gsl-1.14 while should be better to link with cygwin gsl. Agreed. If this is a showstopper, I can check out if it works OK with the stock GSL. To the best of my knowledge astrometry.net doesn't actually change anything in GSL, they just omit what's not required. +3 at the moment ... -- jussi
Re: [ITP] astrometry.net-0.38-1
On 18 October 2011 07:21, Jussi Kantola wrote: +3 at the moment ... +1 Chris -- Chris Sutcliffe http://emergedesktop.org http://www.google.com/profiles/ir0nh34d
Re: [ITP] astrometry.net-0.38-1
On 10/5/2011 6:01 PM, Charles Wilson wrote: On 10/4/2011 4:02 AM, Jussi Kantola wrote: category: Science requires: cairo libcairo2 python-cairo libnetpbm10 netpbm libpng14 libjpeg8 zlib zlib0 python python-numpy pkg-config cygwin sdesc:Astrometry.net astrometrical solver. ldesc:Astrometry.net analyses an astronomical image (photograph of the night sky) to output, among other things, the central coordinate of the image, the visible field of view, and the list of deep sky objects potentially visible within the field. +1. However, as Corinna pointed out, there are some issues with the packaging. I'll take a look this weekend and come up with some suggestions for you. FYI, you probably don't need to list both zlib and zlib0 as requires: (most packages need just the dll, in zlib0). Ditto for cairo and libcairo2 -- you probably only need libcairo2. Also, it is no longer recommended to include 'cygwin' in the requires: list -- it is assumed that cygwin is a requirement for everything. -- Chuck +1 . Same comment about missing package procedure This upstream package is not really thought for easy porting, it inglobes a lot of library including gsl-1.14 while should be better to link with cygwin gsl. Regards Marco
Re: [ITP] astrometry.net-0.38-1
Bumping up (is this cool, or am I being too spammy?) -- jussi
Re: [ITP] astrometry.net-0.38-1
On Mon, Oct 10, 2011 at 09:54:18PM +0300, Jussi Kantola wrote: Bumping up (is this cool, or am I being too spammy?) You are being spammy. cgf
Re: [ITP] astrometry.net-0.38-1
On Fri, Oct 7, 2011 at 5:20 PM, Marco Atzeri wrote: Hi Jussi, could you clarify the usage/scope of usr/lib/astrometry/libbackend.so ? I'm not a programmer of astrometry.net, and don't know how it works internally (except that it's looking for stars in the image to be solved, then constructs quads from the stars and matches the quads to an index). However I had to modify backend-main.c so that the config file (which defines the location of index files) could be read from cygwin's preferred location, /usr/share/astrometry/etc/backend.cfg. From backend-main.c: /** * Accepts an augmented xylist that describes a field or set of fields to solve. * Reads a config file to find local indices, and merges information about the * indices with the job description to create an input file for 'blind'. Runs blind * and merges the results. */ Not supposing everyone's an amateur astronomer, please allow me to decipher: field = field of view, starfield, the area of the sky that's contained in the image that is to be solved; solving = figuring out the sky coordinates of the field indices = files containing the known coordinates of celestial objects (esp. stars) blind = blind solver/solving, ie. no other fore-knowledge of the contained field is required for what I see, none of the 77 exe you have in usr/bin is using it. At least solve-field.exe (and/or the tools called by solve-field) uses it, as it was complaining about missing files before I did the modification. I don't know enough about the linking of cygwin software to comment why it's not showing in f.e. ldd /usr/bin/solve-field.exe. -- jussi
Re: [ITP] astrometry.net-0.38-1
Sorry for my linebreaking, gmail has been messing it up lately :/ -- jussi
Re: [ITP] astrometry.net-0.38-1
I've updated the package as per instructions from this thread (fixed the paths, dropped the unnecessary libs). Hopefully I got it right this time -- I've found a new respect for package maintainers of all my favorite distros, this stuff requires more concentration and care than actually programming the contents! ;) As a side note, I occasionally have to run the program (solve-field.exe) twice, as on the first attempt it dies complaining of a slowly loading DLL (_functools). Is this normal? I'm doing this in a Win7 virtual machine... wget \ http://rubor.org/astrometrytortilla/astrometry.net/astrometry.net-0.38-1.tar.bz2 \ http://rubor.org/astrometrytortilla/astrometry.net/astrometry.net-0.38-1-src.tar.bz2 \ http://rubor.org/astrometrytortilla/astrometry.net/setup.hint -- jussi
Re: [ITP] astrometry.net-0.38-1
On 10/4/2011 1:02 AM, Jussi Kantola wrote: snip It's a package that our project (http://sourceforge.net/projects/astromate/) would benefit immensely from, as the routine of installing cygwin, selecting (all of the) the required packages and compiling astrometry.net is beyond the skills of the target audience (amateur astronomers). Not included in any of the major distros. snip I'm curious, partly for myself. Would it not be a fairly simple alternative for your users if you distributed a binary package and told them that installing Cygwin is a prereq for running your binary (assuming it is)? List, is there a reason this should not be done? Thanks, Peter
Re: [ITP] astrometry.net-0.38-1
I'm curious, partly for myself. Would it not be a fairly simple alternative for your users if you distributed a binary package and told them that installing Cygwin is a prereq for running your binary (assuming it is)? List, is there a reason this should not be done? That's how it'll be done if the package doesn't get the required votes. In this case the present packaging would be fine, ie. it would go under /usr/local. The downside of this would be that users will have to satisfy the dependencies by theirselves -- not a big trouble when compared with having to compile the software. Got a good D'OH moment out of Corinnas remark -- will fix the packaging ASAP. -- jussi
Re: [ITP] astrometry.net-0.38-1
On 10/4/2011 4:02 AM, Jussi Kantola wrote: category: Science requires: cairo libcairo2 python-cairo libnetpbm10 netpbm libpng14 libjpeg8 zlib zlib0 python python-numpy pkg-config cygwin sdesc:Astrometry.net astrometrical solver. ldesc:Astrometry.net analyses an astronomical image (photograph of the night sky) to output, among other things, the central coordinate of the image, the visible field of view, and the list of deep sky objects potentially visible within the field. +1. However, as Corinna pointed out, there are some issues with the packaging. I'll take a look this weekend and come up with some suggestions for you. FYI, you probably don't need to list both zlib and zlib0 as requires: (most packages need just the dll, in zlib0). Ditto for cairo and libcairo2 -- you probably only need libcairo2. Also, it is no longer recommended to include 'cygwin' in the requires: list -- it is assumed that cygwin is a requirement for everything. -- Chuck
Re: [ITP] astrometry.net-0.38-1
+1
Re: [ITP] astrometry.net-0.38-1
Hi Jussi, On Oct 4 11:02, Jussi Kantola wrote: Hi folks -- I'm trying this for the second and final time, just in case this got ignored earlier because of formal mistakes in the submission, or holidays, or ... It's a package that our project (http://sourceforge.net/projects/astromate/) would benefit immensely from, as the routine of installing cygwin, selecting (all of the) the required packages and compiling astrometry.net is beyond the skills of the target audience (amateur astronomers). Not included in any of the major distros. Included is a small subset of the freely available Tycho-2 star catalogue, which can be used for testing the installation: cd /usr/local/astrometry/examples; ../bin/solve-field.exe apod4.jpg For a comprehensive sky catalogue, see http://trac.astrometry.net/browser/trunk/src/astrometry/GETTING-INDICES category: Science requires: cairo libcairo2 python-cairo libnetpbm10 netpbm libpng14 libjpeg8 zlib zlib0 python python-numpy pkg-config cygwin sdesc:Astrometry.net astrometrical solver. ldesc:Astrometry.net analyses an astronomical image (photograph of the night sky) to output, among other things, the central coordinate of the image, the visible field of view, and the list of deep sky objects potentially visible within the field. wget \ http://rubor.org/astrometrytortilla/astrometry.net/astrometry.net-0.38-1.tar.bz2 \ http://rubor.org/astrometrytortilla/astrometry.net/astrometry.net-0.38-1-src.tar.bz2 \ http://rubor.org/astrometrytortilla/astrometry.net/setup.hint first of all, you need 5 votes from other package maintainers to get the package into the distro, since it's not in any major Linux distro. Apart from that, your packaging is wrong. You've put everything under /usr/local/astrometry, but for an official distro package, you should follow the usual directory layout as outlined in http://cygwin.com/setup.html#package_contents Nothing ever in the distro should go into /usr/local. That dir is reserved for *local* stuff on the user's machine. UI binaries under /usr/bin, include files under /usr/include/astrometry, helper stuff under /usr/share/astrometry, etc. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Project Co-Leader cygwin AT cygwin DOT com Red Hat