RE: Setup ready #2

[Robert Collins]

> -Original Message-
> From: S. Cowles [mailto:[EMAIL PROTECTED]] 
> Sent: Saturday, March 16, 2002 3:39 AM
> To: CygWin-Apps
> Subject: Re: Setup ready #2
> 
> 
> 
> When using the setup-snapshot setup-20020315.exe for "install 
> from local directory", I get multiple "Can't open (null) for 
> reading" errors. Setup then aborts without going through the 
> other packages in the queue for installation.  I found no 
> installation messages in setup.log.full after the abort.  I'd 
> be happy to provide whatever info might be helpful if this is 
> worth debugging and not pilot error.

Can you reproduce this? If so, then please do a run, and email me:
1) an ls -lR of your local dir.
2) The content of all the setup.ini files in the local dir (ie tar cjf
setupfiles.tar.bz2 `find -name 'setup.ini'`)
3) The two log files created in /var/setup/

Thanks,
Rob

RE: Setup ready #2

[Robert Collins]

> -Original Message-
> From: Pavel Tsekov [mailto:[EMAIL PROTECTED]] 
> Sent: Saturday, March 16, 2002 2:50 AM
> To: Robert Collins
> Cc: CygWin-Apps
> Subject: Re: Setup ready #2
> 
> 
> Friday, March 15, 2002, 2:01:22 PM, you wrote:
> 
> RC> Ok, Setup200202 now
> 
> RC> * and the new bin, src tickbox arrangement is in place.
> 
> Ok, another thing... I don't really know if this is a bug or 
> feature, but thought it may be worth to share...
> 
> All the setup-200202 versions I have locally - that is 
> 20020225, 20020315 and a custom build from the end of 
> February, won't write a /etc/setup/package_name-src.lst.gz 
> when installing the source for a package. 

Correct. Tracking source packages was deprecated and removed some time
back. I don't think the current released setup tracks it either. 

There were some long discussions (with a lot of rabbit holes) about
source package management around the time when this happened.

Rob

Re: RFP: UPX (Was Re: reducing binary distribution size with UPX)

[Christopher Faylor]

On Fri, Mar 15, 2002 at 04:42:27PM +0300, egor duda wrote:
>Hi!
>
>Friday, 15 March, 2002 Robert Collins [EMAIL PROTECTED] wrote:
>
>RC> I vote for including UPX... and Lapo makes two. Do we need a third? And
>RC> are there any objections?
>
>Does anybody ever tried to measure if upx impose any performance
>penalties? If i understand things correctly, upx compress executable
>file and attach a small "decompressor" stub to it. Then, when
>executable starts, this stub decompresses original executable image.
>This will totally defeat the features that most modern OSes have,
>mapping pages from executable and loading them on demand, sharing
>common read-only pages between different instances of one application,
>etc. I really don't understand what's the point in saving disk storage
>worth several cents (1byte == $1e-7), while increasing memory
>footprint and reducing speed. Hey, just read upx docs, they contain
>all these points already.
>
>Not that i'm against inclusion of upx to cygwin distro -- it's a
>normal package like many others after all, but i really don't
>understand why somebody would want to use such a program.

Excellent points.  This is, IMO, an argument against using upx for
all (any?) cygwin binaries.

cgf

Re: reducing binary distribution size with UPX

[Christopher Faylor]

On Fri, Mar 15, 2002 at 01:48:13PM +0100, Lapo Luchini wrote:
>> Uhmm, UPX should be part of the distribution first, as a maintained
>> pacakge, before folk start packing distributed binaries with it. Do we
>> have a volunteer to maintain UPX?
>
>UPX is quite cross-platform: you can use win32 version to package lonux
>a.out such as linux verison to package win32 PE.
>Moreover an UPX-compressed EXE is completely self-sufficient from UPX
>itself, has no memory overhead and decompresses very very fast (10Mb/sec
>on the author's Pentium133 as upx.sourceforge.net says).
>But if a cygwin native version is needed nonetheless I could volunteer to
>package it.

I think this is a useful addition to the cygwin packages but I don't see
why it should be a requirement that it be available as a package before
people start using it.

It sounds from your description like I'll be able to run this on linux,
where I make all of my packages.  Is that right?  That's the only way
this will be useful for me.

cgf

Re: Setup ready #2

[S. Cowles]

When using the setup-snapshot setup-20020315.exe for "install from local
directory", I get multiple "Can't open (null) for reading" errors. Setup
then aborts without going through the other packages in the queue for
installation.  I found no installation messages in setup.log.full after
the abort.  I'd be happy to provide whatever info might be helpful if
this is worth debugging and not pilot error.

[EMAIL PROTECTED]

Re: Setup ready #2

[Pavel Tsekov]

Friday, March 15, 2002, 2:01:22 PM, you wrote:

RC> Ok, Setup200202 now

RC> * and the new bin, src tickbox arrangement is in place.

Ok, another thing... I don't really know if this is a bug or feature,
but thought it may be worth to share...

All the setup-200202 versions I have locally - that is 20020225,
20020315 and a custom build from the end of February, won't write a
/etc/setup/package_name-src.lst.gz when installing the source for a
package. It doesn't matter if you install source only, source after
the package is installed or source and binary at the same time.

Re: RFP: UPX (Was Re: reducing binary distribution size with UPX)

[Roger]

On Fri, 2002-03-15 at 08:42, egor duda wrote:
> Hi!
> 
> Friday, 15 March, 2002 Robert Collins [EMAIL PROTECTED] wrote:
> 
> RC> I vote for including UPX... and Lapo makes two. Do we need a third? And
> RC> are there any objections?
> 
> Does anybody ever tried to measure if upx impose any performance
> penalties? If i understand things correctly, upx compress executable
> file and attach a small "decompressor" stub to it. Then, when
> executable starts, this stub decompresses original executable image.
> This will totally defeat the features that most modern OSes have,
> mapping pages from executable and loading them on demand, sharing
> common read-only pages between different instances of one application,
> etc. I really don't understand what's the point in saving disk storage
> worth several cents (1byte == $1e-7), while increasing memory
> footprint and reducing speed. Hey, just read upx docs, they contain
> all these points already.
> 
> Not that i'm against inclusion of upx to cygwin distro -- it's a
> normal package like many others after all, but i really don't
> understand why somebody would want to use such a program.
> 
> Egor.mailto:[EMAIL PROTECTED] ICQ 5165414 FidoNet 2:5020/496.19
> 

i tend to agree. keep it basic. keep it simple.  you could probabely
spend a life-time just trying to shrink size, etc.

one might see a diff if all files were compressed with upx, but as egor
mentioned, this would probabely seriously hinder system performance.

not too metion, the binary versions are still i386 only.  i already see
a big difference just recompiling for i686 platform, but how many users
really re-compile cygwin for usage?
-- 
Roger
-
Verify my pgp/gnupg signature on my HomePage:
http://www.alltel.net/~rogerx/about/index.html
l



signature.asc
Description: This is a digitally signed message part

Re[4]: Setup ready #2

[Pavel Tsekov]

Just to confirm that the latest snapshot fixes both of the issues I've
reported. Thanks, Rob! :)

RC> Yup.. thanks very much for noticing this. It's fixed, and a new snapshot
RC> uploaded (same name though).

np :)

Re: RFP: UPX

[Lapo Luchini]

> This will totally defeat the features that most modern OSes have,
> mapping pages from executable and loading them on demand, sharing
> common read-only pages between different instances of one application,
> etc.

This can be a stron objection for exes that ofter are loaded more than once
(apache and the like).
Just for the records Egor is referencing to this: (taken from upx.html of
upx120w.zip)

NOTES FOR WIN32/PE

The PE support in UPX is quite stable now, but definitely there are still some
incompabilities with some files.

Because of the way UPX (and other packers for this format) works, you can see
increased memory usage of your compressed files. If you start several instances
of huge compressed programs you're wasting memory because the common segements
of the program won't get shared across the instances. On the other hand if
you're compressing only smaller programs, or running only one instance of
larger programs, then this penalty is smaller, but it's still there.

If you're running executables from network, then compressed programs will load
faster, and require less bandwidth during execution.

--
Lapo 'Raist' Luchini
[EMAIL PROTECTED] (PGP & X.509 keys available)
http://www.lapo.it (ICQ UIN: 529796)

RE: Re[2]: Setup ready #2

[Robert Collins]

Oh, and BTW it was a interaction between me and the String class, not
the iostream class at all - I was writing one byte too many.

Rob

RE: Re[2]: Setup ready #2

[Robert Collins]

Yup.. thanks very much for noticing this. It's fixed, and a new snapshot
uploaded (same name though).

Rob

> -Original Message-
> From: Pavel Tsekov [mailto:[EMAIL PROTECTED]] 
> Sent: Saturday, March 16, 2002 1:28 AM
> To: Robert Collins
> Cc: CygWin-Apps
> Subject: Re[2]: Setup ready #2
> 
> 
> Got it ? I sent it to the mailing list right after the first mail.
> 
> RC> I don't see any files
> 
> 

Re[2]: Setup ready #2

[Pavel Tsekov]

Got it ? I sent it to the mailing list right after the first mail.

RC> I don't see any files

RE: Setup ready #2

[Robert Collins]

> -Original Message-
> From: Pavel Tsekov [mailto:[EMAIL PROTECTED]] 
> Sent: Saturday, March 16, 2002 1:11 AM
> To: Robert Collins
> Cc: CygWin-Apps
> Subject: Re: Setup ready #2
> 
> 
> Hello Robert,
> 
> Friday, March 15, 2002, 2:01:22 PM, you wrote:
> 
> Well, let me report another issue... The new setup.exe (codename
> setup-200202) seems to break /etc/setup/package_name.lst.gz 
> files by adding an extra null terminator character after the 
> newline character.
> 
> For more info see attached files. The one with the extension 
> .NEW is produced from setup-20020315.exe just some minutes ago.

I don't see any files

Fwd: Re: Setup ready #2

[Pavel Tsekov]

Ooops... here are the attachments


make.lst.gz.NEW
Description: Binary data


make.lst.gz
Description: GNU Zip compressed data

Re: Setup ready #2

[Pavel Tsekov]

Hello Robert,

Friday, March 15, 2002, 2:01:22 PM, you wrote:

Well, let me report another issue... The new setup.exe (codename
setup-200202) seems to break /etc/setup/package_name.lst.gz files
by adding an extra null terminator character after the newline
character.

For more info see attached files. The one with the extension .NEW is
produced from setup-20020315.exe just some minutes ago.

I wild guess is that this is caused by the new String class.

This "bug" breaks the uninstall functionality. Packed is removed from
installed.db but package contents are not removed at all.

void
packagemeta::uninstall ()
{
  if (installed)
{
  /* this will need to be pushed down to the version, or even the source level
   * to allow differences between formats to be seamlessly managed
   * but for now: here is ok
   */
  hash dirs;
  String line = installed->getfirstfile ();

  try_run_script ("/etc/preremove/", name);
  while (line.size())
{

I think we got here for the first time only and then when the zero char is
read we are fd :(

I'm sorry for being such a pain in the ass :(

RC> Ok, Setup200202 now

RE: RFP: UPX (Was Re: reducing binary distribution size with UPX)

[Robert Collins]

> -Original Message-
> From: egor duda [mailto:[EMAIL PROTECTED]] 
> Sent: Saturday, March 16, 2002 12:42 AM
> To: Robert Collins
> 
> Not that i'm against inclusion of upx to cygwin distro -- 
> it's a normal package like many others after all, but i 
> really don't understand why somebody would want to use such a program.

Very good points. There are some cases where it is definitely useful.
I'd certainly do some performance testing before using on often called
software - ie ash. But for something like setup.exe itself, download
time reductions would be well worth a couple of extra page faults at
runtime. As for mapping the image against the disk image or the
pagefile, I think it's back to case by case testing. Certainly I can
imagine that with a compressed exe on a NTFS partition, or a FAT16
compressed volume, that making the least random access hits back against
the .exe is more efficient.

Rob

Re: RFP: UPX (Was Re: reducing binary distribution size with UPX)

[egor duda]

Hi!

Friday, 15 March, 2002 Robert Collins [EMAIL PROTECTED] wrote:

RC> I vote for including UPX... and Lapo makes two. Do we need a third? And
RC> are there any objections?

Does anybody ever tried to measure if upx impose any performance
penalties? If i understand things correctly, upx compress executable
file and attach a small "decompressor" stub to it. Then, when
executable starts, this stub decompresses original executable image.
This will totally defeat the features that most modern OSes have,
mapping pages from executable and loading them on demand, sharing
common read-only pages between different instances of one application,
etc. I really don't understand what's the point in saving disk storage
worth several cents (1byte == $1e-7), while increasing memory
footprint and reducing speed. Hey, just read upx docs, they contain
all these points already.

Not that i'm against inclusion of upx to cygwin distro -- it's a
normal package like many others after all, but i really don't
understand why somebody would want to use such a program.

Egor.mailto:[EMAIL PROTECTED] ICQ 5165414 FidoNet 2:5020/496.19

RE: RFP: UPX (Was Re: reducing binary distribution size with UPX)

[Robert Collins]

> -Original Message-
> From: Earnie Boyd [mailto:[EMAIL PROTECTED]] 
> Sent: Saturday, March 16, 2002 12:28 AM

> 
> Does UPX come with an API library that you can just use in setup?

That's a nice idea, and on a related note I'm considering compressing
setup.exe with UPX once it's a package. If UPX doesn't have a good API,
I'm sure we can wrap it if need be.

Rob

Re: RFP: UPX (Was Re: reducing binary distribution size with UPX)

[Earnie Boyd]

Robert Collins wrote:
> 
> > -Original Message-
> > From: Lapo Luchini [mailto:[EMAIL PROTECTED]]
> > Sent: Friday, March 15, 2002 11:48 PM
> 
> > But if a cygwin
> > native version is needed nonetheless I could volunteer to package it.
> 
> IMO we should have a fully self-hosted distribution. At the moment, with
> the _single_ exception of postgresql, every package here can be rebuilt
> from source, to a version equivalent to what the package maintainer
> posted, on a cygwin system, with the tools that the distribution has. So
> yes, UPX should be a package before it's used to make packages.
> 
> I vote for including UPX... and Lapo makes two. Do we need a third? And
> are there any objections?
> 

Does UPX come with an API library that you can just use in setup?

Earnie.

_
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com

RE: Setup.exe chooser view strangeness

[Robert Collins]

I can duplicate this (I think). I should have a fix checked in shortly.

Thanks for the report..

Rob

> -Original Message-
> From: Pavel Tsekov [mailto:[EMAIL PROTECTED]] 

Am I the only one observing this behaviour ?

PT> If I go to the chooser window and select Devel->automake I don't see

PT> the dependent packages (automake-*, autoconf*) selected i.e. the 
PT> still stay to 'Skip'. Now if press View button and cycle to the 
PT> categories view - the dependent packages are properly displayed as 
PT> selected.

RFP: UPX (Was Re: reducing binary distribution size with UPX)

[Robert Collins]

> -Original Message-
> From: Lapo Luchini [mailto:[EMAIL PROTECTED]] 
> Sent: Friday, March 15, 2002 11:48 PM

> But if a cygwin 
> native version is needed nonetheless I could volunteer to package it.

IMO we should have a fully self-hosted distribution. At the moment, with
the _single_ exception of postgresql, every package here can be rebuilt
from source, to a version equivalent to what the package maintainer
posted, on a cygwin system, with the tools that the distribution has. So
yes, UPX should be a package before it's used to make packages.

I vote for including UPX... and Lapo makes two. Do we need a third? And
are there any objections?

Rob

Setup ready #2

[Robert Collins]

Ok, Setup200202 now
* checks for /etc/passwd.lnk and /etc/group.lnk,
* it already only creates missing /etc/passwd and /etc/group files.
* I still cannot duplicate the local install issue,
* and the new bin, src tickbox arrangement is in place.
* Win98SE may still have a graphics issue, I've received no input on
this.

I'm uploading a new snapshot, and will announce a "last-chance" on
cygwin@ for the win98SE issue.

Cheers,
Rob



 Old message ===


setup200202 has the following remaining key issues:

doesn't detect links for /etc/passwd and /etc/group
overwrites both /etc/passwd and /etc/group if either are missing (or
links)
(new) may have issues on win98SE.
(new) has an unconfirmed issue with local installs.
(new) cannot select source only if a package is not already installed.

I'm going to put a quick fix in for the first 1 or maybe 2 points.
Likewise for the last one. The 3rd I cannot help address - can anyone
here shed some light on it? The fourth likewise - it's unconfirmed. 

Once points 1,(maybe 2) & 5 are addressed, IMO we can release this.
Chris?

Rob

Re: reducing binary distribution size with UPX

[Lapo Luchini]

> Uhmm, UPX should be part of the distribution first, as a maintained
> pacakge, before folk start packing distributed binaries with it. Do we
> have a volunteer to maintain UPX?

UPX is quite cross-platform: you can use win32 version to package lonux
a.out such as linux verison to package win32 PE.
Moreover an UPX-compressed EXE is completely self-sufficient from UPX
itself, has no memory overhead and decompresses very very fast (10Mb/sec
on the author's Pentium133 as upx.sourceforge.net says).
But if a cygwin native version is needed nonetheless I could volunteer to
package it.

--
Lapo 'Raist' Luchini
[EMAIL PROTECTED] (PGP & X.509 keys available)
http://www.lapo.it (ICQ UIN: 529796)

RE: reducing binary distribution size with UPX

[Robert Collins]

> -Original Message-
> From: Lapo Luchini [mailto:[EMAIL PROTECTED]] 
> Sent: Friday, March 15, 2002 10:55 PM
> To: CygWin-Apps
> Subject: Re: reducing binary distribution size with UPX
> 
> 
> > No sideeffects;)
> > I'm using it since a year now.
> > Fetch my binary: http://familiehaase.de/cywgin/compression/upx/
> 
> rsync.exe 2.5.4-1 shrinks from 167936 down to 72704, and .bz2 
> archive too shrinks from 95901 down to 94571 (I didn't expect 
> this one)... anyone has objections if I pack future rsync 
> version with UPX, or it's "green"?

Uhmm, UPX should be part of the distribution first, as a maintained
pacakge, before folk start packing distributed binaries with it. Do we
have a volunteer to maintain UPX?

Rob

Re: [ANN] Updated: rsync-2.5.4-1

[Corinna Vinschen]

On Wed, Mar 13, 2002 at 10:21:47PM +0100, Lapo Luchini wrote:
> Ready at the same usual address:
> http://www.lapo.it/tmp/rsync-2.5.4-1.tar.bz2
> http://www.lapo.it/tmp/rsync-2.5.4-1-src.tar.bz2

Uploaded.

> BTW: I think having 2.4.6-3 as "prev" version is better than having
> 2.5.1-1 or 2.5.2-1 as it is a far more "solid" version (yes, it's still
> available at the same address as 2.5.4-1 up there).

I don't think it matters.

Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Developermailto:[EMAIL PROTECTED]
Red Hat, Inc.

Re: could someone please upload a new version of curl for me?

[Corinna Vinschen]

On Wed, Mar 13, 2002 at 01:33:47PM -0500, Roth, Kevin P. wrote:
> Would someone kindly upload a new version of curl for me? This is version 7.9.5-1, 
>and has lots of bugfixes and other improvements over the previous version. The files 
>are available as follows:
> 
>  Binary: http://curl.haxx.se/download/curl-7.9.5-1-cygwin.tar.bz2
> 
>  Source: http://curl.haxx.se/download/curl-7.9.5-1-src-cygwin.tar.bz2
> 
> As always, the only change I'll need you to make is to remove the "-cygwin" from the 
>filenames. No change to setup.hint is needed.
> 
> Please let me know once this has been completed so I can announce it.

Done.  Sorry for the delay...

Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Developermailto:[EMAIL PROTECTED]
Red Hat, Inc.

Re: Setup.exe chooser view strangeness

[Pavel Tsekov]

Hello Pavel,

Wednesday, March 13, 2002, 11:31:04 AM, you wrote:

PT> Return-Path: <[EMAIL PROTECTED]>
PT> Delivered-To: [EMAIL PROTECTED]
PT> Received: (qmail 12196 invoked by alias); 13 Mar 2002 11:36:14 -
PT> Delivered-To: [EMAIL PROTECTED]
PT> Received: (qmail 12193 invoked by uid 8); 13 Mar 2002 11:36:14 -
PT> Received: from sources.redhat.com (209.249.29.67)
PT> by mail.syntrex.com with SMTP id smtpd46RFA7; Wed, 13 Mar 2002 06:36:04 EST
PT> Received: (qmail 30457 invoked by alias); 13 Mar 2002 10:31:23 -
PT> Mailing-List: contact [EMAIL PROTECTED]; run by ezmlm
PT> Precedence: bulk
PT> Sender: [EMAIL PROTECTED]
PT> List-Unsubscribe: 
PT> List-Subscribe: 
PT> List-Archive: 
PT> List-Post: 
PT> List-Help: , 

PT> Delivered-To: mailing list [EMAIL PROTECTED]
PT> Received: (qmail 30380 invoked from network); 13 Mar 2002 10:31:17 -
PT> Date: Wed, 13 Mar 2002 11:31:04 +0100
PT> From: Pavel Tsekov <[EMAIL PROTECTED]>
PT> X-Mailer: The Bat! (v1.53d) UNREG / CD5BF9353B3B7091
Am I the only one observing this behaviour ?

PT> If I go to the chooser window and select Devel->automake I don't
PT> see the dependent packages (automake-*, autoconf*) selected i.e.
PT> the still stay to 'Skip'. Now if press View button and cycle to the
PT> categories view - the dependent packages are properly displayed as
PT> selected.

Re: reducing binary distribution size with UPX

[Lapo Luchini]

> No sideeffects;)
> I'm using it since a year now.
> Fetch my binary: http://familiehaase.de/cywgin/compression/upx/

rsync.exe 2.5.4-1 shrinks from 167936 down to 72704, and .bz2 archive
too shrinks from 95901 down to 94571 (I didn't expect this one)...
anyone has objections if I pack future rsync version with UPX, or it's
"green"?

--
Lapo 'Raist' Luchini
[EMAIL PROTECTED] (PGP & X.509 keys available)
http://www.lapo.it (ICQ UIN: 529796)