Re: Updated: 1.7.0-34 (...and a plea)

[Brian Dessent]

Corinna Vinschen wrote:

> Uh, I just remembered something I forgot so far.  We don't even need two
> setup.exe versions.  Setup-1.7 decides which setup ini file to use
> dependent on the Windows version.  NTs get setup-2.ini, 9Xes get
> setup.ini.  It even changes its layout a bit.  For instance, for 9Xes
> you still get the choice of text/bin mounts, for NTs you don't.  So we
> really only need a single setup.exe in future, AFAICS :)

Right, and since the 'release/' part is all in the .ini file anyway
setup.exe doesn't actually need to be changed at all to do the
switchover, it can be controlled entirely on sourceware just by telling
upset which release dir goes with which ini file.

Here's my question: Do we really need to inflict this 8.7GB of churn[1]
on every mirror just for what seems to me to be a cosmetic reason for
wanting the directory name to be "release/"?  Can't we just say that
release-2/ is now what the dir is called and move on?  In other words,
the switchover procedure would be "mv setup-1.7.exe setup.exe".

Users who continue to use an old setup.exe would still get the 1.5
distro until they redownload it, but that's okay because you need the
new setup.exe to cope with 1.7 changes anyway so that's a feature not a
bug.  Users who do redownload setup.exe will get the right one
predicated on whether they're running NT or not.

Brian

[1]:
[EMAIL PROTECTED] ~/ftp]$ du -hs release release-2
5.2Grelease
3.5Grelease-2

Re: readline build questions

[Brian Dessent]

Eric Blake wrote:

> I'm trying to resolve a bash bug caused by relying on auto-import rather
> than __declspec(dllimport): http://cygwin.com/ml/cygwin/2008-11/msg00340.html

There is another potential workaround that doesn't involve modifying the
readline headers.  If there are only a few places that compare function
pointers, then just compare against the _imp__FOO symbol directly.

extern char *_imp__rl_tab_insert;

...

if (function_returning_function_pointer () == _imp__rl_tab_insert)
  {
/* function returned pointer to rl_tab_insert() in the DLL.  */
  }

This will however fail if you try to link bash against a static readline
because there will be no sythesized _imp__FOO symbol, causing link
failure.  But here the breakage/restriction is limited to bash, rather
than being in the readline headers, i.e. it doesn't affect all readline
clients.

> Is it still possible to allow clients to choose between static and dynamic
> readline at link-time without supplying any compile-time flags, or does
> the fact that I am conditionally using __declspec mean that I have to
> adjust my conditions in readline.h, and that clients that want to link
> statically now have to define READLINE_STATIC in their own source?

Unfortunately, yes.  A call to a function explicitly declared dllimport
can only be satisfied by linking to a DLL containing that function, as
essentially the compiler has inlined the indirection of the thunk stub
into the call site.  So the client of such a library needs an explicit
compile-time way of declaring intent to link statically, removing the
__declspec.

> One other idea I had was to quit distributing a static libreadline.a, and
> only offer the dynamic version.  Does anyone see any problems with the
> idea of no longer providing a static library?

You could make the argument that any consumers of libreadline in the
distro should already be using the shared version anyway on general
principle, so this shouldn't affect them.  But then there could be
potential users that want to statically link it into their own
programs.  I'd say continue to provide the static version, but just note
in the readme the new requirement to configure with
CPPFLAGS=-DREADLINE_STATIC if they intend to statically link.  It's a
reasonable request given that it's not an uncommon idiom which is found
in other libraries, and it fixes a real bug.  (What's more, if anyone
complains you can tell them that the alternative under consideration was
simple removal of the static version in We're Just Mean fashion.)

Brian

Re: First time packager questions

[Brian Dessent]

Ralph Hempel wrote:

> 1. Should my build make just a .dll file to link against, or should
> I also provide a .a library for static linking?

You need to provide an import library for the DLL regardless of whether
you decide to also support a static version of the library.

"Yaakov (Cygwin Ports)" wrote:

> > 3. I'm currently using Cygwin 1.7 and gcc-4 to build and test
> >this out, but should I do it for 1.5 as well?
> 
> gcc-4.3 is not yet finalized; it should not be used for building
> packages going into the distro.

And obviously it should go without saying that you can't upload packages
built using 1.7 to the main distro either.

Brian

Re: [ITA] X.Org X11R7.4

[Brian Dessent]

"Yaakov (Cygwin Ports)" wrote:

> Each library subpackage is already labeled (runtime), (development), or
> (source) in sdesc.

Yes, that's kind of my point: say "dummy placeholder for source package"
and not just "source" so that its purpose (and the reason why it's
hidden from the user) is clear.

Brian

Re: [ITA] X.Org X11R7.4

[Brian Dessent]

"Yaakov (Cygwin Ports)" wrote:

> Because this means jumping through a lot more hoops, believe me.  This
> method (where there's no binary package corresponding to the source
> package) has precedence in Debian as well.

If it really does save you time than I remove my objection.  Can I just
request that the sdesc of such packages say something informative and
explicit about their purpose, e.g.:

sdesc: "X.Org Xfixes library (dummy placeholder for source package)"

I realize this string will never be seen by end-users but it does help
to self-document the purpose behind the package's existance when one is
troubleshooting setup by looking at setup.ini or whatnot.

Brian

Re: [ITA] X.Org X11R7.4

[Brian Dessent]

Brian Dessent wrote:

> I don't like the precedent this creates -- it's bad UI.  The user is
> supposed to be able to get the source by just putting an X in the Src
> box of the normal package.  But with this scheme they can no longer do
> that, they have to know to first show hidden packages and then to select
> the Src column of an empty dummy package that is otherwise never
> displayed, used, or needed.

Errr, nevermind.  That's nonsense.  Setup will follow the
external-source line and do the right thing when the user selects Src of
any of the packages.

> Why can't you just associate the source
> package with one of the two existing binary packages instead of
> inventing a third, empty, hidden package to associate it with?

But I still think this makes more sense.

Brian

Re: [ITA] X.Org X11R7.4

[Brian Dessent]

"Yaakov (Cygwin Ports)" wrote:

> libXfixes3 and libXfixes-devel have category: X11 and external-source:
> libXfixes.  AFAIK (and please correct me if I'm wrong) upset and/or
> setup require a binary libXfixes package to go alongside the source
> package, so they are empty.   I don't want the empty binary libXfixes to
> pollute the package list, so I used the _source category for libXfixes
> so that it's hidden by default.

I don't like the precedent this creates -- it's bad UI.  The user is
supposed to be able to get the source by just putting an X in the Src
box of the normal package.  But with this scheme they can no longer do
that, they have to know to first show hidden packages and then to select
the Src column of an empty dummy package that is otherwise never
displayed, used, or needed.  Why can't you just associate the source
package with one of the two existing binary packages instead of
inventing a third, empty, hidden package to associate it with?

Brian

Re: gcc-4.3 compatibility and cygwin-1.7 policy questions [Was:: Re: SECURITY vulnerabilities update 2007-Sep-25]

[Brian Dessent]

Charles Wilson wrote:

> Err...but that's a good description of cygwin-1.7, as well. Nobody (as
> far as I am aware) is suggesting that a test/preview gcc-4.3 package
> should be used as a "regular" compiler for cygwin-1.5.

I thought so as well, but people are already putting stuff built with
gcc4 into the distro:
.

> The question is, do we take a flyer on gcc-4.3 in the cygwin-1.7
> sandbox, hoping to get /all/ the kinks worked out -- in both gcc-4.3 and
> in cygwin-1.7 -- by the time cygwin-1.7 goes "gold".

I guess all I'm saying is that the minimum precondition before we should
even think about starting to recompile anything is that gcc4 should
default to shared and versioned libgcc.  Until then it's just going to
create more headaches to deal with later.

Brian

Re: gcc-4.3 compatibility and cygwin-1.7 policy questions [Was:: Re: SECURITY vulnerabilities update 2007-Sep-25]

[Brian Dessent]

Charles Wilson wrote:

> Well, if we continue -- at present -- with static libstdc++, then would
> we need to continue -- at present -- with static libgcc even for C
> libraries?  For example:
> 
> cygncurses-N.dll : if this C library is compiled using -shared-libgcc
> 
> then
> 
> cygncurses++-N.dll : this C++ library can't be linked (right?) It's C++,
> but depends on cygncurses-N.dll. From what I understand, you have to
> have static libgcc and static libstdc++, or shared libgcc and shared
> libstdc++, you can't mix them. And because you can't link against
> cygncurses-N.dll (which was linked against the shared libgcc) without
> specifying -shared-libgcc when linking your client...boom.
> 
> Or, am I wrong on that (I'd love to wrong about that) -- if so, then you
> CAN do what would effectively be -shared-libgcc -static-libstdc++?

I'm not aware of anything that would preclude mixing static libstdc++
and shared libgcc.  It needs some testing, obviously.

> updated gcc-4.3). Version bump the C++ libraries again.  However, if one
> of the "issues" is the versioning of the libgcc shared library, then the
> C libraries will ALSO have to be rebuilt again -- but they may (or may
> not) have to be version bumped again at that time.  They probably will.
>  Clients from scenario 2/phase 1 expect the "old" cyggcc_s.dll -- which
> was fine with cygncurses-N.dll which also used cyggcc_s.dll.  However,
> this new cygncurses-N.dll depends on cyggcc_s-2.dll so now the client
> will pick up two different runtime support libraries: cyggcc_s.dll
> directly, and cyggcc_s-2.dll via cygncurses-N.dll. That's bad.  So, even
> the C libraries will need the second version bump, for scenario 2/phase 2.

Yes, this is why having an unversioned but shared libgcc in the distro
is such a poison.  With the current state of gcc4 it's impossible to win
as maintainer of a C++ library: if you use the default options you get
static libgcc which means your library can't throw or catch exceptions
from other modules.  If you use -shared-libgcc you get a dependence on
an unversioned shared lib which makes the output unsuitable to be
released to the public in the distro because it will only cause
headaches later.  So I consider this gcc4 package to be in a preview
state, but it its output should not be considered suitable for packaging
yet.

Brian

Re: gcc-4.3 compatibility and cygwin-1.7 policy questions [Was:: Re: SECURITY vulnerabilities update 2007-Sep-25]

[Brian Dessent]

Charles Wilson wrote:

> Or is all the worry about about unwinding a C++ only issue? (And of

As far as I can tell, it is.

Note that gcc doesn't[1] emit any unwind tables for C language input,
even on platforms like Linux that have been DW2 for a long time.  So if
you somehow managed to get yourself into a situation where you were
throwing from a C++ frame and the unwinder encountered a C frame, it
would call terminate() there too -- that would be undefined behavior.

> But even if the unwinding differences (sjlj vs dwarf2) are NOT an issue
> for C libraries, will there be incompatibilities between C client apps
> that use static 3.4.5 libgcc.a and my C DLLs that use (static? shared?)
> 4.3 libgcc?

Aside from the EH issue, I don't think that there would be any problem. 
Here is a list of all the functions exported by libgcc.a and
libgcc_eh.a.  With the obvious exception of the EH ones and possibly the
tlsemu ones, I don't see any that would carry any inherent state
information such that they would get confused by one caller getting the
static version and another the one from a libgcc DLL.

Exception handling support:
  _Unwind_Backtrace
  _Unwind_DeleteException
  _Unwind_Find_FDE
  _Unwind_FindEnclosingFunction
  _Unwind_ForcedUnwind
  _Unwind_GetCFA
  _Unwind_GetDataRelBase
  _Unwind_GetGR
  _Unwind_GetIP
  _Unwind_GetIPInfo
  _Unwind_GetLanguageSpecificData
  _Unwind_GetRegionStart
  _Unwind_GetTextRelBase
  _Unwind_RaiseException
  _Unwind_Resume
  _Unwind_Resume_or_Rethrow
  _Unwind_SetGR
  _Unwind_SetIP
  __frame_state_for
  __gcc_personality_v0
  __deregister_frame
  __deregister_frame_info
  __deregister_frame_info_bases
  __register_frame
  __register_frame_info
  __register_frame_info_bases
  __register_frame_info_table
  __register_frame_info_table_bases
  __register_frame_table

Threading support:
  These should all map onto the respective pthreads API functions
  in the Cygwin DLL, so they shouldn't themselves have any state.
  __gnat_default_lock
  __gnat_default_unlock
  __gnat_install_locks
  __gthread_active_p
  __gthread_mutex_lock
  __gthread_mutex_unlock

TLS support:
  These might carry some shared state and require -shared-libgcc.
  However, since this did not exist prior to 4.3 there's no backwards
  compatibility issue either.
  __emutls_get_address
  __emutls_register_common
 
The rest are mostly arithmetic, which shouldn't maintain any state:

DI mode (64 bit long long) arithmetic:
  __muldi3
  __negdi2
  __lshrdi3
  __ashldi3
  __ashrdi3
  __cmpdi2
  __ucmpdi2
  __divdi3
  __moddi3
  __udivdi3
  __umoddi3
  __udivmoddi4
  __udiv_w_sdiv
  
Trapping arithmetic:
  __absvsi2
  __absvdi2
  __addvsi3
  __addvdi3
  __subvsi3
  __subvdi3
  __mulvsi3
  __mulvdi3
  __negvsi2
  __negvdi2

Bit operations:
  __ffssi2
  __ffsdi2
  __clzsi2
  __clzdi2
  __ctzsi2
  __ctzdi2
  __popcountsi2
  __popcountdi2
  __paritysi2
  __paritydi2
  __bswapsi2
  __bswapdi2

Float to an integer power:  
  __powisf2
  __powidf2
  __powixf2

Complex mode arithmetic:
  __mulsc3
  __muldc3
  __mulxc3
  __divsc3
  __divdc3
  __divxc3

integer<->float conversions:
  __fixsfdi
  __fixdfdi
  __fixxfdi
  __fixunssfsi
  __fixunsdfsi
  __fixunsxfsi
  __fixunssfdi
  __fixunsdfdi
  __fixunsxfdi
  __floatdisf
  __floatdidf
  __floatdixf
  __floatundisf
  __floatundidf
  __floatundixf

Misc:
  _alloca
  __chkstk
  __clear_cache
  __enable_execute_stack
  __eprintf
  __gcc_bcmp
  

IMHO, shared libgcc needs to be the default and all C++ libraries (and
applications that link to C++ libraries) will need to be rebuilt to use
it.  But at the moment this is a problem because:

a) gcc-4 still defaults to static libgcc, requiring -shared-libgcc flag
to link with the DLL
b) libgcc DLL is named just cyggcc_s.dll, but it should be versioned
c) the shared gcc runtime package is currently named just
"gcc4-runtime", but it should be both versioned and split into
individual components, i.e. we should have libgcc0, libstdc++6,
libgfortran0, and so on, however:
d) shared libstdc++ doesn't exist yet because of the operator new/delete
overriding issue.

At the moment it looks like our best option is to leave static libstdc++
the default but have a shared option available for code that doesn't
need to override operator new/delete, until (d) is fixed at which point
we make shared the default.  Ideally for future sanity I think we need
to get away from all these static copies of the runtimes being default
and make everything use DLLs, but I understand that's not practical
right now.

Brian

[1] Well, it does if you specifically ask for it with -fexceptions or
-fasynchronous-unwind-tables.  But AFAIK there are no C libraries in the
Cygwin distro built this way.

Re: Setup 2.573.2.3 says GDB 6.8-2 but installs 6.8-0

[Brian Dessent]

Thomas Nilsson wrote:

> In fact, the package is named gdb-6.8-2.tar.bz2 but
> when untar'ed displays binaries from 6.8.0.

The final number after the dash is the revision number of the packaging,
not part of the program's version number, so you should interpret 6.8-2
as "the second incarnation of the 6.8 package", and it indeed does
contain gdb 6.8.

Brian

Re: I could not still install octave-3.0.2-2 by setup.exe

[Brian Dessent]

Tatsuro MATSUOKA wrote:

> Apparantly cygwin setup went well, however no component of octave-3.0.2-2 is 
> installed.
> 
> I cannot indicate what is wrong.

As it says in the announcement this release is marked Experimental. 
When using setup.exe you will never have experimental/test versions
installed unless you ask for them explicitly by selecting the "Exp"
radio button.

Brian

Re: Is there a limitation I don't know about in setup.exe dependency checking?

[Brian Dessent]

Dave Korn wrote:

> want to trace the dependency?  That is to say, is it not allowed/intended to
> satisfy a missing dependency with a "test" version?

I think it tries to satisfy the dependency with whatever the current
global trust level is set to.  So if the radio button is on Curr but
there's no Curr version available, it won't do anything.  But if the
radio button is Exp then it should be able to add in Exp versions.  (I'd
have to dig into the code to see whether this is truly the case or
whether it's just hardcoded to Curr only.  I think that's a bug if
true.)

Now admittedly this is really stupid logic.  It should try to match the
trust level of whatever was just selected when adding dependent
packages.

Brian

Re: MinGW runtime and w32api

[Brian Dessent]

Chris Sutcliffe wrote:

> My question is, should I use the same naming convention for the Cygwin
> distribution, or keep things the status quo?

That doesn't conform to the Cygwin standards, so I'd say no.  If you
wanted to split up mingw-runtime into separate DLL and header packages,
they should be named something like libmingw and libmingw-devel.  But
seeing as how the only file that would be in libmingw is mingm10.dll
(and that's just a tiny stub that hardly ever changes), this seems like
real overkill.

w32api should likewise theoretically be named w32api-devel but I also
don't see any need to rename something gratuitously.

Brian

Re: [ITA] apr1-1.3.3-1

[Brian Dessent]

David Rothenberger wrote:

> I've changed the package structure as follows:
> ~ * libapr1: Runtime library and documentation.
> ~ * libapr1-devel: Development headers and libraries.
> ~ * apr1: Source package.

Something seems wrong here.  At the moment for the ABI=1 version we
have:

@ apr1
sdesc: "The Apache Portable Runtime (development/documentation package)"
category: Libs Devel
requires: cygwin libapr1

@ libapr1
sdesc: "The Apache Portable Runtime (runtime package)"
category: Libs
requires: cygwin

Okay, so you want to rename apr1 to apr1-devel, that makes some sense. 
But why does apr1 continue to contain just the source?  Why not just
have apr1-devel contain the source and make apr1 a completely empty
package?  I don't think category _obsolete should contain any packages
that contain current versions, it should either be older ABI versions
(awaiting remaining clients to be rebuilt) or empty dummy packages to
support renaming/splitting.

Does this also mean that you will take over these as well:

@ aprutil1
sdesc: "Additional utility library for use with the Apache Portable
Runtime (development/documentation package)"
category: Libs Devel
requires: cygwin libaprutil1 apr1 libdb4.2-devel libgdbm-devel

@ libaprutil1
sdesc: "Additional utility library for use with the Apache Portable
Runtime (runtime package)"
category: Libs
requires: cygwin libapr1 libexpat0 libiconv2 libdb4.2 libgdbm4 crypt

Also, there are these four old ABI=0 packages to deal with:

@ apr
sdesc: "The Apache Portable Runtime (development/documentation package)"
category: _obsolete
requires: cygwin libapr0

@ libapr0
sdesc: "The Apache Portable Runtime (runtime package)"
category: _obsolete
requires: cygwin

@ apr-util
sdesc: "Additional utility library for use with the Apache Portable
Runtime (development/documentation package)"
category: _obsolete
requires: cygwin libaprutil0 apr

@ libaprutil0
sdesc: "Additional utility library for use with the Apache Portable
Runtime (runtime package)"
category: _obsolete
requires: cygwin libapr0 expat libiconv2 libdb4.2 libgdbm4 crypt

I can't find anything in current setup.ini that actually requires these
ABI=0 versions (other than themselves) so can't we just delete the above
four outright?


Or, how about the following idea: repurpose the the package named "apr"
(which was previously the ABI=0 one) to be the main documentation and
source package for the current ABI=1, so that we'd have:

@ apr
sdesc: "The Apache Portable Runtime"
category: Libs
requires: libapr1 libapr1-devel

@ apr1
sdesc: "Empty placeholder"
category: _obsolete
requires: libapr1

@ libapr1
sdesc: "The Apache Portable Runtime (runtime)"
category: Libs
external-source: apr
requires: cygwin

@ libapr1-devel
sdesc: "The Apache Portable Runtime (headers)"
category: Libs Devel
external-source: apr
requires: libapr1

Brian

Re: [RFC] 1.7 Packaging: Obsolete packages

[Brian Dessent]

Corinna Vinschen wrote:

> Something's obviously missing...

Yes, I led you astray, sorry.  That is going to purge absolutely
everything that setup knows about packages.  In order to get that back
it would be necessary to re-parse setup.ini and re-scan the local
package directory for its contents to see what's available.

What's needed is rather than nuking the database, to undo the
information about what versions are currently installed.  This is rather
dicey because each packagemeta contains a std::set of packageversions,
each of which could have come from setup.ini, or from the parsing of the
filename in installed.db.  So we can't just remove everything from that
set.  But if you don't remove anything you'd leak (or end up with
duplicates, or end up with nonsense versions) because the code in the
installed.db reader (below) creates a new packageversion
unconditionally.  Really what we want to do is undo this:

  packageversion binary = 
cygpackage::createInstance (pkgname, inst, instsz, f.ver,
package_installed,
package_binary);
 
  pkg->add_version (binary);
  pkg->set_installed (binary);
  pkg->desired = pkg->installed;

I think what might be required is to add another field to packageversion
to denote whether it came from reading setup.ini or from reading
installed.db.  (Remember that it's possible that the installed version
is not Curr nor Prev nor Exp, it could be anything.)  Then add a method
to packagemeta that removes from 'versions' any that came from
installed.db, leaving the ones that came from setup.ini.  (This will
need an operator delete to balance the operator new in
cygpackage::createInstance otherwise it leaks.)  Then we can call that
method on each packagemeta in the packagedb::packages vector.  We'll
also need to set the 'desired' and 'installed' members to point to
something sensible, because otherwise they are potentially dangling if
they pointed to something we removed.  I'm not sure what that something
is, but to find out you can put some debugging statements in and find
out what everything looks like after parsing setup.ini and scanning the
local package dir but prior to reading installed.db to find out how to
reverse the changes.

Brian

Re: [RFC] 1.7 Packaging: Obsolete packages

[Brian Dessent]

Corinna Vinschen wrote:

> IMHO, the replaced-by would add nothing in terms of less maintainence
> burden.  The old package still has to be tweaked in one way or another.
> The only extra work without replaced-by is to create empty tar archives
> for the old packages, which is really simple.

As I do not maintain a huge number of packages I really can't comment on
what maintainers need or want.  Certainly we shouldn't waste time
designing something that's not going to be helpful.  But I do seem to
recall that Yaakov stated early in this thread that some kind of help
with renaming/splitting packages would save a great deal of work for
him, so perhaps we should wait for him or someone else with tons of
packages to comment.

Brian

Re: [RFC] 1.7 Packaging: Obsolete packages

[Brian Dessent]

Brian Dessent wrote:

> - the Replaced-by method would not allow a determined user to continue
> using an old version of a package without upgrading.  With the current
> scheme they can just mark the existing package as "Keep" (or select a
> "Prev" version) which has the effect of blocking the upgrade since it
> works on the principle of the bumped version triggering the action.  But
> with a Replaced-by keyword doing the action instead of the presence of a
> bumped version, there would be no choice.

And thinking about this some more, we could work around that issue by
implementing the treatment of the Replaced-by keyword "as-if" the user
had manually deselected OLDNAME and selected NEWNAME.  Thus if they
select "Keep" then that action is undone and the list of operations is
cleared as expected, and all packages stay at what they're set to now. 
Of course we'd need to hook it into the right places so that these
"as-if" selections are not just done once but every time that the trust
level is set to "Curr".  Otherwise, you'd miss the upgrade if you
clicked on Keep then clicked again on Curr (or Exp for that matter.)

So to sum up an implementation skeleton:

- add the keyword to the lexer and parser
- add an additional packagedb data member that contains a vector of
OLDNAME->NEWNAME pairs found from the result of parsing the ini file
- add code that runs after reading the installed.db that changes any
occurance of OLDNAME to NEWNAME in any package's dependency spec
- add code in the "set global trustval" codepath that runs after all the
exising calculations to set the transitions "as-if" the user had
deselected OLDNAME and selected NEWNAME for each pair, recomputing full
recursive dependencies each time -- but *only* if setting the trustval
to curr or exp, not keep.

Brian

Re: [RFC] 1.7 Packaging: Obsolete packages

[Brian Dessent]

Brian Dessent wrote:

> - the Replaced-by would have to be transitive in the dependency
> computation code as well.  So if a maintainer renames package OLDNAME to

And, as a corollary to that: Replaced-by should accept only a single
packagename as predicate, since we have this requirement of replacing
any occurance of OLDNAME with NEWNAME in the internal dependency
database.  I suppose it could in theory take a list of multiple packages
but then we'd have to deal with replacing one thing with multiple things
and that just seems overly complicated.  If a one-to-many relationship
is required it can be achieved by setting the "requires:" of NEWNAME to
pull in the extra ones.

Brian

Re: New Setup for Cygwin 1.7 on cygwin.com

[Brian Dessent]

Corinna Vinschen wrote:

> Why didn't upset pick up the change?

Perhaps it has to do with the design of upset being incremental, i.e.
taking the current setup-2.ini as input and applying updates to it
rather than generating it from whole each time.  It might be confused
upon not seeing a "requires:" at all in the .hint and so it leaves the
existing "requires:" in the .ini unchanged.  If this is the case then
listing an explicit empty "requires:" in the .hint would probably work
around it.  If true then it sounds like more of a bug to fix than
something we want to kludge around.

Brian

Re: [RFC] 1.7 Packaging: Obsolete packages

[Brian Dessent]

Corinna Vinschen wrote:

> Ok, I'll have a look.  Any idea about my other question?  How to remove
> the entire installed.db package DB when the user goes back to the root
> dir selection dialog so we can reload from another location, should the
> user choose one?

My aborted unfinished attempt at a 1.7 setup also suffered from this. 
The plan of attack that I had was to move the code that reads
installed.db out of the packagedb ctor and into its own private or
static function.  Then write a new member function that first frees the
in-memory copy (if it exists) and then calls that reader function.  Then
hook this function into the wizard dialog so that it's called every time
you move off the rootpage.

As far as the actual freeing of memory, I think it would go something
like

for (vector ::iterator i = packages.begin ();
 i != packages.end (); ++i)
  {
delete *i;
  }
packages.clear();

(And likewise for sourcePackages.)  Yes, it royally sucks that setup has
this horrific mix of containers as well as explicit pointer management. 
Ideally it would be great to eliminate all explicit new/delete
housekeeping and instead make them smart pointers, but that's a big
project.

> Is that really a big point?  The "replaces" stuff is meant to seemlessly
> replace one package with (usually) a functionally at least equivalent
> package.  There shouldn't be much of a difference for the user.

You mean is it not a big point that the uninstallation of the packages
listed in "Replaces:" wouldn't be indicated anywhre?  Yes, I suppose
it's somewhat of a minor point.  But I really do think that setup has
enough odd and inexplicable behavior already that I hesitate to add more
-- being able to always double check the precise set of
install/remove/upgrade operations that are about to take place with the
"Partial" view is a useful debugging tool.

> OTOH, a "replaces" rule does not allow to replace an obsolete package
> automatically.  So, if we want to upgrade the user automatically to the
> new packages, we would *still* have to provide empty obsolete packages
> with a setup.hint requiring the new packages.
> 
> What would the "replaces" rule buy us again?

I don't know.  Maybe what is needed is a key that works in the other
direction, "Replaced-by:" with the semantics of: if this package is
currently installed then de-install it and install everything in the
specified list.  So to rename/split a package you would upload the new
package(s) under the new name(s), move the old package to category
_obsolete (so that it's not shown) and add a "Replaced-by: newname"
line.  This would cause a silent upgrade for everyone the next time they
ran setup, the same way that bumping the version and uploading an empty
file does today.

Off the top of my head, complication with this idea:

- the Replaced-by would have to be transitive in the dependency
computation code as well.  So if a maintainer renames package OLDNAME to
NEWNAME, then a package that "requires:" OLDNAME would have to have
OLDNAME replaced with NEWNAME in the internal representation of the
dependency list.  It's tempting to say that the maintainer should just
add NEWNAME to the "requires:" of OLDNAME, and let the requirement
chain.  But I don't think that would fly because what we're saying with
this keyword is that OLDNAME won't be installed any more.  This is in
opposition to the current method where OLDNAME does actually stay
installed (in the mind of setup) but contains an empty payload so has no
footprint.

- the Replaced-by method would not allow a determined user to continue
using an old version of a package without upgrading.  With the current
scheme they can just mark the existing package as "Keep" (or select a
"Prev" version) which has the effect of blocking the upgrade since it
works on the principle of the bumped version triggering the action.  But
with a Replaced-by keyword doing the action instead of the presence of a
bumped version, there would be no choice.

Brian

Re: New Setup for Cygwin 1.7 on cygwin.com

[Brian Dessent]

Corinna Vinschen wrote:

> No, it hasn't.  I already removed this dependency a week ago.

Perhaps something wonky with unionfs is happening then because I
clearly see it in the setup-2.ini on sourceware at this moment:

[EMAIL PROTECTED] ~/ftp]$ awk '/^@ base-cygwin$/,/^requires:/' setup-2.ini
@ base-cygwin
sdesc: "Upgrade helper scripts from 1.5.x to 1.7.x"
ldesc: "Upgrade helper scripts from 1.5.x to 1.7.x"
category: Base
requires: bash coreutils

Brian

Re: New Setup for Cygwin 1.7 on cygwin.com

[Brian Dessent]

Corinna Vinschen wrote:

> All we did the last couple of days seems to have helped only marginally,
> if at all.  This morning I ran a 1.7 Base install from scratch, and the
> dependency order is still as screwed up as before:

But base-cygwin still has "requires: bash coreutils".  That's just going
to pull in _update-info-dir and mess everything up.  Remove that and we
should be fine without messing with _update-info-dir's requires line.

Brian

Re: [RFC] 1.7 Packaging: Obsolete packages

[Brian Dessent]

Corinna Vinschen wrote:

> IIUC, the ConnectedLoopFinder::visit() function is the core function
> which creates the dependency order.  What I don't get is this: The

That code is concerned with creating a topological ordering for the
specific purpose of running postinstall scripts.  This isn't called
until after the "remove old and unpack new" install phase is finished
and so the 'installed' member has already been set (e.g. the end of
Installer::installOne.)  Packages that are not selected or were earlier
uninstalled are irrelevant here; this is only about figuring out what
order to do postinstalls.

If you want to see a general example of dependency handling when a
package is selected, see ChooserPage::changeTrust (what is called when
you change everything at once, e.g. Curr to Exp) or
PickPackageLine::click (what is called when you click on a package.)  In
both cases the main workhorse is packagemeta::set_requirements and in
turn packageversion::set_requirements.  The logic for Replaces would
probably have to be wedged in there.

But it's a lot harder than just adding some code to deselect the
packages -- consider if setup followed the "Replaces:" advice and
unselected some packages because the user selected a new package, but
then they changed their mind and deselected that same package.  If we
don't go back and re-enable those packages then we potentially leave
them with a broken system.  We could delay this processing until later
in the process when the user no longer can change their mind, but then
the package selection pane becomes a lie.  We will potentially be
removing stuff that isn't marked as remove, and the "Selected" view no
longer accurately serves as a "here's a list of everything I'm about to
do" summary which I think is a valuable feature.

Brian

Re: New Setup for Cygwin 1.7 on cygwin.com

[Brian Dessent]

Charles Wilson wrote:

> You want cygwin, base-* installed at the same time.

This sentence doesn't make sense to me.  As if it's possible that some
of these might *not* be installed?  They're all in category Base so
they're all getting selected by setup for install on every run
regardless of anything that the dependencies might have to say on the
matter.  I don't understand why the extra level of complication of
meta-packages or dependency groups is needed; all we need to do is trim
or adjust the dependencies to ensure that base-cygwin's postinstall
sorts to the head of the line.  For these core packages that are in Base
it doesn't really matter that their "requires:" line accurately lists
each thing they need, because they're all getting installed anyway.

Brian

Re: New Setup for Cygwin 1.7 on cygwin.com

[Brian Dessent]

Christopher Faylor wrote:

> I thought the whole point was to install cygwin first.  How can you run base
> packages first in a new install when there is no cygwin1.dll?

The ordering only has meaning in the context of running postinstall
scripts, as the postinstall phase doesn't start until after all files
from all packages have been unpacked.

Brian

Re: New Setup for Cygwin 1.7 on cygwin.com

[Brian Dessent]

Corinna Vinschen wrote:

> How in the world can cygwin and the base-cygwin/base-passwd packages be
> so far from the root in the dependency tree?  Why is libintl3 in the list
> but libintl8 isn't?  Is the dependecy mechanism really reliable?

Essentially, because the Cygwin package depends on _update_info_dir. 
That brings in a dependency on texinfo, which brings in libiconv2,
libintl3, libncurses8.  libncurses8 brings in terminfo which brings in
bash and coreutils which brings in tzcode which brings in gawk.  And
that's pretty much the list of things that run before base-cygwin:

libintl3 texinfo _update-info-dir gawk tzcode libiconv2 coreutils
terminfo libncurses8 libreadline6 bash

You have to remember that the topological ordering code throws out an
edge when it detects a loop.  So if it's looking at the dependencies of
FOO to figure out what to run before FOO's postinstall and it finds a
package that depends on FOO it's going to just silently discard that
edge because there's nothing meaningful it can do with it.  If it
didn't, it would just recurse forever on the loops.  So it doesn't
matter that practically all of the above also depend on 'cygwin', which
would logically mean 'cygwin' should come before them.  But that logic
just leads to infinite recursion; setup has to consider only one edge in
the loop and it chooses whichever one it saw first.

Can we special-case upset not to add _update-info-dir to the Cygwin
package?

Brian

Re: New Setup for Cygwin 1.7 on cygwin.com

[Brian Dessent]

Corinna Vinschen wrote:

> I don't see _update-info-dir in the bash requires.

Are you looking at the setup.hint or the generated setup.ini?  IIRC it's
added by upset automatically if it detects the package has .info files.

> However, when looking through the dependencies, I see more problems.
> 
>   cygwin requires base-passwd
> 
>   base-passwd requires bash
> 
>   bash requires cygwin
> 
> We really need a strategy to fix the dependencies so that postinstall
> scripts of crucial packages run first.  And base-cygwin should run
> before every other package.

Didn't Eric upload a new bash package that is postinstal-less?  Or is
there an older one in release-2/ that shadows the newer one in release/?

To make base-cygwin run first, it should suffice to list it as a
dependency of e.g. bash or cygwin and to have it not depend on
anything.  Rationale: its dependencies (bash and coreutils) are in Base
and should be installed regardless so it shouldn't hurt to not list
them.  All listing them does is cause loops.

Brian

Re: New Setup for Cygwin 1.7 on cygwin.com

[Brian Dessent]

Corinna Vinschen wrote:

> What I still don't get is that, in my local install, the update-info-dir,
> coreutils, and terminfo postinstall scripts ran before the bash postinstall
> script, even though all of them list bash as requirement.  Any idea?

It's a circular dependency -- _update-info-dir and bash both list each
other as requirements.  In such a situation with loops it's impossible
to give a topological ordering that satisfies both constraints, so it
comes down to chance (or rather the order that things are evaluated) as
to which goes first.  It looks like _update-info-dir happens to get to
the top.

Brian

Re: [PATCH] Two fixes for setup postinstall handling.

[Brian Dessent]

Christopher Faylor wrote:

> If the one is _update-info-dir then it already relies on bash and I
> changed the postinstall script to use /bin/bash rather than /bin/sh.

I thought the other thread concluded it was terminfo whose setup.hint
needed a "requires: bash".

Brian

Re: [PATCH] Two fixes for setup postinstall handling.

[Brian Dessent]

Dave Korn wrote:

>   30 packages have it right.  202 have it wrong.
> 
>   This suggests to me that relying on a manual system is not working.

Point taken.  However in the vast majority of those cases the
topological order is not significantly adversely affected, it's only a
select few (or one?) packages that are at issue.   As a short term fix
if we can correct those errant few that do actually cause a scratch
install to go off the rails I think we should do that since it's such an
easy fix that requires no releasing or upgrading anything.

For the long term, we can add special casing for one special package, or
we can transition to a world where that one package does not need
special treatment; or we can do both.  In either case there is a time
delay: in the former, it is users continuing to use older versions of
setup, in the latter it's that we cannot demand or expect immediate
maintainer action on Eric's part.

So I suppose my point then is that we should do the short term fix
regardless.

Brian

Re: [PATCH] Two fixes for setup postinstall handling.

[Brian Dessent]

Christopher Faylor wrote:

> If this is necessary, I don't like the idea of special casing bash.
> Can't we add a keyword to setup.ini so this could be configurable?

I don't like this either.  I feel it's not necessary.  If we just fix
the setup.hints for any packages that have postinstalls but don't list
bash in "requires:" then this problem goes away, and it is a fix that
works with any version of setup without requiring any user upgrades.

Brian

Re: [patch] a new setup.exe parameter to specify packages

[Brian Dessent]

Christopher Faylor wrote:

> Maybe it's time to start looking for additional setup maintainers.
> Anyone want to volunteer?

I would be very pleased to see this happen, since it's clear that the
current amount of manpower that I can donate is lacking.

Brian

Re: setup.ini "depends:"

[Brian Dessent]

"Larson, Aaron" wrote:

> The attached setup.ini file describes three packages, A, B, and C,
> where A and B have a "1.1.1" and "2.2.2" versions.  I want version
> 1.1.1 of A to require 1.1.1 of B and similarly for A&B versions
> 2.2.2.  I'm expecting that when I select version 1.1.1 of A, setup
> will change B's version to 1.1.1, but I don't get that behavior.  In
> fact, I can see no effect of my "depends:" entries at all.  I've tried
> both our own minimally modified (changed the address of the
> mirrors.lst) and also a pristine copy of setup.exe version 2.573.2.2.
> The log file does not show any syntax errors or warnings.

Not possible.  Setup just doesn't support this.  If you want this kind
of dependency you have to make the version number part of the package
name, i.e.

@ a-1.1.1
sdesc: "A"
requires: b-1.1.1

@ a-1.1.1
sdesc: "second version of A"
requires: b-2.2.2

While the documentation you linked does mention "depends:" and it may
even be in the ini parser grammar, it was never fully implemented AFAIK,
so I recommend not using it.

Brian

Re: Package Grep source and Program not installed functionality

[Brian Dessent]

John Morrison wrote:

> > This already exists, it's spelled "cygcheck -p".
> 
> I didn't know.  Thanks for the sarcasm.  God I'd forgotten just how nasty
> this list can be.  And all for wanting to make cygwin a 'nicer'
> environment for new/inexperienced users.

Seriously, no snark intended.  You asked about adding feature X to
package-grep.cgi, but feature X already exists and is used by cygcheck
to implement -p.

As far as auto-suggesting things when commands aren't found, I think
there are a lot of implications to consider:

- privacy aspects of every user's misspelled commands ending up in
sourceware logs
- latency aspects of delaying the prompt by a HTTP roundtrip and a
sourceware grep for each missing command
- increased load on sourceware
- implications for offline/disconnected users
- large amounts of output when a command matches a string in lots of
packages

As far as I know ubuntu can simply query the local APT database which is
a different prospect entirely than initiating a HTTP request to a remote
server.

Brian

Re: Package Grep source and Program not installed functionality

[Brian Dessent]

John Morrison wrote:

> Would it be possible to extend the cgi
> 
> http://cygwin.com/cgi-bin2/package-grep.cgi
> 
> to (optionally) output plain text instead of the formatted HTML?
> 
> Add a output=text instruction?  If the source is available I'd be willing
> to see if I could do the mod.  I was thinking of some kind of command line
> which would simplify the users finding out what package they would need to
> install to access a particular tool, for example on my ubuntu machine;

This already exists, it's spelled "cygcheck -p".

Brian

Re: [RFC] 1.7 Packaging: Obsolete packages

[Brian Dessent]

"Yaakov (Cygwin Ports)" wrote:

> Could setup-1.7 know if the last installation was 1.5, and simply ignore
> the installed.db?

As outlined in 
the way that it will work is entirely dependent on what you enter for
the root location.  It will start up with the value from your existing
1.5 install; if you leave it the same it will read the installed.db from
that location and attempt to upgrade.  If you change the root location
to something else it will forget everything it ever knew about that
previous value (like anything it read from the installed.db there) and
do a new install to that location.  In either case it will write the
value to the new registry key so that it will start with that value the
next time it's run.

Brian

Re: Patches for setup HEAD

[Brian Dessent]

Ralph Hempel wrote:

> Is this on anyone's radar at all? Is there something I should
> be doing that I haven't?

The text/binary one as I already explained is completely pointless as
all the mount table code will be going away soon so it would be just
another thing to have to rip out.  I don't have time right now to review
the other two, and frankly any free time that I do get in the near
future will be spent on 1.7 features, so please just have some patience
and eventually we'll get to them.

Brian

Re: [RFC] 1.7 Packaging: Toolchain

[Brian Dessent]

Corinna Vinschen wrote:

> I don't know if that's a problem for std::wstring, but newlib is still
> missing the wprintf family of functions.

Based on the defintion of GLIBCXX_ENABLE_WCHAR_T in

the list of missing functions before wchar_t can be enabled is:

fgetwc
fgetws
fputwc
fputws
fwide
fwprintf
fwscanf
getwc
getwchar
putwc
putwchar
swprintf
swscanf
ungetwc
vfwprintf
vswprintf
vwprintf
wcsftime
wcstod
wcstok
wprintf
wscanf

Brian

Re: setup.exe for 1.7 and patches?

[Brian Dessent]

Ralph Hempel wrote:

> What I'm trying to get clear on is whether I shoud be trying to get
> the setup for 1.7 (snapshot 2.592) working with the patches.

No.  All patches should be against mainline.

Brian

Re: New Cygwin 1.7.0-18 in release-2

[Brian Dessent]

Corinna Vinschen wrote:

> > Actually I don't think it can handle any type of empty tarball can it?
> 
> I don't know.  I thought it handles one type and complains about the
> other.  I didn't actually test it.

It can handle the type created by "bzip2 

Re: [setup] Extra files under source control

[Brian Dessent]

Charles Wilson wrote:

> which are under source control in the setup repository really should
> /not/ be stored in CVS.  They are regenerated every time you run
> doconfigure, anyway.

It makes sense to not have them in CVS from a consistency standpoint. 
However, those dirs are intended to be vendor branches (or whatever CVS
calls it when you just bulk add external files) so that hopefully
keeping a current version just means importing an upstream release
tarball.  I don't know how CVS copes with that, I'd have to read up in
the CVS book.  Dave, do you know?

Brian

Re: Fix memleak in setup

[Brian Dessent]

Charles Wilson wrote:

> Any objections if I check this in?

Looks OK to me, go ahead.

Brian

Re: [setup patch] paint older-to-install versions red

[Brian Dessent]

Reini Urban wrote:

> Setup has a strange versioning logic:
> The setup.ini lastmodified date overrides the package number
> 
> This patch marks the spin button red of such packages which installed
> version numbers are higher than the current version of setup.exe, when
> setup wants to overwrite "newer" versions.
> Usually I have to click the spin buttons on those a couple of numbers.
> It's a useful visual feedback of todo's.

I understand the problem, but I don't like the solution at all:

Firstly, it's "mystery meat" UI.  A user sees a package in red.  What
does this mean?  It's indecipherable unless they happen to have read
this thread.  We'll get questions.

Secondly, we have the problem of non-default color schemes.  What we
have now is compatible because it's implemented as "use the currently
configured background color, and invert it for the foreground."  But
when you start hardcoding colors you have no way to know whether they
will be visible at all given the color scheme the user has configured.

Thirdly, I don't want to open the flood gates to colors.  I know that it
seems tempting to start painting all kinds of things in different colors
to make it easier to navigate, but without some forethought or some kind
of design guidelines it's just going to be an ad-hoc mess.  And, we're
already doing absolutely horrid in the accessibility department as is.

How about instead a command line flag to say "don't downgrade anything
even if [curr] of the setup.ini I'm told to read says to"?  Then you
won't need to be visually indicated of anything because you won't have
to manually cycle any versions to keep.

Brian

Re: Requirements depth in setup.exe

[Brian Dessent]

"Thrall, Bryan" wrote:

> We have a custom cygwin mirror for distributing our software and
> occasionally run into problems where a package that is listed in the
> setup.ini as being required isn't installed like it should be. For
> example, xterm is required (eventually) but isn't installed by default.
> The problem seems to be a limitation in setup.exe:
> packageversion::set_requirements() has a hardcoded recursion depth of 5,
> and as best I can tell, xterm for our purposes has a depth of 6.
> Increasing the limit to 20 allowed xterm to be set to install like it
> should be.

For the record, as of right now the longest dep chains[1] in setup.ini
top out at 12:

octave-htmldoc -> octave-doc -> octave -> gnuplot -> xorg-x11-bin-dlls
-> xorg-x11-base -> xorg-x11-libs-data -> tar -> bzip2 -> coreutils ->
tzcode -> gawk

octave-headers -> octave-devel -> octave -> gnuplot -> xorg-x11-bin-dlls
-> xorg-x11-base -> xorg-x11-libs-data -> tar -> bzip2 -> coreutils ->
tzcode -> gawk

There are many more at lesser lengths that are still greater than 5, so
it's clear that 5 is too small.

However, I'm concerned that this patch touches a lot more than just a
recursion/looping safety valve.  It seems like you're duplicating the
existing functionality of the 'visited' flag in a very inefficient way
by maintaining that set.  I'm going to have to play with this some more
before I can come to a conclusion.

Brian

[1] I wrote a rather brute force perl script to compute this:
.  The output takes a
while to compute and looks like
.

Re: http://sourceware.org/bugzilla/ setup.exe default assignee

[Brian Dessent]

Christopher Faylor wrote:

> >We only got three spam mails from bugzilla in the last years.  So it
> >could be useful.  Or to Brian Dessent, our maintainer.  He should
> >decide.
> 
> Brian?
> 
> In case it matters, just changing it to Brian is a couple minutes less work
> than sending it to cygwin-apps.

I tend to think that having reports echoed on the mailing list makes
sense, that way anyone reading can jump in and triage.

Alternatively, can't we just use unassigned at sourceware.org as the
default as is done with binutils and gcc?

Brian

Re: [rurban: Re: setup.exe command line parameters]

[Brian Dessent]

Dave Korn wrote:

>   I'm ok with that, asuming that the answers to these two questions are
> reasonable and proportionate to a Rea-Soon-Now release timeframe:

Well see that's the problem with this plan -- there have been so many
changes since the last release that I'd want to do a RC first and
advertise its existance on the main list for a week or two of public
tire kicking or so before going live.  Past experience has always shown
that this is required because virtually nobody follows the -apps lists
and does regular testing of snapshots, and problems always seem to crop
up.

But, alternatively, if we decide just to go with the release
branch+verification backport I think we can bypass this and just go live
now if you're confident enough about it.  But when the idea of slipping
in more and more things comes up I start to feel uneasy and want to go
back to having a public RC to prevent egg on our face.  Do you see the
dilemma?  I'd really rather not rush a release with a bunch of changes,
so can we just push out the verification change only and then work on
stabilizing HEAD and doing a RC without the time pressure?

Brian

Re: [rurban: Re: setup.exe command line parameters]

[Brian Dessent]

Dave Korn wrote:

>   Yes, but we haven't actually made that release yet, so I don't see why not
> to bung in a couple of user feature requests while I'm working on it.

In that case we should just smooth over the sharp edges on HEAD and do a
real release and forget about backporting.  With the addition of the
"check for and cope with a 'tar -T /dev/null' header of 512 \0s" fix I
think we'd be close, since in this context we're not talking about the
1.7 functionality.  That is certainly going to be better than picking
and choosing a couple of enhancements, which seems rather arbitrary to
me.

Brian

Re: [rurban: Re: setup.exe command line parameters]

[Brian Dessent]

Dave Korn wrote:

>   We already have a "-p host:port" proxy argument since a little while back.
> Think I may have only applied it to mainline rather than branch.  Will
> combine the best bits from all these patches.

I'd like to quash this idea that things should go on branches. 
Everything goes to HEAD.  The only reason we're talking about a branch
is because it was decided that we wanted to make an out-of-cycle release
of the recent signature verifying stuff.  But that's a special
exception, and I don't want to make any others.

There is no "1.7 branch".  When completed the work for 1.7 will be
enabled via #defines, and there will remain only one codebase.

Brian

Re: Patch for unattended setup (updated)

[Brian Dessent]

"Dr. Frank Lee" wrote:

> Apologies, _this_ is the new patch.

> -
> -theLog->exit (0);
> +if (rebootneeded) {
> +  theLog->exit (IDS_REBOOT_REQUIRED);
> +} else {
> +  theLog->exit (0);
> +}
>}

The GNU style has a brace on its own line.

> +  if (unattended_mode) {

Likewise.

> +  for_each(packages.begin(), packages.end(), 
> visit_if(mem_fun(&packagemeta::addToCategoryBase), 
> mem_fun(&packagemeta::isManuallyWanted)));

I'm not sure this is very performant.  'isManuallyWanted' is a rather
expensive function and it seems like a waste to have to perform all of
this:

> +static StringOption PackageOption ("", 's', "software", "Software packages 
> to include");
> +
> +bool packagemeta::isManuallyWanted() const
> +{
> +  string packages_option = PackageOption;
> +  string tname;
> +  /* Split the packages listed in the option up */
> +  string::size_type loc = packages_option.find(",",0);
> +  bool bReturn = false;
> +  while ( loc != string::npos ) {
> +tname = packages_option.substr(0,loc);
> +packages_option = packages_option.substr(loc+1);
> +bReturn = bReturn || (name.compare(tname) == 0);
> +loc = packages_option.find(",",0);
> +  }
> +  /* At this point, no "," exists in packages_option */
> +  bReturn = bReturn || (name.compare(packages_option) == 0);
> +  return bReturn;
> +}

...once for every instance of every package, which is hundreds. 
Couldn't the string parsing stuff be done just once to convert it to a
vector or list, so that it doesn't have to be reparsed over and over?

Brian

Re: [RFU] libgcrypt-1.4.1-1

[Brian Dessent]

[resend due to sourceware spam filter]

Gergely Budai wrote:

> I thought that cygwin-apps is the primary list for discussing packaging 
> issues, but thank you for the hint, I will do so.

It is, but every package maintainer must also be prepared to scan the
main list for bug reports of users of their packages.  It's not
reasonable to expect that users would necessarily know in advance
whether the problem they're experiencing is a packaging bug or not.  And
even if they did know they would have to know of the existance of the
cygwin-apps list.  And even if they did know, they'd have to first
subscribe since cygwin-apps is a closed list whereas anyone can simply
send an email to the main cygwin list without first subscribing.

Brian

Re: [RFU] libgcrypt-1.4.1-1

[Brian Dessent]

Gergely Budai wrote:

> Package location:
> =
> wget \
>   http://home.arcor.de/thuffir/cygwin/libgcrypt/libgcrypt-1.4.1-1-src.tar.bz2 
> \
>   http://home.arcor.de/thuffir/cygwin/libgcrypt/libgcrypt-1.4.1-1.tar.bz2

This package is broken -- the import lib and static lib have been
stripped which removes their symbol tables and renders them useless.

Testcase:

$ cat >foo.c <
int main() { return !gcry_check_version (NULL); }
EOF

$ gcc -o foo foo.c `libgcrypt-config --cflags --libs`
/tmp/ccTal48j.o:foo.c:(.text+0x32): undefined reference to `_gcry_check_version'
collect2: ld returned 1 exit status

$ nm /usr/lib/libgcrypt.dll.a

d000187.o:
nm: d000187.o: no symbols

d00.o:
nm: d00.o: no symbols

...

d02.o:
nm: d02.o: no symbols

d01.o:
nm: d01.o: no symbols

$ nm /usr/lib/libgcrypt.a 

libgcrypt_la-visibility.o:
nm: libgcrypt_la-visibility.o: no symbols

libgcrypt_la-misc.o:
nm: libgcrypt_la-misc.o: no symbols

...

mpih-sub1-asm.o:
nm: mpih-sub1-asm.o: no symbols

mpiutil.o:
nm: mpiutil.o: no symbols

The [prev] version 1.4.0-1 is OK; the above commands work.

Brian

Re: [PATCH] - setup.exe --text-mode and --current-user-only options

[Brian Dessent]

Servaas Goossens wrote:

> Please consider the following patch to setup. (I tried to follow the
> patch submission guidelines, please correct me if I missed something).

This definitely conflicts with the direction that setup is moving in. 
In the 1.7 patches that I've been (slowly!) working on, the text/bin
choice is completely removed from setup because it's no longer stored in
the registry.  To get textmode with 1.7 you edit fstab.

Brian

Re: Perl 5.10 stabilization

[Brian Dessent]

"Yaakov (Cygwin Ports)" wrote:

> a .dll.a also exists.  I'm not sure if libtool's behaviour is by design
> (for fear of undefined symbols), or simply an oversight.  Chuck?

The part of ltmain.sh you're referring to is where it refuses to link to
a static library when *creating* a shared library.  It refuses to do
that because it cannot be done portably -- for example on most ELF
platforms (with the exception of i386) shared libraries must contain
only PIC code, and normally static libs are not built PIC.

Libtool should have no problem linking a final executable against a
combination of shared and static libs.

Brian

Re: setup.exe: Invalid or unsupported tar format

[Brian Dessent]

Christopher Faylor wrote:

> Yes, I'm confused by this.  I did replace all of the zero length tar files
> with the 46 byte versions.  That's what I've always done when creating an
> empty package.

Here is a summary of the situation:

The current stable/release version of setup will never warn if it can't
understand a package, it will just silently skip it.  Mostly, this is
what you want, as it means it will cope just fine with both the 14b and
46b packages.

However a while ago someone reported that a package wasn't installing
and after investigation it was because the maintainer had used something
like cmake with its own libtar implementation that used an incompatible
magic.  Setup was silently refusing to install the package even though
everything indicated that it was to be installed -- very confusing.  I
added support for this new magic value and rewrote the tar checking code
to explicitly warn if it cannot grok a package.  This change existed
only in testing/snapshot versions so few people would have experienced
it.

Unbeknownst to me at the time, this meant it would report on both the
14b packages (since they contain nothing after decompression) and the
46b packages (since they don't actually contain a tar header but 10k of
\0 which is unrecognisable to setup as a valid tar header.)  I admit
that I don't regularly install packages from _obsolete so I never saw it
in any of my testing.

Some time later someone reported using a setup snapshot and seeing the
warning for an empty 14b package.  At the time I attributed this to the
14b package not being a 46b package, and added code to setup to skip the
14b package without a warning, assuming incorrectly that the 46b case
also worked as no one had reported that -- until now.

What I intend to do is add another special-case for checking that if the
first 512 bytes are all zeros to assume it's an empty tar file and
suppress the warning.  This will mean that both 46b and 14b packages are
correctly handled, as well as still preserving the warning if a real
package is unrecognisable.  But until I can add that, if you are using a
snapshot version only the 14b packages will be correctly handled; the
46b ones will warn.  This is mostly just a nuisance as installation
continues after the warning is dismissed and it's not like you're
missing anything from the empty package, but it's a nuisance nonetheless
and setup ought to be able to handle both types of empty package.

I don't think there's any need to convert anything on the mirrors again
because the next actual release version of setup should cope with both,
and people using snapshots should be able to deal.

Brian

Re: setup.exe: Invalid or unsupported tar format

[Brian Dessent]

"Thrall, Bryan" wrote:

> I've built setup.exe from source (updated today), and the ChangeLog says
> Brian Dessent fixed the "Invalid or unsupported tar format" problem with
> empty tar.bz2 files, but I am still seeing the error popup.
> 
> I have an empty meta-package to pull in other packages copied from one
> of the _obsolete packages. I understand these are created via 'tar -T
> /dev/null -cjf foo.tar.bz2' (see
> http://cygwin.com/ml/cygwin-apps/2008-04/msg00105.html), and diff
> reports the results identical; the file size is the expected 46 bytes.

That fix was for empty bzipped files, i.e. the result after
decompression is a 0 byte file, so it's not exactly relevant to this
situation.

> The setup.exe source (install.cc, Installer::installOne) looks like it
> expects an error peeking from try_decompress in this situation, but
> running in the debugger shows try_decompress->peek() successfully
> reading '0'. Note that if you bunzip2 the empty package, you get a 10K
> tarball which is all zeroes, so success isn't completely unexpected...

This code should not have ever been reached if the file contained a
valid tar header, since archive::extract should not have returned NULL
in that case.  For a valid-yet-empty tar file (i.e. what the 46 byte
thing is supposed to be) archive::extract should succeed but the first
call to next_file_name will return an empty string.

But it seems that "tar -T /dev/null" doesn't in fact produce anything
resembling a valid tar file, it simply spits out 10k of zeros.  And I
checked all the old existing 46 byte .tar.bz2 files, and they are in
fact just 10k of bzipped zeros.  Sigh.  Why does tar do this?   This
causes ::extract to return NULL (since it looks nothing like a tar file)
but it's also not an empty file, so peek doesn't fail either.

And again, to recap: the reason for generating an error instead of
silently moving on  in this situation was to diagnose the case where the
maintainer used a strange tar implementation to create a package --
without the error the package will be silently skipped which is very
confusing.  But it seems like this nonsense 10k of zeros will have to be
special cased in order to be able to detect the desired error case from
the bogus error case.

In the mean time, I suggest you use a real empty file as workaround
(i.e. bzip2 file.tar.bz2) as that should correctly be
detected as an empty package if you're using the current setup.

Brian

Re: package lint suggestions

[Brian Dessent]

"Yaakov (Cygwin Ports)" wrote:

> | - contains postinstall/preremove script without requiring bash (this
> | matters for the order in which scripts are run)
> 
> Isn't that an overkill; bash is in Base, after all.

It matters for the order that the postinstall scripts are run, because
that order is determined by the topo sort on the 'requires:'.

The bash package installs /bin/bash.exe and copies that to /bin/sh.exe
in its postinstall if there is not already a /bin/sh.exe that isn't
ash.  This means if you simultaneously install bash and some other
package that has a postinstall script (as during a new installation),
and if that other package doesn't list bash in the 'requires', then
there is a chance that the other package's postinstall will be run
before bash's postinstall, and consequently that postinstall will fail
with "invalid interpreter" if it starts with "#!/bin/sh" as postinstalls
are always invoked with "/bin/bash -c" which honors the shebang.  But if
the package 'requires:' bash then its postinstall will always be run
after bash's postinstall, avoiding the issue.

We have historically lucked out here for several reasons:

- many postinstalls don't have a shebang, or explicitly list #!/bin/bash
- outside of the case of a fresh new install, /bin/sh tends to always
exist
- the bash postinstall tends to be near the front of the topo order
anyway

(So I suppose technically the lint rule should be that if you have a
postinstall that stats with "#!/bin/sh" you must 'require:' bash.)

Alternatively, if bash were to directly install both /bin/bash and
/bin/sh instead of copying the latter in a postinstall then we would no
longer have this requirement.

Brian

Re: setup-1.7 can't be used to update, only install

[Brian Dessent]

Charles Wilson wrote:

> Anyway, by the time I got to the chooser, setup-1.7 thought that I was
> doing a brand new installation. I had all the base files selected!
> That's not right.

Urgh, you're right.  You'd have to temporarily switch the 1.5 mounts to
point to the 1.7 tree in order to get setup-1.7 to update an existing
1.7 tree... ugly as heck.

I have some patches mostly finished for setup to make this not quite so
crude.  There are still some corner cases that don't work correctly
which is why I haven't published anything yet.  I'll try to wrap it up
this weekend.

Brian

Re: package lint suggestions

[Brian Dessent]

Christopher Faylor wrote:

> I know that I'm missing some obvious stuff here.  What am I missing?

- setup.hint sdesc that starts with the package name

- setup.hint missing any of: sdesc, requires, category

- sanity checking for setup.hint category - I don't want to say hardcode
a list of allowed categories, but it would be nice to catch typos and
prevent accidentally creating new categories.

- contains postinstall/preremove script without requiring bash (this
matters for the order in which scripts are run)

- man pages not gzipped (this would certainly have to be a warning only
as many packages don't do this)

- installs conf file directly into /etc instead of putting it under
/etc/defaults/ and installing it with a postinstall/preremove only if
the user hasn't customized it

- two packages owning the same file - this would require some kind of
global state (e.g. a tied hash with a backing dbm file) in the lint
script so it may be prohibitive, but it would be good to catch two
packages that accidently overlap

Brian

Re: Putting my packages up for adoption

[Brian Dessent]

Warren Young wrote:

> > Otherwise, you potentially violate the GPL licensing requirement to
> > provide the source as it would disappear from the mirrors entirely as
> > new 'foo' versions pushed the old one off.
> 
> So what I need to do, then, is rebuild the 1.95.8 package so it _only_
> builds the DLL package?

I think it's up to you how much you want to edit the package.  The
important thing is that you have a source package for libfoo that can
be used to build a libfoo binary package.  If that src package also
generates foo and foo-devel binary packages (i.e. obsolete ones for the
old ABI that would never be uploaded) then I don't think anybody would
complain.  But they'd never be used, as the whole point of the
compatibility package is that it's just for things that haven't been
rebuilt yet.

> I can see also building the devel package, but
> only if there's a lot of unwillingness to port to expat 2, and to know
> whether that's going to happen, we have to release without and see what
> happens, right?

Well that's a whole other issue.  The normal library versioning scheme
only versions the binary runtime DLL package, and does not allow for
anything but a single version of the -devel (and main) package.  If you
want to support the ability to have more than one -devel package it's
*much* more work because you have to rename all the packages, and you
have to make sure the headers for each -devel package get placed into
their own versioned directory.

For example, a normal library:

foo:  /usr/share/doc/foo-X.Y/readme.html
foo-devel:/usr/include/fooheader.h
libfoo0:  /usr/bin/cygfoo-0.dll
libfoo1:  /usr/bin/cygfoo-1.dll

versus:

fooX.Y:   /usr/share/doc/foo-X.Y/readme.html
fooX.Z:   /usr/share/doc/foo-X.Z/readme.html
fooX.Y-devel: /usr/include/foo-X.Y/fooheader.h
fooX.Z-devel: /usr/include/foo-X.Z/fooheader.h
libfooX.Y:/usr/bin/cygfoo-0.dll
libfooX.Z:/usr/bin/cygfoo-1.dll

And then all clients that use libfoo have to "#include
" or "CPPFLAGS=-I/usr/include/foo-X.Y" or some
other scheme to select the desired version.  Unless the library was
designed to have this kind of versioned interface from the start it's
not a very workable situation without modifying all downstream clients,
which is why it's rather rare and the normal layout doesn't support it. 
For one example see the 'db' packages.

Brian

Re: Putting my packages up for adoption

[Brian Dessent]

Warren Young wrote:

> Yes, but the package name changed, so I think all I have to do is ensure
> the RFU request says to leave libexpat0 alone, right?  Then other
> package maintainers can rebuild for libexpat1 at their leisure.  Unless,
> that is, it's a big deal to port to expat 2?

There's more to it than that.  When libfoo transitions to libfoo
there is a need to create an explicit source package for libfoo,
because building from the 'foo' -src package of foo now creates
libfoo.  In other words, you need to make a copy of the old source
corresponding to the  abi and make it into its own -src package so
that it remains possible to rebuild libfoo from sources on the
mirrors.  This means that the setup.hint for libfoo no longer simply
contains an 'external-source' line but has its own -src package. 
Otherwise, you potentially violate the GPL licensing requirement to
provide the source as it would disappear from the mirrors entirely as
new 'foo' versions pushed the old one off.

This is completely separate from the issue you mention that you need to
keep the binary compatibiliy package around until all packages in the
distro have been rebuilt against libfoo, at which point you can
finally retire it (which means bumping the release number and uploading
a corresponding new package that contains an empty tar file, with
category changed to _obsolete.)

Brian

Re: tcp-wrappers for 1.7.0

[Brian Dessent]

Corinna Vinschen wrote:

> Wasn't there a guy on the cygwin ML a couple of weeks ago who claimed
> to have solved the problem with unresolved symbols?  There was some
> shoulder patting on the list, but nothing came out of it so far, right?

That would be FlexDLL:  which is
essentially a cute trick to implement a dlopen/dlsym interface that
allows for symbols to remain unresolved at link-time and instead be
dynamically resolved by a ld.so-like dynamic linker at runtime.

It's a neat concept, and as far as I can see it works fine.  But the
fact that it is built around explicit dynamic runtime linking (i.e.
dlopen/dlsym) means that it doesn't help in the more general case where
you want the operating system loader to have ELF-like semantics too, not
just for programs that have plugins.

Brian

Re: 1.7 packages

[Brian Dessent]

"Yaakov (Cygwin Ports)" wrote:

> * where can a list of practical changes between 1.5 and 1.7 be found?

Probably the best list is:
.  However,
things are always in flux.

> * can 1.7 be installed and/or run in parallel to 1.5, or is a separate
> box required?
> * how is 1.7 to be installed?

They can't be run in parallel (simultaneously), but they can be
installed in parallel.  1.7 reads the mount table in /etc/fstab, which
means if you keep a 1.7 tree it won't see the mounts of your normal 1.5
tree in the registry.

The current hitch is that the work in setup has not been completed to
compliment this, so it's a little hacky.  To install a 1.7 tree in
parallel to a 1.5 tree, this is the procedure:

1. Stop all 1.5 services and processes.
2. Save and then remove the 1.5 mount table, e.g. "mount -m
>/bin/mounts.bat; umount -A".  You might also need "umount -u -A" if you also 
>have user mounts, I don't remember if -A removes both system and user.  Make 
>sure the mount table is empty before continuing.
3. (Optional) Temporarily rename your 1.5 tree to something else just to
make sure it's not touched by setup, e.g. "ren \cygwin \cygwin-foo". 
This shouldn't be strictly necessary if you've ensured the mount table
is empty.
4. Run  making sure to select a
new/different root dir for the 1.7 location.
5. (Optional) Rename your 1.5 tree back if you renamed it in 3.
6. Remove the useless registry mounts that setup-1.7 just made and
restore the 1.5 registry mounts by running the mounts.bat created in 2. 
Make sure that this batch file calls the 1.5 mount command, e.g. by
running it in the bin dir of the 1.5 tree.

At this point you should have two working trees: the 1.5 tree with its
mount table in the registry and the 1.7 tree with its mount table in
fstab.  You can verify that everything is setup correctly as running the
1.5 mount command should show the 1.5 tree as root and the 1.7 mount
command should show the 1.7 tree as root.

>From this point on you should be able to do any testing you want, as
long as you don't try to have things running from both trees at once. 
Services of course will present a bit of a problem; you'd have to either
install a second copy with a different name or always remove/reinstall
when switching, and of course they shouldn't be used simultaneously.

Once setup has been updated this procedure will be a lot simpler, but
the above should work until then.  You can continue to update the 1.5
tree by using the normal setup.  To update the 1.7 tree you will
unfortunately have to run through the above each time, as setup-1.7 will
still see the 1.5 mount table in the registry, so you have to hide it
first and restore the 1.5 one afterwards.

> * how are 1.7 packages to be uploaded and organized?

As far as I know everything works the same, except that you upload the
files under release-2/.  Everything else should be automatic.

> * for how long will maintainers be expected to maintain two sets of
> releases?

IMHO, only until 1.7 is released.  After that, the 1.5 tree will be left
to grow weeds and drift off into the mists of time.  Of course, if you
feel like releasing updates into the 1.5 tree (e.g. security patches)
I'm sure that would be fine, but I don't think there's any expectation
of service once 1.7 is finally out the door and made current.  But this
is all just MHO, others may have a different view.

Brian

Re: [HEADSUP] Let's start a Cygwin 1.7 release area

[Brian Dessent]

P?teris K?avi?š wrote:

> So why can't the registry entry be  instead of the fixed
> 'setup\rootdir' in order that the setup application could populate a
> drop-down list of all currently installed Cygwins?

It could.  But I think Chuck made a good point that the ability to have
multiple installed copies extends to what you might call "expert users",
i.e. they must know the ins and outs of not actually ever using these
two installs at once.  And so we need to be careful not to give the
impression that this is supposed to work in the general case.

> Maybe all cyg*.exe native Windows applications could use this same
> algorithm to compute their root.

cygcheck and strace are the only things that use the code in question,
and strace only uses it for the purpose of resolving the win32 filename
of the output file specified with -o.

> Would this problem go away if cygrunsrv.exe used the same algorithm to
> compute its root?

No, the problem is that the operating system's service configuration in
the registry needs the path to cygrunsrv.exe -- it needs to know the
path to the image to execute, and we of course have no control over
that.  The only workaround I can think of to providing an absolute path
would be to use the REG_EXPAND_SZ key type's capability to expand
variables from the environment, e.g. %CygwinRoot%\bin\cygrunsrv.exe. 
However, this doesn't really solve anything, it just shifts the problem
to making sure this variable is set.  And even if you figured out a way
to do that, the change in environment would not take effect in the
services.exe process until the next time it was started, i.e. a reboot. 
It would still be easier just to edit the service config after moving
the files.

> I have (finally) started using an environment where I share all my
> personal files on a separate disk drive accessible from all my bootable
> operating systems, whether it's Vista, WS2008, or Debian.  The drive
> names (and mount points) vary.  It's the same as moving install trees,
> and it would be nice if Cygwin didn't really care that when I started
> the services this time they were started from a different location than
> last time.

In this case the problem has nothing to do with any code in Cygwin.  But
no I don't see your situation as the same, since as long as the location
is consistent on any given system, you can install the service once and
it will continue to be correct from there on.

Brian

Re: [HEADSUP] Let's start a Cygwin 1.7 release area

[Brian Dessent]

Charles Wilson wrote:

> (1) we're talking about cygwin-1.7.x
> (2) Either
> (a) only one of those parallel instances is in use at a time,
> including installed services,
> or
> (b) all of those parallel instances have exactly the same version of
> the DLL

Right, I don't think anyone is talking about having two trees in use
simultaneously.

> (3) Even so, doing this is not officially supported (the current
> dual-installation "support" vis-a-vis cygwin-1.5.x and cygwin-1.7.x is
> (a) temporary and
> (b) intended for package maintainers only, in order to prepare the
> cygwin-1.7 packages -- and package maintainers generally know what
> they are doing and what the pitfalls of a dual/multiple installation
> are.

At the moment this is quite experimental, yes; but I don't see any
reason why it would be a temporary arrangement.  Once we get the changes
in setup.exe and 1.7 released, it will still be possible to keep
multiple parallel installs by just changing the root dir in setup to
select which one to install/update.  Why would we want to prevent that
from working?  Isn't this one of the desired payoffs of moving the mount
table out of the registry in the first place?

Brian

Re: [HEADSUP] Let's start a Cygwin 1.7 release area

[Brian Dessent]

P?teris K?avi?š wrote:

> Hmm.  Well, if it's only setup that needs the registry entry, and if
> multiple setups, regardless of , are using the same fixed
> registry entry serially, what's the point of the registry entry?

So that when you run setup it can start out with the same root that you
last used so that you don't have to type it in every time.

> Remembering where the last setup's root was?

That, and also so that the native tools like cygcheck can locate the
root dir.  This will be a minor problem in that if you had two installs,
both cygchecks will think the root is the last one that you updated with
setup.  Corinna, perhaps cygcheck needs to compute the root relative to
its location like the DLL instead.  This means cygcheck will always have
to be in /bin, but I think that's fine; it's not like it has ever been
anywhere else.

> Which could have been moved by the user afterwards?

Manually moving an installed cygwin tree would be a silly thing to do,
it would break all your services for example.  Anyway, if someone were
silly enough to do that they would just need to enter the new location
in setup; but this will be the case whether or not setup remembers its
last install root, so I don't see how this is pertinent.

Brian

Re: [HEADSUP] Let's start a Cygwin 1.7 release area

[Brian Dessent]

P?teris K?avi?š wrote:

> Does the Cygwin DLL know its own path?  If so, how about
> 
> \Software\Cygwin\
> 
> ... to allow multiple parallel Cygwin installs?

I think you've missed the context of this discussion.  This key will
never be read or written by the DLL, and the DLL already supports
parallel installs by the fact that it uses /etc/fstab.

Brian

Re: [HEADSUP] Let's start a Cygwin 1.7 release area

[Brian Dessent]

Corinna Vinschen wrote:

> I don't know either.  I'm not going to change what's in Cygwin
> right now since it's seldomly used anyway.  So, for now, let's
> just agree on
> 
>   \Software\Cygwin\setup\rootdir
> 
> so I can patch utils/path.cc and upload a new cygwin 1.7.0-4, ok?

Okay, let's go with that.

Brian

Re: [HEADSUP] Let's start a Cygwin 1.7 release area

[Brian Dessent]

Corinna Vinschen wrote:

> I would suggest to use a setup subkey, because you might find that
> you have to store more than one value at one point.  It would be
> cleaner this way, imho.  And originally we came up with the version
> number, too, so what about "setup-2" or "setup"?

Traditionally setup has stored all of its state in files under
/etc/setup.  I think this is a good thing to continue, since it makes it
easy to access and manipulate them from scripts.  Given that we're
moving the mount table out of the registry and into /etc, I don't see
why we'd want to do the reverse with setup's bits.  So in that sense, I
don't see a need for a subkey that will only ever have one key.

On the other hand, I'd hate for a bad decision now to make things
annoying in the future.  Let me ask this: what's the layout for the
other items that will still be in the registry?  Are we still looking
at:

\Software\Cygwin\Program Options\default
\Software\Cygwin\Program Options\foo.exe
\Software\Cygwin\Program Options\bar.exe
\Software\Cygwin\heap_chunk_in_mb
\Software\Cygwin\heap_slop_in_mb

This is kind of ugly too, we have a mix of things with spaces and
underscores.  How about the following:

\Software\Cygwin\setup\rootdir
\Software\Cygwin\options\heap_chunk_in_mb
\Software\Cygwin\options\heap_slop_in_mb
\Software\Cygwin\options\defaults
\Software\Cygwin\options\app-specific\foo.exe
\Software\Cygwin\options\app-specific\bar.exe

Or, perhaps:

\Software\Cygwin\rootdir
\Software\Cygwin\heap_chunk_in_mb
\Software\Cygwin\heap_slop_in_mb
\Software\Cygwin\default_options
\Software\Cygwin\app-specific\foo.exe
\Software\Cygwin\app-specific\bar.exe

I really don't know.

(The version number I think is implicit in the fact that this is a
totally new key that's never been used before.  If in the very distant
future setup needs yet another namespace it can have a "setup-2" subkey
then.)

Brian

Re: [HEADSUP] Let's start a Cygwin 1.7 release area

[Brian Dessent]

Jörg Schaible wrote:

> > That reminds me of something... I've always been a bit annoyed that
> > people always resort to manually editing the registry to
> > remove Cygwin,
> > when umount is the documented and preferred method.  But now, we will
> > have no such utility to suggest and so to remove Cygwin the
> > user will be
> > forced to use regedit.
> 
> regtool remove {HKLM,HKCU}\Software\Cygwin\???

I suppose I didn't state my point very well.  Yes, there are Cygwin
tools for manipulating the registry.  But the registry is an
implementation detail, and you should not need to know anything about
registry keys to uninstall Cygwin.  At the moment this can be
accomplished via "umount -A" (and "cygrunsrv -R") with zero knowledge of
where the mount table happens to be implemented, or that there even is a
registry.

With the mount table no longer stored in the registry, we will have no
way around saying "you must remove such-and-such key from the
registry."  This is bad, IMHO.

A separate issue is that even if we document a proper procedure for
removing Cygwin (as we currently do in the FAQ), users often don't read
manuals and just plunge in and try to delete stuff.  They fail because
they forgot to stop services first, or because they don't understand
file ACLs.  By adding a new key in the registry we make it increasingly
likely that in their fervor to dive in and delete things without reading
docs they will miss yet another step.  I want to finally solve this for
good by making a program that encapsulates all this knowledge so that
they don't have to RTFM, and so that they just have a nice thing to
click on like every other well-behaved Windows program.  I think it's
been a requested feature more than once.

Brian

Re: [HEADSUP] Let's start a Cygwin 1.7 release area

[Brian Dessent]

Corinna Vinschen wrote:

> it just occured to me that I need the exact key name which setup will
> use to store the root directory of the installation.  I nedd it to get
> cygcheck working, even if it's not installed into the same path as the
> Cygwin DLL.
> 
>   "{HKLM,HKCU}\Software\Cygwin\???"
> 
> ??? = "Setup\root" maybe?  Or with a version number?

A good question.  I haven't implemented anything yet, so we can choose
whatever we want.  I wonder if it's really necessary to have a whole
subkey for this since at the moment there is only the one value that
setup will read.  How about simply "\Software\Cygwin\rootdir"?

That reminds me of something... I've always been a bit annoyed that
people always resort to manually editing the registry to remove Cygwin,
when umount is the documented and preferred method.  But now, we will
have no such utility to suggest and so to remove Cygwin the user will be
forced to use regedit.  Combined with the fact that most users trip over
the fact that sometimes ACLs on files like /var/log/sshd.log require
them to take ownership before deleting, I would like to be able to offer
an uninstaller.  This would handle checking for running Cygwin processes
and prompting to exit them (or terminating them), stopping and
uninstalling all Cygwin services, deleting the entire Cygwin tree,
removing start menu and desktop icons, removing everything from the
registry, and whatever else is necessary.  It would be the nuclear
option, deleting without remorse all traces with one click; I think
users want this.  It would of course be a standalone MinGW app like
setup.exe, except that it would get installed somewhere in the tree, and
added to the add/remove applet.  Anyway, SHTDI and everything but I just
want to float the idea to gauge response.

Brian

Re: Who maintains tcltk? I have a patch for you.

[Brian Dessent]

Eric Toombs wrote:

> Oh ok. So in other words, the source is provided as a reference and not
> necessarily for building the package again.

It is provided for building the package again in the same context that
it was originally built in.  You can't change variables like the
versions of libraries used and not expect minor differences; if you used
the version of the Cygwin library that was current in Feb 2006 at the
time the package was created it would build OOTB.

> You mean in the setup, I install the latest version of the gnu debugger
> and that will allow tcl to build out of the box? What does gdb have to
> do with tcl?

The packages are related in that the tcltk package is the specific copy
of tcl/tk that is bundled with the insight source tree, not a general
purpose "tcl/tk in its own right" -- it exists for the sole purpose of
supporting insight.  Insight is packaged with gdb, as it's really just
an enhanced version of gdb.

But I had incorrectly assumed that since these two packages are related
that with the new gdb release there would be a corresponding new tcltk
release, but I now see that is not the case and the old 20060202 version
of tcltk remains current despite insight being updated to 6.8-2.

Brian

Re: Who maintains tcltk? I have a patch for you.

[Brian Dessent]

Eric Toombs wrote:

> Well then it should probably be included with the source tree that is
> setup.exe downloads by default, unless source distributions are
> generally not expected to compile out of the box.

You're looking at an old version from 2006.  How can you possibly expect
that to contain a patch from 2007?  The source package contains the
source as it existed when the package was built, and when that package
was built in 2006 it did build correctly OOTB.  By trying to build it
against current Cygwin you remove that context, and it's not unexpected
for errors to pop up as a result.  Until they invent time travel, it
will not be possible to write code that is immune to future external
changes.

It would be very wrong for the source package to not correlate exactly
to what was used to build the binary package, as that would violate the
GPL and common sense.

Anyway, upgrade to the current gdb-6.8-2 and it will include the patch
and should build OOTB.

Brian

Re: Who maintains tcltk? I have a patch for you.

[Brian Dessent]

Eric Toombs wrote:

> I apologize in advance if this is the wrong place to post this. I would
> have mailed the package's maintainers directly but the "MAINTAINERS"
> file in the source tree appeared to belong to gcc's source tree, along
> with every other piece of documentation at the top of the source
> tree--kinda weird.

The toplevel is shared between a number of projects: gcc, binutils,
gdb/insight/sim, newlib, cygwin, etc.  They are all just portions of one
large tree, although with gcc using SVN now they are physically two
trees, but in reality it's still one large logical tree.  A tarball
release for each individual project contains a subset of the whole tree,
toplevel+some number of subdirs.

The tcl dirs are part of insight, so patches should go to the insight
mailing list.

> --- tcl/win/tclWinPipe.c.old2006-02-02 14:47:17.00100 -0500
> +++ tcl/win/tclWinPipe.c2008-04-20 00:28:05.62500 -0400
> @@ -1229,7 +1229,7 @@
>  BuildCommandLine(execPath, argc, argv, &cmdLine);
> 
>  #ifdef __CYGWIN__
> -cygwin_internal (CW_SETUP_WINENV);
> +cygwin_internal (CW_SYNC_WINENV);
>  #endif
> 
>  if ((*tclWinProcs->createProcessProc)(NULL,

This has been fixed for some time already:
.

Brian

Re: setup-1.7 defaults to "Just me"

[Brian Dessent]

Corinna Vinschen wrote:

> I mean, token_info was an UCHAR array, so it's not aligned while
> the token_group information might require a 4 or 8 byte alignment
> which you now get by chance.  Probably you'd be better off not
> using `char buf[size]' but
> 
>   PTOKEN_GROUPS groups = (PTOKEN_GROUPS) alloca (size);

Interesting:

#include 
#include 

int main (int argc, char **argv)
{
  
  char vla[argc * 250];
  char *alloca_array = (char *) alloca (argc * 250);
  char normal_array[1024];
  
  printf ("alignof(vla) = %d\n", __alignof__ (vla));
  printf ("alignof(alloca_array) = %d\n", __alignof__ (alloca_array));
  printf ("alignof(normal_array) = %d\n", __alignof__ (normal_array));
  return 0;
}

Returns:

alignof(vla) = 1
alignof(alloca_array) = 4
alignof(normal_array) = 1

So, perhaps this is working now by accident?

> for the GetTokenInformation call.  However, it also didn't say anything
> about alignment requirements of ZwQueryDirectoryFile but it failed
> on W2K for that reason nevertheless.

Don't you love discovering these little pearls of joy inside the Win32
API?  They're like little rays of sunshine.  No wait, the opposite.

Brian

Re: setup-1.7 defaults to "Just me"

[Brian Dessent]

Corinna Vinschen wrote:

> Alignment?

Perhaps.  Do you know if gcc aligns VLAs higher than their natural
alignment?  The nonworking code was just "UCHAR foo[1024]" whereas the
working was "char buf[size]", which I cribbed from
grp.cc:internal_getgroups().

Brian

Re: setup-1.7 defaults to "Just me"

[Brian Dessent]

Corinna Vinschen wrote:

> Not really, no.  1024 bytes should be big enough in most cases.  Did you
> look for the desired size which now gets returned by the NULL call?

Yes, it was in the neighborhood of 230 bytes or so.  It wasn't that the
buffer was too small, and I would have expected
ERROR_INSUFFICIENT_BUFFER anyway in that case.  It was just something
about the buffer that it didn't like.

Brian

Re: setup-1.7 defaults to "Just me"

[Brian Dessent]

Corinna Vinschen wrote:

> I just found that, regardless of my privileges, setup-1.7 defaults
> to install for "just me" instead of "all users" while the standard
> setup defaults to "all users".  Why does that happen?

This is very odd.  Setup's is_admin() was returning 0 despite the user
belonging to the administrators group, and debugging revealed that the
cause was the following:

 // Get the group token information
 UCHAR token_info[1024];
 PTOKEN_GROUPS groups = (PTOKEN_GROUPS) token_info;
 DWORD token_info_len = sizeof (token_info);
 status =
   GetTokenInformation (token, TokenGroups, token_info, token_info_len,
 &token_info_len);
 CloseHandle (token);
 if (!status)
   return 0;

For whatever reason, GetTokenInformation was failing and returning
ERROR_NOACCESS when passed a 1024 byte buffer.  I changed the code to
first call it with NULL to get the desired size, and that seems to have
satisfied it.  I cannot explain why it would object to a 1024 byte
buffer, maybe you have some idea.

I'll install an updated setup-1.7.exe in a moment.

Brian

Re: [HEADSUP] Let's start a Cygwin 1.7 release area

[Brian Dessent]

Eric Blake wrote:

> I hate unnecessary spaces.  Can we go with "Cygwin" rather than "Cygwin
> Project", so that scripts using /proc/registry don't have to worry about
> the space?

The only reason I suggested "Cygwin Project\Cygwin" is that it's
supposed to be a two level hierarchy, company\product, matching the
current "Cygnus Solutions\Cygwin" arrangement.

However, after a quick jaunt through HKLM\Software on this machine I see
that there are a good number of programs that forego the two level
custom and just put keys under HKLM\Software\Foo.  So, with that in
mind, are there any objections to simply HK{LM,CU}\Software\Cygwin?

Brian

Re: [HEADSUP] Start of Cygwin 1.7 release cycle

[Brian Dessent]

Corinna Vinschen wrote:

> I see.  So what we have in newlib is how it's defined on Linux.
> Howver, shouldn't autogen have the same problem on Linux then?
> If not, any idea why?

I suppose it's because on linux, HAVE_FOPENCOOKIE would be set and this
code would be skipped.  It was only when HAVE_FOPENCOOKIE was not
defined but HAVE_FUNOPEN was that it tried to include the typedefs.  The
HAVE_FOPENCOOKIE codepath was unusable on Cygwin because it assumed
presence of  when fopencookie() is present, and while newlib
has fopencookie() it has no such libio.h.

I haven't looked at CVS autogen but Bruce said he was going to fix both
of these issues so hopefully this is all resolved now.

Brian

Re: [HEADSUP] Start of Cygwin 1.7 release cycle

[Brian Dessent]

Corinna Vinschen wrote:

> They do?  How and Why?  Is that something which should be rather fixed
> in newlib instead of in the autogen configuration?

The BSD implementation of funopen() doesn't explicitly define any types
for the cookie functions, but simply says they should match the
signatures of read(2), write(2), lseek(2), and close(2).  Autogen tried
to define the following if it detected that funopen() exists:

 typedef int (cookie_read_function_t )(void *, char *, int);
 typedef int (cookie_write_function_t)(void *, const char *, int);
 typedef fpos_t  (cookie_seek_function_t )(void *, fpos_t, int);
 typedef int (cookie_close_function_t)(void *);

However the newlib implementation explicitly defines these types as:

typedef ssize_t cookie_read_function_t(void *__cookie, char *__buf, size_t __n);
typedef ssize_t cookie_write_function_t(void *__cookie, const char *__buf, 
size_t __n);
# ifdef __LARGE64_FILES
typedef int cookie_seek_function_t(void *__cookie, _off64_t *__off, int 
__whence);
# else
typedef int cookie_seek_function_t(void *__cookie, off_t *__off, int __whence);
# endif /* !__LARGE64_FILES */
typedef int cookie_close_function_t(void *__cookie);

So you got an error because the types differ.  I don't see anything
wrong with the newlib definitions here, as they match the prototypes
of read/write/etc.  I'm not sure why autogen was trying to define them
using int instead of size_t or ssize_t, but that's what it was doing
and it was apparently succeeding because the BSD headers didn't have
any typedefs.

Brian

Re: [HEADSUP] Let's start a Cygwin 1.7 release area

[Brian Dessent]

Corinna Vinschen wrote:

> Having said that, should we really rename the registry keys, what do we
> do with the "Program Options" and the two "heap_foo" values?  Should
> they be moved to the new registry key?  If yes, should the postinstall
> script I created a couple of days ago also move them?

For the heap_chunk settings, couldn't they simply be moved into $CYGWIN?

Brian

Re: [HEADSUP] Start of Cygwin 1.7 release cycle

[Brian Dessent]

Andrew Schulman wrote:

> (1) Do all packages that include compiled code need to be rebuilt for Cygwin
> 1.7?  IOW, is ABI compatibility broken from 1.5?  Also, I presume that there
> would be no need to rebuild any packages that don't include compiled code, 
> e.g.
> packages that depend only on Perl or Python.

No, it should be fully ABI backwards compatible, it's just that
recompilation will be required to enable the new features.  For example
if the app allocated its path buffers based on PATH_MAX (260 in 1.5,
4096 in 1.7), it will not be able to take advantage of long pathnames
even if the DLL would support it.

And likewise for ipv6 support, fopencookie, etc.  Those features would
not have been detected by the package's configure script and thus
disabled in 1.5, but they would be detected and enabled when rebuilt
against 1.7.  And it's that fact (new codepaths exposed in the app) that
needs testing.

> (2) If I understand right, the implication here seems to be that although 
> Cygwin
> 1.7 will support the above features, it's going to be up to us package
> maintainers to ensure, or at least just test, that our packages support them
> too.  But, there seems to be no requirement about that.  That is, we could 
> just
> dump our packages as they are into Cygwin 1.7, and not test them for any of 
> the
> new supported features, assuming our consciences allow this.  Correct?

Well, part of the implication of the above is that some packages might
not build against 1.7 without tweaks.  For example I recently tracked
down a configure issue in autogen where it assumed the BSD signatures of
the types used with funopen(), which differ from the implementation in
newlib.  funopen/fopencookie didn't exist in 1.5 and are new in 1.7, so
these issues would have never been exposed when building the package
against 1.5, but the new feature exposed the codepath in autogen that
tried to use funopen, which needed some tweaks to work with Cygwin.

In my completely unofficial opinion I'd say that it's always at the
maintainer's judgement how to deal with these things.  In most cases if
a feature does not work you can simply disable it by configuring with
--disable-foo, or e.g. in the case of autogen by configuring with
"ac_cv_func_fopencookie=no ac_cv_func_funopen=no ./configure ..." forced
those features to be disabled since the configure tests were broken. 
But ideally it would be best if whatever compatibility problems arise
can be dealt with so that the package builds as close to OOTB as
possible, and so that the new functionality is enabled in the app.

> (3) For packages that have been tested, are we going to have a standard and/or
> common location or format in which to put our "tested against Cygwin 
> 1.7/tested
> for long file name/UTF-8/IPv6/etc. support" results?  Or just note the results
> or not in our README.Cygwin files?  The latter seems fine to me, I'm just
> wondering.

I can't speak for anyone else but the way I invisioned this working is:

- maintainer uploads 1.7-built package to the 1.7 release area
- other 1.7 testers install it in their 1.7 tree
- problems are reported to this list
- maintainers fix the issues, upload updated packages to the 1.7 release
area
- after sufficient iterations of this, the 1.7 area is declared ready
for prime time, and is switched over to the default distro instead of
being a sandboxed area

Of course if you want to note in the README.Cygwin the changes/tweaks
that were necessary for each package bump, that would probably help
everyone.  But the fact that the package exists in the 1.7 area at the
time that it goes live would be the main indicator that it's passed
somebody's definition of (at least minimal) working.  In other words, if
there's some really hairy unsolvable problem with a 1.7-built package,
it should be removed from the 1.7 area and the old 1.5-built version
used until the problem is fully resolved.

> (4) It seems that it might be useful to develop standard unit tests for some 
> of
> these features.  Long file name support is simple enough-- we just have to
> create some path names longer than 260 characters, and try feeding them to our
> applications.  But for UTF-8 and IPv6 support, it could take each of us a lot 
> of
> time to work out tests on our own.  Or at least, it would take me a while 
> since
> I'm mostly ignorant of them...  A standard, documented approach would sure 
> help
> me, and maybe other packagers too.  I freely admit though that I am not
> volunteering to do this.

Yes, I'm sure all of those things would be helpful.  We don't have to be
perfect either.  We should be realistic in that the number of people
that are clued enough to monitor this list and download a separate
setup.exe and follow the steps to install an experimental distro is
going to be much smaller than the general userbase of Cygwin, so we
won't be able to suss out every problem before the 1.7 area goes live. 
I think that's fine, we just need to get things into a goo

Re: [HEADSUP] Let's start a Cygwin 1.7 release area

[Brian Dessent]

Igor Peshansky wrote:

> But one thing to consider is that by adding a registry key, we're setting
> things up for one dominant installation of Cygwin.  So people juggling
> multiple installations really *will* have to go edit the registry to
> switch -- no more nice "mount -m" approach, since the mounts will no
> longer apply anyway.

Can you clarify what you mean here?  I don't understand.  You're talking
about maintaining separate installs, but part of the plan for setup.exe
that I outlined supports this directly by the fact that if you change
the value of the root install dir at step 2, setup then forgets anything
it knew about the previous value, enabling you to keep around as many
parallel installs as you want.

Or do you mean keeping around an old-style 1.5 tree and a new-style 1.7
tree simultaneously?  In that case you'd be using the old setup.exe
(which doesn't know a thing about $newkey) for the 1.5 tree so I still
don't see any conflict.  Where does the need to edit the registry
manually arise?

Brian

Re: [HEADSUP] Let's start a Cygwin 1.7 release area

[Brian Dessent]

Christopher Faylor wrote:

> Yes, I know.  I just don't think it clarifies anything to put a "Red
> Hat" in the registry.

I was thinking "Cygwin" would be better as well, but since it is
supposed to be a two-level heirarchy how about
"HK{LM,CU}\Software\Cygwin Project\Cygwin".  It has always seemed to me
that we actively try to de-emphasize any association that Red Hat has in
the actual day-to-day operation of the project, other than owning the
copyrights and having their own commercial fork.  Likewise with the
"Company ID" in the resource strings for cygwin1.dll, listing Red Hat
always seemed a bit off to me, but I recognise that if we have to list a
real company that Red Hat is the obvious choice.

Brian

Re: [HEADSUP] Let's start a Cygwin 1.7 release area

[Brian Dessent]

Corinna Vinschen wrote:

> I also found, when setup doesn't have the mount points it doesn't seem
> to be able to take the /etc/setup/installed.db from the path entered as
> root directory.  It installs happily all base packages again a second
> time.
>
> Probably we will still need *some* registry entry.  Setup could create
> one for its own purpose, just to find the installation path.  Maybe the
> registry value should use a version number, so that we can have two
> parallel installation again, should we ever release a -3 distro...

Here's what I had in mind for the logic:

On startup, setup looks for $newkey.  If found, it uses the Win32 path
stored there as the starting value for the root dir/install location. 
If not found, it looks at $oldkey.  If that's not found, it uses a
default of c:\cygwin.

In any case, if the user changes this value on the second step of the
installer, that new value becomes the effective value immediately;
anything obtained from the first value should be discarded.  Access to
files in the POSIX namespace is done purely relative to this root, e.g.
setup creates an internal mount table of its own that maps the install
dir to "/" (and for the time being duplicating /{bin,lib} to
/usr/{bin,lib}, subject to change as well.)  At the end of setup, the
effective value of the root dir is written out to $newkey.

Where:
  newkey = some new key whose location is yet to be decided
  oldkey = the location corresponding to the root dir of the current
registry mount table

In both of the above there is the implicit notion of looking first in
HKLM\Software and then falling back to HKCU\Software if not found,
allowing the non-admin user to install Cygwin as they please.  The
location where it was found is used as the starting value for
justme/allusers, and if the user modifies the setting during the run the
resulting destination at the end is changed accordingly.

The above is meant to achieve the following objectives:

- the setup tool and other 3rd party tools have a key telling if Cygwin
is installed and where
- at startup setup can find its last-* from the last run and use them as
defaults so that the user doesn't have to re-enter them every time
- installing multiple trees becomes a matter of simply changing the root
dir at step 2 of the install process

Unresolved issues with this plan:

- What are we going to do about text/binary mode?  To maintain the
setting, setup would have to be taught to parse/write fstab.  Ugh.  Plan
B would be to say that if you want textmode you have to edit fstab
yourself.  That has the advantage of making it harder to use textmode,
which leads to fewer bugreports.  On the other hand, the small army of
insane people that do actually use textmode will probably be mad that
they have to manually edit config files (the horror!) because the setup
tool no longer has a choice.

- Do we really want to pick a new key for $newkey, or wedge it into the
same existing location somehow?  I admit that I do find it a bit silly
that Cygwin still installs under "Cygnus Solutions", and since a 1.7 DLL
will not even look at the registry I guess it is proper to move to a
totally new key for this setting.  And of course for the 1.7 testing
period we'd want it to be separate.  But I mean long term, are we making
3rd parties lives easier or harder by having two totally different
places/formats to check for an existing install of Cygwin?

Brian

Re: [HEADSUP] Let's start a Cygwin 1.7 release area

[Brian Dessent]

Corinna Vinschen wrote:

> When I start setup-1.7, it seems to find the already existing c:/cygwin
> installation.  Even though I choose "C:/cygwin-2" as installation path,
> the new setup-2.ini content is compared against the already existing
> installation.  The result is that setup-1.7 offers to install only
> the single changed package:  cygwin itself.

Right, it reads /etc/setup/installed.db (where /etc is derived from the
mount table) to see what the current versions are.

> Why is setup using the information from the existing installation
> even if I choose to install into another directory?  Is it because
> the mount points in the registry are looked at?  That's not correct
> anymore, given that the registry mount points are not used for the
> new installation anyway.
> 
> Is it possible to change setup so that it ignores the registry mount
> points?  And it must nort write new ones either, btw...

Oh.  Teaching it not to read or write to the mount table in the registry
is going to be a lot more work than just telling it to read a different
.ini file.  I'll see what I can do, but until then I think we'll just
have to tell people to do the switching manually (mount -m/umount -A).

Brian

Re: [HEADSUP] Let's start a Cygwin 1.7 release area

[Brian Dessent]

Corinna Vinschen wrote:

> Brian, could you please prepare a temporary setup-1.7.exe which
> only downloads from the release-2 area?

Okay, I uploaded  which fetches
setup-2.{ini,bz2}.  It identifies itself as version 2.590-1.7special on
the splash page so that we can tell it apart from the normal.

Brian

Re: [wwwdocs patch] mention cygport as "method three"

[Brian Dessent]

Christopher Faylor wrote:

> Brian would you mind making another change to setup.html and see if it
> is auto-updated?

It seems to be working fine now.  The file in sourceware/htdocs/cygwin
is automatically updated and is mode 775, group cygwin.

Brian

Re: [wwwdocs patch] mention cygport as "method three"

[Brian Dessent]

Corinna Vinschen wrote:

> Thanks, please apply.

I checked in the change a day or two ago
(http://sourceware.org/cgi-bin/cvsweb.cgi/htdocs/setup.html?cvsroot=cygwin)
but the live copy (http://www.cygwin.com/setup.html) still has not
updated.  Is the automatic trigger script broken?

Brian

Re: setup.exe missing mgwbz2-1.dll?

[Brian Dessent]

"Dr. Frank Lee" wrote:

> Building from CVS goes cleanly but I am unable to run the resulting
> binary from cmd.exe:
> 
> "setup.exe - Unable to Locate Component
> This application has failed to start because mgwbz2-1.dll was not found.
> Re-installing the application may fix this"
> 
> If I run this from cygwin itself the application runs correctly,
> loading mgwbz2-1.dll from /bin. (Verified by renaming that dll file:
> setup.exe fails to start, error code 53.)

You're right, it should have linked with the static libbz2 and zlib.  I
made a change in this area recently and didn't notice anything different
because I was using a cross compiler that doesn't have shared versions
of those libs in its sysroot.  I just committed a fix that should take
care of it.

> I was expecting a stand-alone setup.exe - was that incorrect of me? I see
> setup/bz2lib contains only a CVS directory and nothing else, if that's
> relevant.

Setup used to have a bundled libbz2, but that was before there was a
MinGW libbz2 in the distro, so now we just use that.

Brian

Re: Patch for unattended setup (updated)

[Brian Dessent]

"Dr. Frank Lee" wrote:

> 'which flex' suggests that flex is installed in /usr/bin/flex. 'which
> bison' suggests /usr/bin/bison is installed.

Did you add those packages after configure was run?  If so you'll need
to re-run configure (or delete config.cache and run ./config.status
--recheck) otherwise the autoconf variables will still be setup to
specify invoking the 'missing' script in place of flex/bison.

This 'missing' script is a feature of autoconf meant to help in
situations where the generated file is in fact distributed to the user
in the tarball (i.e. it doesn't need to be remade), but the timestamps
are not correctly preserved by the user or the filesystem in use, such
that 'make' thinks it needs to be remade.  Autoconf substitutes the
'missing' script in place of developer tools that the configure script
found to be missing, in the hope that such an attempt to regenerate the
file will succeed despite the missing tool, given that a sufficiently
up-to-date generated file does exist.

But none of the above applies in this case because the generated file
really doesn't exist, it's not just a case of timestamps being
incorrect.

Brian

Re: Patch for unattended setup (updated)

[Brian Dessent]

"Dr. Frank Lee" wrote:

> I'm trying to compile from CVS and running into a few problems. Might I
> pick your brains, please?

Sure, but please use the mailing list rather than sending to me
directly.

> I see I should install some packages, which I do. I run bootstrap.sh which
> appears to run correctly, and ./configure with a number of options which
> appears to run fine. And then I run 'make', resulting in the command:
> test -f inilex.cc || /bin/sh ./cfgaux/ylwrap inilex.ll .c inilex.cc --
> flex  -8
> failing with exit code 1.
> 
> This is a clean install of cygwin and I'm rather at a loss to diagnose
> much further. inilex.cc certainly doesn't exist, so it seems the ylwrap
> script is failing but this is starting to get beyond my area of
> expertise...

inilex.cc is generated from inilex.ll by flex.  The ylwrap script is a
wrapper for invoking flex and handling the renaming of the output file
to the correct name.  I appears you don't have flex installed though, in
which case ylwrap can't do anything.

I see that README doesn't list flex as required, which I'll fix shortly.

BTW there is a similar thing with iniparse.cc being generated from
iniparse.yy with bison, which is also required when building from CVS
and not currently listed in README.

Brian

Re: Patch for unattended setup (updated)

[Brian Dessent]

"Dr. Frank Lee" wrote:

> I attach a re-working of my previous patch to setup.exe to support the '-p
> package1[,package2[...]]' command line option. This patch also intercepts
> all
> message boxes and returns some default values to the calling function for
> unattended installations. A patched setup.exe will return
> IDS_REBOOT_REQUIRED
> (=118) if a reboot seems desirable.
> 
> This patch applies to setup version 2.578 (the most recent I could
> compile).
> 
> (http://www.sp.phy.cam.ac.uk/~rl201/cygwin-setup-unattended-patch.diff
> contains
> this patch and http://www.sp.phy.cam.ac.uk/~rl201/setup-patched.exe the
> resulting executable - usual caveats about running such binaries apply!)

Sorry but there are a number of problems with this.

1. Please don't send a patch with CRLF line endings when the file has LF
line endings.

2. Patches need to be against current CVS, not an 8 month old version. 
For example, CVS setup has already had a -p option (for specifying proxy
options) for several months, so you need to pick a different name.  Or
stated differently: if, when applied to HEAD, your patch won't compile
then it's a serious problem with the patch that needs fixing, since it
can only be applied to HEAD.  I just uploaded a 2.588 snapshot yesterday
so I know that HEAD builds fine.

3. You need to follow the the GNU coding style, e.g. opening brace on
its own line, 2 space indents, space before '(' and on both sides of
'='.  I know that not all of setup is consistent this way, but it
doesn't help to make the problem worse by introducing more.

4. There is no ChangeLog entry.

5. This looks like a messy work in progress; hunks like these are
unacceptable:

-static bool rebootneeded;
+// RFL static bool rebootneeded;

+// RFL
+#include "state.h"
+

+  //case MB_CANCELTRYCONTINUE:
+  //  return IDCONTINUE;
+  //  break;

+  // RFL suspects this code should really be in the _custom_MessageBox
routine but code
+  // placed there doesn't get executed (for the 'Files in-use have been
replaced' message,
+  // at least.
+  if (unattended_mode) {

In other words, if something needs to be removed, it should be removed
not commented out.  Also, patches should not have marker/vanity
comments.

Brian

Re: [HEADSUP] Let's start a Cygwin 1.7 release area

[Brian Dessent]

Corinna Vinschen wrote:

> My favorite would be release-2.  It has nothing to do with the DLL

Sounds fine to me as well.

Brian

Re: [HEADSUP] Let's start a Cygwin 1.7 release area

[Brian Dessent]

Corinna Vinschen wrote:

> Yes.  I'm just wondering if we could do without the final rename of the
> directories.  The Cygwin release area isn't that small, > 4 Gigs.  A
> simple top-level directory rename will force all mirrors world-wide to
> copy the whole area.  Twice, because we will have two release areas.
> 
> If that's a minor concern, we do as above.  Otherwise, I'd suggest to
> create a "release-2" area and to update just setup.exe accordingly.

We can still rename the .ini files without renaming the directory trees,
it's all just a matter of how the upset cron job runs.  I'm not sure if
that really buys any clarity, but what it does do is allow us to "swap
in" the entire staging area atomically once 1.7 is released.  Otherwise,
only users who updated their setup.exe would ever see the 1.7 release. 
I can almost see the email messages now: "Is Cygwin dead?  I haven't
seen a new release of the DLL or any updated packages in the last 
months?!?!"

Brian

[wwwdocs patch] mention cygport as "method three"

[Brian Dessent]

This is a patch for <http://cygwin.com/setup.html> to officially mention
cygport as acceptible.

Brian2008-04-08  Brian Dessent  <[EMAIL PROTECTED]>

* setup.html: Mention cygport as method three.

Index: setup.html
===
RCS file: /cvs/cygwin/htdocs/setup.html,v
retrieving revision 1.105
diff -u -p -r1.105 setup.html
--- setup.html  26 Sep 2007 09:42:02 -  1.105
+++ setup.html  9 Apr 2008 03:24:38 -
@@ -362,7 +362,7 @@ etc...
Ensure that your package handles binary only systems, textmode only 
systems, and hybrid systems correctly. 
   
   Package 
Source
-  There are two accepted ways to package the source code for Cygwin 
packages.
+  There are three accepted ways to package the source code for Cygwin 
packages.
   Method One
   

@@ -420,6 +420,7 @@ boffo-1.0-1/CYGWIN-PATCHES/setup.hint

   
   Method Two
+  This method is sometimes referred to as the "g-b-s" method, after the 
filename of the http://sourceware.org/cgi-bin/cvsweb.cgi/~checkout~/packaging/templates/generic-build-script?content-type=text/plain&cvsroot=cygwin-apps";>generic-build-script
 template.
   
In a packaging technique inspired by rpms and debs, you may create 
a -src tarball which simply contains:
  
@@ -466,6 +467,10 @@ tar xvjf ~/sources/boffo-1.0-1-src.tar.b
  

   
+  Method Three: cygport
+  The technique of method two became popular to many maintainers, 
however it suffers from a number of drawbacks when applied on a wide scale.  
The cygport http://cygwin-ports.cvs.sourceforge.net/*checkout*/cygwin-ports/cygport/README";>README
 explains a number of these problems.
+  The cygport framework is a response to these issues, and 
borrows concepts from the Gentoo portage system.  It separates the g-b-s into a 
small file containing the package-specific parts and moves the main script 
infrastructure into shared files.  For more information on using 
cygport consult the documentation and sample port files.
+  Source packages created with cygport have a similar 
structure to those created with method two, except that they contain a 
'boffo-1.0-1.cygport' file in place of the 'boffo-1.0-1.sh' 
script.  The binary package is built by running 'cygport boffo-1.0-1 
all' instead of './boffo-1.0-1.sh all', and so on for 
prep, compile, package, finish, etc.
   Creating a package 
postinstall script
   If your package requires certain commands to be executed after the 
files in the package are installed, include them in a file in the package 
called /etc/postinstall/package.sh or 
/etc/postinstall/package.bat.
   If the file's name ends in ".sh", it is executed with the Cygwin 
shell; if it ends in ".bat", it is executed with the DOS command interpreter. 
If it doesn't end with either of these suffixes, it is ignored.

Re: [ITA] libXft/libXft2/libXft-devel-2.1.7: A client-side font API for X applications

[Brian Dessent]

Brian Dessent wrote:

> As Angelo tried to report on the wrong mailing list, this is totally
> broken.  How are you supposed to compile any X11 apps after installing
> this package, without manually adding -I/usr/X11R6/include/X11 to
> CPPFLAGS?  Simply installing libXft-devel causes all the X11 headers to
> no longer work, because the only thing left under /usr/include/X11 is
> now Xft/.

Furthermore, with the current situation, if you install libXft-devel and
then reinstall xorg-x11-devel, the libXft headers (and anything else you
might have installed in the directory /usr/include/X11) are deleted when
the symlink is remade!  That is clearly broken.  You can't have two
packages sharing ownership of a directory like that.  Please fix.

Brian

Re: [ITA] libXft/libXft2/libXft-devel-2.1.7: A client-side font API for X applications

[Brian Dessent]

"Dr. Volker Zell" wrote:

> This package likes to install the header files to
> 
>   /usr/include/X11/Xft/*.h
> 
> xorg-x11-devel installs a symlink
> 
>   /usr/include/X11 -> ../X11R6/include/X11
> 
> Given that the X11 hierarchy is deprecated, I like to remove this
> symlink in a preremove script.
> 
>  preremove --
> #!/bin/bash
> 
> if [ -L /usr/include/X11 ] ; then
>   rm -f /usr/include/X11
> fi
>  preremove --

As Angelo tried to report on the wrong mailing list, this is totally
broken.  How are you supposed to compile any X11 apps after installing
this package, without manually adding -I/usr/X11R6/include/X11 to
CPPFLAGS?  Simply installing libXft-devel causes all the X11 headers to
no longer work, because the only thing left under /usr/include/X11 is
now Xft/.

It doesn't make any sense to try to deprecate the X11R6 tree until we
actually have working xorg-* packages that use the new layout.  libXft
should not and does not "own" /usr/include/X11, so it's very surprising
that installing an ancillary library causes a bunch of headers to move. 
xorg-x11-devel should "own" that directory and when we get an X11
maintainer that package should be responsible for moving it out of the
X11R6 space, but not until then, and not by a random library.

Brian

Re: [ITA] fftw3

[Brian Dessent]

Brian Dessent wrote:
> 
> Charles Wilson wrote:
> 
> > As no one was actually able to install this file, is it OK to replace it
> > on sourceware (perhaps deleting the .md5sum file) without bumping the
> > version number?
> 
> Yes, I considered doing that myself.  Sounds like a good idea.

Crap.  It looks like there are a lot of these on sourceware:

[EMAIL PROTECTED] release]$ find . -name \*.tar.bz2 -size -45c -printf
"%s %P\n"
14 GNOME/libbonobo2/libbonobo20/libbonobo20-2.10.1-1.tar.bz2
14 GNOME/libbonobo2/libbonobo20/libbonobo20-2.14.0-1.tar.bz2
14 GNOME/gnome-libs/gnome-libs-1.4.2-2.tar.bz2
14 jpeg/libjpeg-devel/libjpeg-devel-6b-11.tar.bz2
14 autoconf/autoconf-devel/autoconf-devel-2.59-2.tar.bz2
14 autoconf/autoconf-devel/autoconf-devel-2.59-2-src.tar.bz2
14 autoconf/autoconf-stable/autoconf-stable-2.13-6-src.tar.bz2
14 autoconf/autoconf-stable/autoconf-stable-2.13-6.tar.bz2
14 pkgconfig/pkgconfig-0.17.2-3-src.tar.bz2
14 pkgconfig/pkgconfig-0.17.2-3.tar.bz2
14 automake/automake-devel/automake-devel-1.9.2-2-src.tar.bz2
14 automake/automake-devel/automake-devel-1.9.2-2.tar.bz2
14 automake/automake-1.7.9-2.tar.bz2
14 automake/automake-stable/automake-stable-1.4p6-3-src.tar.bz2
14 automake/automake-stable/automake-stable-1.4p6-3.tar.bz2
14 automake/automake-1.7.9-2-src.tar.bz2
14 _obsolete/more/more-2.11o-3.tar.bz2
14 _obsolete/more/more-2.11o-3-src.tar.bz2
14 _obsolete/agetty/agetty-2.1-2.tar.bz2
14 _obsolete/agetty/agetty-2.1-2-src.tar.bz2
14 _obsolete/libtool/libtool-devel/libtool-devel-1.5.10-2-src.tar.bz2
14 _obsolete/libtool/libtool-devel/libtool-devel-1.5.10-2.tar.bz2
14 _obsolete/libtool/libtool-1.5b-2.tar.bz2
14 _obsolete/libtool/libtool-stable/libtool-stable-1.4.3-3-src.tar.bz2
14 _obsolete/libtool/libtool-stable/libtool-stable-1.4.3-3.tar.bz2
14 _obsolete/libtool/libtool-1.5b-2-src.tar.bz2
14 _obsolete/setsid/setsid-0.0-4.tar.bz2
14 _obsolete/setsid/setsid-0.0-4-src.tar.bz2
14 _obsolete/fftw3-dev/fftw3-dev-3.1.2-2.tar.bz2

The reason this hasn't been a widely reported problem is that the
message box for invalid tar files only exists in the setup snapshot
versions, not the released version.  But it will be after the next
release, whenever that is.

The feature was added because someone reported that a package was
mysteriously not installing, and after some debugging the reason turned
out to be that it had been created with some alternative tar-program
that used a different magic.  So setup was taught to recognize that new
magic, as well as report future failures explicitly rather than silently
ignoring the package.  However, setup cannot distinguish the difference
between EOF (i.e. 0 byte file) and magic mismatch.

I think it would be prudent to teach setup not to complain about bzipped
0-byte files, but I still feel that on principle if you call something
".tar.bz2" it had ought to be a valid tar file.  I'll work on a patch.

Brian