Re: Binary distributions

[Smylers]

On January 29th Tyler MacDonald wrote:

> Sébastien Aperghis-Tramoni <[EMAIL PROTECTED]> wrote:
> 
> > Did anybody here have played with CPANPLUS::Dist::Deb?
> >   http://search.cpan.org/dist/CPANPLUS-Dist-Deb/
> > 
> > Believing its documentation, it should build a valid Debian package
> > and take care of its dependencies (dunno if that means just listing
> > or actually adding them in the package).
> 
> This excited me, but even with all the debian helper utils installed
> and CPANPLUS on default settings running as root, it crashes not being
> able to find the right dist_cpan object out of CPANPLUS. :-(

For what it's worth, this now works for me, on Ubuntu 5.10.  I had to
install the March release of CPANPLUS (so it isn't surprising that it
didn't work in January!).

Note the version of CPANPLUS on debian.pkgs.cpan.org isn't recent enough
-- so you have to go outside of Apt and install CPANPLUS manually.

So if anybody is wanting .deb packages of Cpan distributions, this does
seem to be a reasonable way of doing it, and without having to know
anything about the internals or format of either sort of packages.

Basically you just need to do:

  $ cpan2dist --format CPANPLUS::Dist::Deb Crypt::GeneratePassword

then wait for cpan-libcrypt-generatepassword-perl_0.03-1_all.deb to
spring into existence.

Smylers

Re: Binary distributions

[Adam Kennedy]

Offer Kaye wrote:

On 2/6/06, Adam Kennedy wrote:

But I'm not really too worried any more, the CamelPack means it's much
easier not to just install from source than use the PPM system.

s/not/now/



Installing from souce == compiling every module that needs it... How
is that *easier* than installing a pre-compiled package?

Of course I'm assuming that the PPM method *works* as intended...
obviously it doesn't... but in an ideal world, all else being equal,
PPM would be easier, IMHO.


I didn't say it uses less resources, I said it's easier.

Installing from source is easier because I have some level of control if 
something goes wrong. I can contact the author or take over 
maintainership if needed.


With PPM I'm reliant on one company to create PPMs (in the default case) 
over which I have no control, and if there's a problem I don't really 
have any influence or ability to get it fixed.


Adam K

Re: Binary distributions

[demerphq]

On 2/5/06, Offer Kaye <[EMAIL PROTECTED]> wrote:
> BTW Gozer have you looked at the first line:
> Cannot forceunlink D:\cpanrun\build\5-8-0\lib\auto\List\Util\Util.dll:
> Permission denied at D:\cpanrun\build\5-8-0\lib/File/Find.pm line 874
>
> Maybe the script is trying to delete a file that the system thinks is
> in use? Or something similiar? Windows has that annoying habit of
> refusing to delete things it thinks are in use...

This is a known issue on Win32 that the MakeMaker people and the
Module Build people are aware of.

I've produced a patch for the install behaviour and I'm planning to
put together a patch for the uninstall behaviour Real Soon Now. Ive
also volunteered to maintain a new distro which would be split out of
the MakeMaker distro so that people can upgrade the install behaviour
without upgrading the whole MakeMaker package.

Randy Sims put together an initial version of this package including
the install patch at

   

Cheers,
Yves




--
perl -Mre=debug -e "/just|another|perl|hacker/"

Re: Binary distributions

[Offer Kaye]

On 2/6/06, Smylers wrote:
>
> So it seems the extra level of subdirectories are causing List::Util
> (and a whole bunch of other modules) not to show up in the main perl
> dist page.
>
> Is Cpan Search's heuristic for what gets included documented anywhere?

Now that I think about it, I seem to recall reading somewhere about
dual-life modules not being listed in the main Perl page. Dual-life
modules are those that are both core AND are distributed seperatly on
CPAN. Don't remember where I saw that, and don't know if it's true or
relevant here...

Cheers,
--
Offer Kaye

Re: Binary distributions

[Smylers]

Offer Kaye writes:

> I see what you mean... what threw me off was that [List::Util and
> Scalar::Util are] not listed under:
> http://search.cpan.org/dist/perl-5.8.8/

Well spotted!  List/Util.pm (including pod) is here:

  http://search.cpan.org/src/NWCLARK/perl-5.8.8/ext/List/Util/lib/List/Util.pm

Math::Complex (for instance) is included in the above list;
Math/Complex.pm is here:

  http://search.cpan.org/src/NWCLARK/perl-5.8.8/lib/Math/Complex.pm

So it seems the extra level of subdirectories are causing List::Util
(and a whole bunch of other modules) not to show up in the main perl
dist page.

Is Cpan Search's heuristic for what gets included documented anywhere?
In other words, is it that perl5.8.8 is failing in some way to meet some
published criteria for distributions to have modules listed; or is it
that no such spec exists and therefore perl5.8.8's dist is reasonable,
it's just that Cpan Search isn't sufficiently thorough in looking for
places where modules might be hiding?

I note that Kobes's Search doesn't have List::Util listed on the Perl
distro page either:

  http://cpan.uwinnipeg.ca/dist/perl

The code behind that is available, so we can presumably look to see what
the heuristic is ...

Smylers

Re: Binary distributions

[Smylers]

Offer Kaye writes:

> On 2/5/06, Offer Kaye wrote:
> 
> > [http://ppm.activestate.com/BuildStatus/5.8-windows/windows-5.8/Scalar-List-Util-1.15.txt
> 
> 
> Something funky here...  Last night I looked at "Scalar-List-Util"...
> but the correct name as Tyler said is "Scalar-List-Utils", with an "s"
> at the end.

If you have a look at the Backpan directory of Graham Barr (the
maintainer of Scalar-List-Utils) you'll see that mostly he's called the
distribution Scalar-List-Utils-version.tar.gz, but in May last year
version 1.15 was released as Scalar-List-Util-1.15.tar.gz, without the
"s".

This shouldn't matter: Perl dependencies are on modules, not on
distributions.  And the names of the modules didn't change when the
package was named differently.  If this is confusing ActiveState then
that sounds like a bug in their system.

Smylers

Re: Binary distributions

[A. Pagaltzis]

* Offer Kaye <[EMAIL PROTECTED]> [2006-02-06 09:15]:
>Installing from souce == compiling every module that needs it...
>How is that *easier* than installing a pre-compiled package?

You don’t need sit there turning a crank while the compiler does
its job. Does it take longer? Sure. Is it harder? No.

And if the alternative to compiling from source is not being able
to install the module in question at all, then it certainly is
“easier” in that sense.

Regards,
-- 
Aristotle Pagaltzis // 

Re: Binary distributions

[Offer Kaye]

On 2/6/06, Adam Kennedy wrote:
>
> > But I'm not really too worried any more, the CamelPack means it's much
> > easier not to just install from source than use the PPM system.
>
> s/not/now/
>

Installing from souce == compiling every module that needs it... How
is that *easier* than installing a pre-compiled package?

Of course I'm assuming that the PPM method *works* as intended...
obviously it doesn't... but in an ideal world, all else being equal,
PPM would be easier, IMHO.

Cheers,
--
Offer Kaye

Re: Binary distributions

[Adam Kennedy]

But I'm not really too worried any more, the CamelPack means it's much 
easier not to just install from source than use the PPM system.


s/not/now/

sigh

Adam K

Re: Binary distributions

[Adam Kennedy]

Tyler MacDonald wrote:

Offer Kaye <[EMAIL PROTECTED]> wrote:

Just an example, IO::All [1] version 0.33 has been available since Dec
17, 2004. It passed testing many times, at least according to its
testers page [2]. My default 5.8.7 ActivePerl distribution lists
IO::All version 0.17 .


According to
http://ppm.activestate.com/BuildStatus/5.8-windows/windows-5.8/IO-All-0.33.txt
it is not being built because Spiffy failed it's tests. According to
Spiffy's log, *it* did not build becuase Scalar-List-Utils failed to
build... Which is the same module that both Adam Kennedy and David Golden
are complaining about. (I wonder how many modules are being held back over
this one bug?)


Certainly hundreds. Even within just my modules it's several dozen, 
including the entire PPI tree, and everything related.


Including all recursion, possibly as many as 1000+ distributions are 
dead over that one problem.


And it keeps hanging around, and it keeps not getting fixed...

And the only thing that makes the situation worse, is that it never 
seems to be acknowledged, and we don't get any information.


But I'm not really too worried any more, the CamelPack means it's much 
easier not to just install from source than use the PPM system.


Adam K

Re: Binary distributions

[Offer Kaye]

On 2/6/06, Yitzchak Scott-Thoennes wrote:
>
> http://perldoc.perl.org/perl58delta.html#New-Modules-and-Pragmata
>

I see what you mean... what threw me off was that it is not listed under:
http://search.cpan.org/dist/perl-5.8.8/

Cheers,
--
Offer Kaye

Re: Binary distributions

[Yitzchak Scott-Thoennes]

On Mon, Feb 06, 2006 at 08:16:11AM +0200, Offer Kaye wrote:
> OT question - why is Scalar-List-Utils listed as "CORE"? It is not
> part of the Perl5 core

http://perldoc.perl.org/perl58delta.html#New-Modules-and-Pragmata

Re: Binary distributions

[Offer Kaye]

On 2/5/06, Offer Kaye wrote:
>
> [3] 
> http://ppm.activestate.com/BuildStatus/5.8-windows/windows-5.8/Scalar-List-Util-1.15.txt
>

Something funky here...
Last night I looked at "Scalar-List-Util"... but the correct name as
Tyler said is "Scalar-List-Utils", with an "s" at the end.

Looking at [1] I see listed both "Scalar-List-Util" version 1.15, with
a status of "FAIL" of Windows, and "Scalar-List-Utils" version "N/A",
with a status of "CORE" on Windows. If you look at the report for it,
it also fails.

"Scalar-List-Utils" seems to be the correct one... CPAN doesn't know
about any "Scalar-List-Util"... Gozer, maybe if you removed the false
entry (Scalar-List-Util) it will solve the problem and let
Scalar-List-Utils pass correctly?

OT question - why is Scalar-List-Utils listed as "CORE"? It is not
part of the Perl5 core, does it mean it is part of ActiveState's core?
Just wondering...

[1] http://ppm.activestate.com/BuildStatus/5.8-S.html

Cheers,
--
Offer Kaye

Re: Binary distributions

[Tyler MacDonald]

Offer Kaye <[EMAIL PROTECTED]> wrote:
> >Phillipe Chaisson aka "Gozer" (one of the mod_perl authors) is
> > responsible for the ActiveState PPM repositories now,
> Hi Gozer, nice to meet you. Gratz on ActiveState's move to a new
> company, good luck :)

It's a Good Thing for all companies involved, but as a developer for
Sophos, I'll certainly miss having them upstairs. (Not to mention the
@activestate.com email addy ;-)

- Tyler

Re: Binary distributions

[Tyler MacDonald]

Offer Kaye <[EMAIL PROTECTED]> wrote:
> Just an example, IO::All [1] version 0.33 has been available since Dec
> 17, 2004. It passed testing many times, at least according to its
> testers page [2]. My default 5.8.7 ActivePerl distribution lists
> IO::All version 0.17 .

According to
http://ppm.activestate.com/BuildStatus/5.8-windows/windows-5.8/IO-All-0.33.txt
it is not being built because Spiffy failed it's tests. According to
Spiffy's log, *it* did not build becuase Scalar-List-Utils failed to
build... Which is the same module that both Adam Kennedy and David Golden
are complaining about. (I wonder how many modules are being held back over
this one bug?)

> I'm not sure which of the reasons you listed are valid, as a user I
> have no real way of knowing... that's why I thought an automated
> Windows-PPM sub-domain on CPAN which would be updated automatically
> for every new package would be great. What you are saying, that
> ActiveState already have such a system, is news to me, especially in
> view of examples such as the above... I would rather use CPAN, but the
> PPM works fine for me, only it's not as updated as CPAN.

I think it's a combination of "too much to do, too little time" and
the QA philosophy of ActiveState. On ppm.activestate.com, if the unit tests
fail, you don't get a PPM; I guess this "protects" ActivePerl users from
installing modules that won't work properly on their install.

ppm.cpan.org would be a good idea and probably not too hard to
implement. What would it's mandate be? Provide every package out there, even
if unit tests fail, so long as it acutally built? Who is going to provide
the icky sticky windows box with VC++-compiled versions of all the libraries
people expect to be able to use with perl (libgd, libmysqlclient,
apache2, etc)? I certainly don't want to lay my hands on any windows box.
;-)

> I'd like to clarify something - I don't really expect Gozer to handle this
> problem... or a problem with any other module... there are so many, how
> can a single person keep up? It would be better if the system were
> automated... and worked :)

I think if the module is close to the metal and a lot of other
modules depend on it, it would be worth ActiveState's time to investigate
the problem and either fix the build system or fix the module. It's a big
payoff if fixing one module means that all of a sudden another 30 or so
automagically build.

> I would rather not use 2 seperate package installation schemes...
> either CPAN or PPM (or something else), not both.

I think those two schemes are exactly what we need for perl. It's
analogous to building and installing a source package, or just installing
the latest RPM/.deb/whatever.


- Tyler

Re: Binary distributions

[Offer Kaye]

On 2/5/06, Tyler MacDonald wrote:
>
>ActiveState always serves the last available version where all tests
> passed on a given platform. It attempts to build and test every package on
> CPAN at least once a week. If something isn't available, it means the tests
> failed, which could mean:
>

Just an example, IO::All [1] version 0.33 has been available since Dec
17, 2004. It passed testing many times, at least according to its
testers page [2]. My default 5.8.7 ActivePerl distribution lists
IO::All version 0.17 .

I'm not sure which of the reasons you listed are valid, as a user I
have no real way of knowing... that's why I thought an automated
Windows-PPM sub-domain on CPAN which would be updated automatically
for every new package would be great. What you are saying, that
ActiveState already have such a system, is news to me, especially in
view of examples such as the above... I would rather use CPAN, but the
PPM works fine for me, only it's not as updated as CPAN.

[1] http://search.cpan.org/dist/IO-All/
[2] http://testers.cpan.org/show/IO-All.html#IO-All-0.33

>
>Phillipe Chaisson aka "Gozer" (one of the mod_perl authors) is
> responsible for the ActiveState PPM repositories now,

Hi Gozer, nice to meet you. Gratz on ActiveState's move to a new
company, good luck :)

> and the he's told me
> that if there's an issue with the build system that causes some particular
> module's build to fail when it shouldn't, he'll fix it right away if it can
> be identified. http://ppm.activestate.com/ contains full logs of every
> single package build success and failure, so if a package isn't available,
> have a look there and see why/how it failed and let him know.
>
>Cheers,
>Tyler
>

Cool! Didn't know about the http://ppm.activestate.com/ . Looking at
the IO-All package, I see it failed because of the prereq Spiffy.
Looking at Spiffy, it failed because of Scalar-List-Util [3]. Which is
funny because it turns out to be a problem for other people, who
mentioned it in other replies to this thread... :)

[3] 
http://ppm.activestate.com/BuildStatus/5.8-windows/windows-5.8/Scalar-List-Util-1.15.txt

I'd like to clarify something - I don't really expect Gozer to handle
this problem... or a problem with any other module... there are so
many, how can a single person keep up?
It would be better if the system were automated... and worked :)

BTW Gozer have you looked at the first line:
Cannot forceunlink D:\cpanrun\build\5-8-0\lib\auto\List\Util\Util.dll:
Permission denied at D:\cpanrun\build\5-8-0\lib/File/Find.pm line 874

Maybe the script is trying to delete a file that the system thinks is
in use? Or something similiar? Windows has that annoying habit of
refusing to delete things it thinks are in use...

Moin Tels,
Regarding what you said, of course the problem is with non pure-Perl
modules. For pure-Perl ones, I can use CPAN... and yes, I'm only
talking about Windows. Forgot there was PPM for other archs until you
reminded me :)
I would rather not use 2 seperate package installation schemes...
either CPAN or PPM (or something else), not both.
As for the problem of different DLLs etc., I don't think any of
Blizzard's 5.5 million customers have ever had a problem installing
WoW because they were missing a DLL from some Office version... Just
to give an example... why should a PPM package be any different? Yes,
I'm a noob when it comes to these things, sorry :)

Cheers,
--
Offer Kaye

Re: Binary distributions

[Offer Kaye]

On 2/5/06, Shlomi Fish wrote:
> On Sunday 05 February 2006 13:54, Tels wrote:
> > Moin Offer Kaye (sorry, can't identify which part of your name which is
> > the one you are called by :-)
> >
>
> Just for everybody's information:
>
> Offer is his first name. (Means "Bambi" in Hebrew).

Moin Tels, don't blame you for getting confused :)

Shlomi, not sure everyone saw Disney movies as a kid ;)
"Offer" means a "young deer" in Hebrew is what Shlomi meant. Another
English word for that is "fawn". But I don't ;)


--
Offer Kaye

Re: Binary distributions

[Shlomi Fish]

On Sunday 05 February 2006 13:54, Tels wrote:
> Moin Offer Kaye (sorry, can't identify which part of your name which is
> the one you are called by :-)
>

Just for everybody's information:

Offer is his first name. (Means "Bambi" in Hebrew). Kaye is his last name. 
(Don't know what it means - it's a name of foreign origin).

Regards,

Shlomi Fish

-
Shlomi Fish  [EMAIL PROTECTED]
Homepage:http://www.shlomifish.org/

95% of the programmers consider 95% of the code they did not write, in the
bottom 5%.

Re: Binary distributions

[Adam Kennedy]

Tyler MacDonald wrote:

Offer Kaye <[EMAIL PROTECTED]> wrote:

Why not start off by providing ppm.cpan.org (as the OP suggested for
linux distors), or something similar? There are many modules that I
want to use where the PPM version provided by ActiveState or some
other repository is badly of out date..


ActiveState always serves the last available version where all tests
passed on a given platform. It attempts to build and test every package on
CPAN at least once a week. If something isn't available, it means the tests
failed, which could mean:

1) Tests are failing because it's windows

2) Tests are failing because they're failing everywhere else as well

3) Tests are failing because of ActiveState's build system

For 3), that could mean because the build system itself is screwed
or they don't have some library available.

Phillipe Chaisson aka "Gozer" (one of the mod_perl authors) is
responsible for the ActiveState PPM repositories now, and the he's told me
that if there's an issue with the build system that causes some particular
module's build to fail when it shouldn't, he'll fix it right away if it can
be identified. http://ppm.activestate.com/ contains full logs of every
single package build success and failure, so if a package isn't available,
have a look there and see why/how it failed and let him know.


How about 
http://ppm.activestate.com/BuildStatus/5.8-windows/windows-5.8/Scalar-List-Util-1.15.txt


I've mentioned this a number of times over the last 6 months, including 
in person at OSCON.


It passes tests, but still gets marked as FAIL, and then everything in 
CPAN that needs Scalar::Util (recursively) does an automatic shortcut 
FAIL without checking to see of the core version is new enough.


Adam K

Re: Binary distributions

[Tels]

Moin Offer Kaye (sorry, can't identify which part of your name which is 
the one you are called by :-)

On Sunday 05 February 2006 07:59, Offer Kaye wrote:
> On 1/28/06, Tels wrote:
> > Of course you must reliaze that, except for pure-perl modules and
> > very controlled environments, binary distributions are doomed to
> > fail.
> >
> > You simple cannot guess what libraries/compiler/system/kernel the
> > user has installed, unless you know the distribution and version
> > *and* require that the user never updates anything.
>
> This email, and the entire discussion that followed, was very Linux
> centric. Correct me if I'm wrong, but for Windows this argument is a
> non-issue, right? I mean, compile a module for Windows and it will
> most likely work for all versions. Or at least the latest ones
> (2000/XP).

This is a common pitfall :) While Windows doesn't have as many 
"distribution" as linux has, there enough to make the "compile once, run 
everywhere" thing to fail in the long run (but as murphy says, you 
realize this only after having spent considerable time to make it work :D

It will work for pure-perl modules (as least when the prereqs are meet), 
that is true. And I have successfully bundled these together with scripts 
and the "use lib" mantra and deploied them on various machines.

But as soon as you need to compile something or need certain libs, DLLs, 
or window components, you step up in the wonderfull world of different 
compilers, linkers, run-time libs, service packs etc. 

I think with linux this is more in the mindset of the developers (you know 
that the systems differ wildly), while a lot windows people tend to 
ignore these things thinking "it doesn't apply to windows right?" and 
then you end up with non-working apps because the author had MS Access, 
service pack 2, the SDK and a compiler (and some unspecified updates that 
came with an MS Office update) installed, while the user machine has none 
of this.

> Why not start off by providing ppm.cpan.org (as the OP suggested for
> linux distors), or something similar? There are many modules that I
> want to use where the PPM version provided by ActiveState or some
> other repository is badly of out date.

Are you talking about ppm for windows or linux?

For linux:

But what would be the point of distributing the binary versions when you 
can download the source and compile/link it yourself?

(Yes, I see the point, but it would bloody much work for a few selected 
distributions.)

For windows:

Why not fix activestates build system first? Their solution seems to be 
"80% there" :)

> I guess that many more people use Perl on Linux boxes, but there are
> still uses for Perl on Windows... ;-)

I pass the chance to make bad puns :)

> It would be wonderful to be able to fully use CPAN on Windows, with
> the same level of comfort that today's pre-packged PPM files already
> provide.

I thought that CPAN already works on windows?

Best wishes,

Tels

-- 
 Signed on Sun Feb  5 12:44:18 2006 with key 0x93B84C15.
 Visit my photo gallery at http://bloodgate.com/photos/
 PGP key on http://bloodgate.com/tels.asc or per email.

 I'm a Sis-sis-sis-sis-sis-sis-sis-sinnahr...



pgptsfeBDEKvn.pgp
Description: PGP signature

Re: Binary distributions

[David Golden]

Tyler MacDonald wrote:

3) Tests are failing because of ActiveState's build system

For 3), that could mean because the build system itself is screwed
or they don't have some library available.

Phillipe Chaisson aka "Gozer" (one of the mod_perl authors) is
responsible for the ActiveState PPM repositories now, and the he's told me
that if there's an issue with the build system that causes some particular
module's build to fail when it shouldn't, he'll fix it right away if it can
be identified. http://ppm.activestate.com/ contains full logs of every
single package build success and failure, so if a package isn't available,
have a look there and see why/how it failed and let him know.


I've been a little disappointed with "right away" (sorry, Philippe).  I 
sent an email to [EMAIL PROTECTED] just over a week ago about 
the build system using an outdated version of Scalar::Util that doesn't 
have "refaddr". (Which is surprising, since they offer an updated one in 
their repository.)  To date, I haven't even received a ticket response 
acknowledging receipt of my message, much less a fix.


A copy of my email follows:

David Golden wrote:
> Dear ActiveState PPM support:
>
> I am the author of Class::InsideOut.  One of the key features of
> Class::InsideOut is that it provides for robust inside-out objects under
> a win32 pseudo-fork (at least for Perl 5.8 which offers the CLONE
> method).  Given that I wrote it in part to work safely with my own
> ASPerl installation on my win32 laptop, I'm frustrated that I can't get
> it to pass your automated build tests and would like some guidance.
>
> In most cases, the problem appears a dependency failure on
> Scalar-List-Utils, and I suspect the other failures are similarly 
related.

>
> Class::InsideOut requires Scalar::Util 1.09.  My ASPerl 5.8.7 Build 815
> seems to have 1.14.  The PPM repositories themselves have
> Scalar-List-Utils 1.18.  However, it seems like the automated build test
> is using an older Scalar-List-Utils at version 1.07.  The list of
> modules at
> http://aspn.activestate.com/ASPN/Modules/Perl?module_name=S&order=name
> shows 1.06.
>
> I'm a little at a loss as to how to proceed.  Dropping my prereq version
> any further isn't possible as I need 1.09, which provides the refaddr
> function.  Could the automated build farm get an upgraded
> Scalar-List-Utils installed?  Even getting to 1.09 with refaddr might
> fix a lot of other distributions that depend on it, too.
>
> I look forward to your thoughts.
>
> Sincerely,
>
> David Golden
>

Re: Binary distributions

[Tyler MacDonald]

Offer Kaye <[EMAIL PROTECTED]> wrote:
> Why not start off by providing ppm.cpan.org (as the OP suggested for
> linux distors), or something similar? There are many modules that I
> want to use where the PPM version provided by ActiveState or some
> other repository is badly of out date..

ActiveState always serves the last available version where all tests
passed on a given platform. It attempts to build and test every package on
CPAN at least once a week. If something isn't available, it means the tests
failed, which could mean:

1) Tests are failing because it's windows

2) Tests are failing because they're failing everywhere else as well

3) Tests are failing because of ActiveState's build system

For 3), that could mean because the build system itself is screwed
or they don't have some library available.

Phillipe Chaisson aka "Gozer" (one of the mod_perl authors) is
responsible for the ActiveState PPM repositories now, and the he's told me
that if there's an issue with the build system that causes some particular
module's build to fail when it shouldn't, he'll fix it right away if it can
be identified. http://ppm.activestate.com/ contains full logs of every
single package build success and failure, so if a package isn't available,
have a look there and see why/how it failed and let him know.

Cheers,
Tyler

Re: Binary distributions

[Offer Kaye]

On 1/28/06, Tels wrote:
>
> Of course you must reliaze that, except for pure-perl modules and very
> controlled environments, binary distributions are doomed to fail.
>
> You simple cannot guess what libraries/compiler/system/kernel the user
> has installed, unless you know the distribution and version *and* require
> that the user never updates anything.
>

This email, and the entire discussion that followed, was very Linux
centric. Correct me if I'm wrong, but for Windows this argument is a
non-issue, right? I mean, compile a module for Windows and it will
most likely work for all versions. Or at least the latest ones
(2000/XP).

Why not start off by providing ppm.cpan.org (as the OP suggested for
linux distors), or something similar? There are many modules that I
want to use where the PPM version provided by ActiveState or some
other repository is badly of out date..

I guess that many more people use Perl on Linux boxes, but there are
still uses for Perl on Windows... ;-)
It would be wonderful to be able to fully use CPAN on Windows, with
the same level of comfort that today's pre-packged PPM files already
provide.

Thanks,
--
Offer Kaye

Re: Binary distributions

[Barbie]

On Sat, Jan 28, 2006 at 09:34:09AM -0800, Tyler MacDonald wrote:
> 
>   From what I gather, CPANPLUS is a linear package building
> system, whereas YACsmoke is a wrapper around that that tries to build as
> many packages as is humanly (er, computerly) possible on a system, with the
> side effect of sending it's results through Test::Reporter. YACsmoke also
> can maintain a database of what it has built and what it hasn't, allowing a
> YACsmoke system to eventually exhaustively build every single package on
> CPAN without building the same one twice. Is this correct?

Sort of. CPANPLUS also maintains a database for the current run, so if
you have a list of 20 distributions you are testing, if 5 of those have
a prerequisite for the same distribution, CPANPLUS internally only
makes/tests it once, but adding each distribution path to @INC. However,
the state information that CPANPLUS stores doesn't include the test
result of a distribution.

CPAN::YACsmoke on the other hand, does record the test result. This
means that if it tried to build a distribution (currently checks agains
the latest version), which reports a FAIL or NA, any distribution that
requires it will have their test results graded as ABORTED and NA
respectively. The ABORTED result does not get reported to CPAN Testers.

The part that is missing currently, is that if a distribution that is a
prerequisite now PASSes, there is no code to actively change all those
ABORTED to NONE, so that the requiring distribution can be retested.
This is part of my todo list.

> > What would the plugins for .deb, .rpm and .ppm do? Currently we just
> > pass the path to CPANPLUS and let CPANPLUS unwrap, make and test the
> > distribution. We then just interrogate the results.
> 
>   These plugins would execute *after* YACsmoke/CPANPLUS has done it's
> work, but before it's cleaned up after itself, and generate the actual
> .deb/.rpm/.ppm files.

I would see this as a different system, perhaps for someone who actively
maintains a repository for that particular install package. I would
prefer to see a code base that uses the YACSmoke database, rather than
used as a plugin. The .cpanplus build directories are not removed once
testing is complete, so the blib directories are all still available
after a test run.

YACSmoke does provide a purge facility, but it doesn't run automatically
after a test run, you have to manually instigate it.

> > Perhaps this is something you meant for CPANPLUS to handle, in which
> > case I'm sure that's something Jos would be interested in, as he has
> > previously looked at some kind of binary support.
> 
>   Well, maybe the actual low-level package generation could be kicked
> down a level, but the whole reason I thought of using YACsmoke for this was
> because of it's "grab everything and try to build it" nature.

YACSmoke would certianly be useful as a first stage, to ensure you
only package distributions that pass the tests for your platform.

> > I plan to find time soon to complete the tests required for the next
> > release of CPAN::YACSmoke, so if what you want does relate to
> > CPAN::YACSmoke I can start taking a look and see what needs to be done
> > to implement it.

I will look into this further, but it'll be lower on my todo list, as it
will be driven by packagers rather than testers.

Barbie.

Re: Binary distributions

[David Cantrell]

Tels wrote:

On Saturday 28 January 2006 08:20, Tyler MacDonald wrote:

That is such an incredibly good idea. I've got plenty of bandwidth
to burn and I'm willing to set up debian.cpan.org.
Of course you must reliaze that, except for pure-perl modules and very 
controlled environments, binary distributions are doomed to fail.


You simple cannot guess what libraries/compiler/system/kernel the user  
has installed, unless you know the distribution and version *and* require 
that the user never updates anything.


There is a reason that modules are compiled/linked against the target 
system prior to installation, and there is also a reason to run the 
tests: to assure that that step really worked.


FreeBSD might get away with that because the user will ever only install 
their ports and they can make sure that they all play together. For 
everything else, this becomes a maintanance nightmare and I wish to be no 
part of that :)


Actually, this isn't so bad on Debian.  The packaging system copes with 
having dependencies on particular versions of other packages, and Debian 
is VERY stable - libfoo just doesn't randomly change version.


--
David Cantrell

Re: Binary distributions

[Andreas J. Koenig]

> On Sun, 29 Jan 2006 17:13:40 -0800, Tyler MacDonald <[EMAIL PROTECTED]> 
> said:

  > Andreas J. Koenig <[EMAIL PROTECTED]> wrote:
 >> FWIW, we're using dh-make-perl to create debian packages from CPAN
 >> modules.

  > Andreas,
  > I've used this tool a few times when a CPAN module wasn't already
  > available in unstable/main, but I havent looked into it too closely. I'm
  > curious, does it do anything about .so's that XS modules need, or build vs.
  > runtime dependancies in environments that support it (Module::Build, etc)?

Recently I haven't had to work with XS modules that have not already
an associated Debian package. Support for modules that have only a
Build.PL seems to be missing. The manpage is not promising too much
when it says:

BUGS
   Several, let me know when you find them.

:/
-- 
andreas

Re: Binary distributions

[Andreas J. Koenig]

> On Sun, 29 Jan 2006 15:05:02 -0800, Tyler MacDonald <[EMAIL PROTECTED]> 
> said:

  > Sébastien Aperghis-Tramoni <[EMAIL PROTECTED]> wrote:
 >> Did anybody here have played with CPANPLUS::Dist::Deb?
 >> http://search.cpan.org/dist/CPANPLUS-Dist-Deb/
 >> 
 >> Believing its documentation, it should build a valid Debian package
 >> and take care of its dependencies (dunno if that means just listing
 >> or actually adding them in the package).

  > This excited me, but even with all the debian helper utils installed
  > and CPANPLUS on default settings running as root, it crashes not being able
  > to find the right dist_cpan object out of CPANPLUS. :-(

FWIW, we're using dh-make-perl to create debian packages from CPAN modules.


-- 
andreas

Re: Binary distributions

[Tyler MacDonald]

Andreas J. Koenig <[EMAIL PROTECTED]> wrote:
> FWIW, we're using dh-make-perl to create debian packages from CPAN
> modules.

Andreas,
I've used this tool a few times when a CPAN module wasn't already
available in unstable/main, but I havent looked into it too closely. I'm
curious, does it do anything about .so's that XS modules need, or build vs.
runtime dependancies in environments that support it (Module::Build, etc)?

Thanks,
Tyler

Re: Binary distributions

[Tyler MacDonald]

Sébastien Aperghis-Tramoni <[EMAIL PROTECTED]> wrote:
> Did anybody here have played with CPANPLUS::Dist::Deb?
>   http://search.cpan.org/dist/CPANPLUS-Dist-Deb/
> 
> Believing its documentation, it should build a valid Debian package
> and take care of its dependencies (dunno if that means just listing
> or actually adding them in the package).

This excited me, but even with all the debian helper utils installed
and CPANPLUS on default settings running as root, it crashes not being able
to find the right dist_cpan object out of CPANPLUS. :-(

- Tyler

Re: Binary distributions

[Sébastien Aperghis-Tramoni]

Gabor Szabo wrote:

> I think I agree. That's what I would like to see solved. If I stick to
> the standard apt-get (or whatever) of my distribution I would like to be
> able to get all the CPAN modules by saying
>
> # apt-get install Module::Name

Did anybody here have played with CPANPLUS::Dist::Deb?
  http://search.cpan.org/dist/CPANPLUS-Dist-Deb/

Believing its documentation, it should build a valid Debian package
and take care of its dependencies (dunno if that means just listing
or actually adding them in the package).

--
Sébastien Aperghis-Tramoni

Close the world, txEn eht nepO.

Re: Binary distributions

[Tyler MacDonald]

Gabor Szabo <[EMAIL PROTECTED]> wrote:
> > You simple cannot guess what libraries/compiler/system/kernel the user
> > has installed, unless you know the distribution and version *and* require
> > that the user never updates anything.
> 
> I think I agree. That's what I would like to see solved. If I stick to the
> standard apt-get (or whatever) of my distribution I would like to be able
> to get all the CPAN modules by saying
> 
> # apt-get install Module::Name

Well in debianland it's actually apt-get libmodule-name-perl :)

The solution to this sounds straightforward, although probably not
simple to implement. Your build box is running the distribution+arch you
want to release the modules for. After building the packages, "ldd" any
binaries found within. This will give you a list of library filenames. In
debian's case, /var/lib/dpkg/info/*.list contains a list of files supplied
by each package on the system. Use that to match dependancies up.

If a package requires something on the system that it *doesn't* link
against (say, a certain executable), that problem won't be solved, but I
think that case is a bit rarer.

- Tyler

Re: Binary distributions

[Tyler MacDonald]

Barbie <[EMAIL PROTECTED]> wrote:
> I'm one of the maintainers/developers for CPAN::YACSmoke. I was
> intrigued by your post about adding a Packager plugin to it. However,
> I'm unclear as to what purpose it would serve. CPAN::YACSmoke is purely
> about reporting on test results. CPANPLUS does the actual work. 

Barbie,

From what I gather, CPANPLUS is a linear package building
system, whereas YACsmoke is a wrapper around that that tries to build as
many packages as is humanly (er, computerly) possible on a system, with the
side effect of sending it's results through Test::Reporter. YACsmoke also
can maintain a database of what it has built and what it hasn't, allowing a
YACsmoke system to eventually exhaustively build every single package on
CPAN without building the same one twice. Is this correct?


> What would the plugins for .deb, .rpm and .ppm do? Currently we just
> pass the path to CPANPLUS and let CPANPLUS unwrap, make and test the
> distribution. We then just interrogate the results.

These plugins would execute *after* YACsmoke/CPANPLUS has done it's
work, but before it's cleaned up after itself, and generate the actual
.deb/.rpm/.ppm files.

> Perhaps this is something you meant for CPANPLUS to handle, in which
> case I'm sure that's something Jos would be interested in, as he has
> previously looked at some kind of binary support.

Well, maybe the actual low-level package generation could be kicked
down a level, but the whole reason I thought of using YACsmoke for this was
because of it's "grab everything and try to build it" nature.

> I plan to find time soon to complete the tests required for the next
> release of CPAN::YACSmoke, so if what you want does relate to
> CPAN::YACSmoke I can start taking a look and see what needs to be done
> to implement it.

Sounds good :)

Thanks,
Tyler

Re: Binary distributions

[Gabor Szabo]

On 1/28/06, Tels <[EMAIL PROTECTED]> wrote:
> Moin,
>
> On Saturday 28 January 2006 08:20, Tyler MacDonald wrote:
> > Gabor Szabo <[EMAIL PROTECTED]> wrote:
> > >
> > > Anyway I think instead of trying to setup our own binary distribution
> > > we might want to make sure there are up to date repositories of
> > > Perl modules for the major distributions
> > > (and I am not talking only about Linux distributions here).
> > > It can be done by helping the people who already maintain some of
> > > these distributions or by setting up repositories such as
> > > debian.cpan.org, fedora.cpan.org, etc...
> >
> >   That is such an incredibly good idea. I've got plenty of bandwidth
> > to burn and I'm willing to set up debian.cpan.org.
>
> Of course you must reliaze that, except for pure-perl modules and very
> controlled environments, binary distributions are doomed to fail.
>
> You simple cannot guess what libraries/compiler/system/kernel the user
> has installed, unless you know the distribution and version *and* require
> that the user never updates anything.

I think I agree. That's what I would like to see solved. If I stick to
the standard
apt-get (or whatever) of my distribution I would like to be able to get all the
CPAN modules by saying

# apt-get install Module::Name

> There is a reason that modules are compiled/linked against the target
> system prior to installation, and there is also a reason to run the
> tests: to assure that that step really worked.
>
> FreeBSD might get away with that because the user will ever only install
> their ports and they can make sure that they all play together. For
> everything else, this becomes a maintanance nightmare and I wish to be no
> part of that :)

This is for those who want specialized systems with non default versions
of libraries/applications etc. They should indeed install from source and run
the tests.

Others should be able to rely on the fact that the distributor has
already successfully ran the tests in a system with the same software
she has.

Gabor

Re: Binary distributions

[Tels]

Moin,

On Saturday 28 January 2006 08:20, Tyler MacDonald wrote:
> Gabor Szabo <[EMAIL PROTECTED]> wrote:
> > I have just moved to Ubuntu and thought I will try to rely on apt-get
> > to install my Perl modules. Quckly I hit a wall and could not install
> > some of the basic modules. I did not have the time to investigate and
> > check if I made a mistake or if there is a .deb repository with the
> > latest CPAN modules for Ubuntu. I reverted to use CPAN.pm.
> > BTW here is an article on how to build Debian packages of Perl
> > modules: http://www.debian-administration.org/articles/78
> >
> >
> > Anyway I think instead of trying to setup our own binary distribution
> > we might want to make sure there are up to date repositories of
> > Perl modules for the major distributions
> > (and I am not talking only about Linux distributions here).
> > It can be done by helping the people who already maintain some of
> > these distributions or by setting up repositories such as
> > debian.cpan.org, fedora.cpan.org, etc...
>
>   That is such an incredibly good idea. I've got plenty of bandwidth
> to burn and I'm willing to set up debian.cpan.org.

Of course you must reliaze that, except for pure-perl modules and very 
controlled environments, binary distributions are doomed to fail.

You simple cannot guess what libraries/compiler/system/kernel the user  
has installed, unless you know the distribution and version *and* require 
that the user never updates anything.

There is a reason that modules are compiled/linked against the target 
system prior to installation, and there is also a reason to run the 
tests: to assure that that step really worked.

FreeBSD might get away with that because the user will ever only install 
their ports and they can make sure that they all play together. For 
everything else, this becomes a maintanance nightmare and I wish to be no 
part of that :)

PS: I just read that Adam Kennedy wrote basically the same things. OOps.

Best wishes,

Tels

-- 
 Signed on Sat Jan 28 11:16:38 2006 with key 0x93B84C15.
 Visit my photo gallery at http://bloodgate.com/photos/
 PGP key on http://bloodgate.com/tels.asc or per email.

 This email violates U.S. patent #5,546,528 and EU patent EP0689133:
 
   
   | Header | Body | Attachements |   |
   |+  +--|
   |  |
   ~  ~



pgpk46VLNNPg9.pgp
Description: PGP signature

Re: Binary distributions

[Tyler MacDonald]

Gabor Szabo <[EMAIL PROTECTED]> wrote:
> I have just moved to Ubuntu and thought I will try to rely on apt-get
> to install my Perl modules. Quckly I hit a wall and could not install some
> of the basic modules. I did not have the time to investigate and check
> if I made a mistake or if there is a .deb repository with the latest CPAN
> modules for Ubuntu. I reverted to use CPAN.pm.
> BTW here is an article on how to build Debian packages of Perl modules:
> http://www.debian-administration.org/articles/78
> 
> 
> Anyway I think instead of trying to setup our own binary distribution
> we might want to make sure there are up to date repositories of
> Perl modules for the major distributions
> (and I am not talking only about Linux distributions here).
> It can be done by helping the people who already maintain some of these
> distributions or by setting up repositories such as debian.cpan.org,
> fedora.cpan.org, etc...

That is such an incredibly good idea. I've got plenty of bandwidth
to burn and I'm willing to set up debian.cpan.org. 

I think the most obvious way to automate this would be to take
advantage of the whole perl package / dependancy / build / test process that
the YACsmoke module already offers us. Maybe
CPAN::YACSmoke::Plugin::Packager, with children ::Deb, ::RPM, ::PPM, etc.
These modules could just stick their built packages into an outgoing
directory (or maybe multiple, "noarch", "i386", etc); some distros would be
able to just nab those and their metainfo and roll a repo out of it, maybe
some we'll have to write tools to do that for as well.

- Tyler

Binary distributions

[Gabor Szabo]

Why do we need to reinvent this wheel ?

Most of the platforms out there have some binary packaging system.
Solaris has their own, Linuxen have rpm/deb or whatever else they have.
ActiveState with its binary Perl distributions have ppm and while that's
not perfect we read that they are working on fixing the issues.
As someone just mentioned the FreeBSD ports are kept very up-to-date.

I have just moved to Ubuntu and thought I will try to rely on apt-get
to install my Perl modules. Quckly I hit a wall and could not install some
of the basic modules. I did not have the time to investigate and check
if I made a mistake or if there is a .deb repository with the latest CPAN
modules for Ubuntu. I reverted to use CPAN.pm.
BTW here is an article on how to build Debian packages of Perl modules:
http://www.debian-administration.org/articles/78


Anyway I think instead of trying to setup our own binary distribution
we might want to make sure there are up to date repositories of
Perl modules for the major distributions
(and I am not talking only about Linux distributions here).
It can be done by helping the people who already maintain some of these
distributions or by setting up repositories such as debian.cpan.org,
fedora.cpan.org, etc...

Gabor