Re: Why make up a Makefile.PL (was: Re: git tarballs / tarfile comments)

[Thomas Klausner]

Hi!

On Wed, Sep 03, 2008 at 03:16:35PM -0400, David Golden wrote:

> There are handful of things on CPAN that are just zipped .pm files.  I

cpants says:
cpants=> select extension,count(*) from dist group by extension ;
 extension | count 
---+---
 tar.gz| 14762
 tgz   |   241
 zip   |   113

I've attachted the list, if this is of any help...

-- 
#!/usr/bin/perl  http://domm.plix.at
for(ref bless{},just'another'perl'hacker){s-:+-$"-g&&print$_.$/}
package
---
 Acme-POE-Knee-1.10.zip
 AI-NeuralNet-BackProp-0.89.zip
 AI-NeuralNet-Mesh-0.44.zip
 Algorithm-Line-Bresenham-C-0.1.zip
 Algorithm-Loops-1.031.zip
 Archive-Tyd-0.02.zip
 BitTorrent-V0.1.zip
 Chatbot-Alpha-2.04.zip
 Convert-MIL1750A-0.1.zip
 CPAN-Test-Dummy-Perl5-Make-Zip-1.03.zip
 Crypt-XXTEA-1.00.zip
 Data-Iterator-0.021.zip
 DBIx-Compare-1.0.zip
 DBIx-Compare-ContentChecksum-mysql-1.0.zip
 DBIx-Fun-0.02.zip
 Devel-TraceSubs-0.02.zip
 DISCO-0.01.zip
 EasyDate-103.zip
 ESplit1.00.zip
 extensible_report_generator_1.13.zip
 ExtUtils-FakeConfig-0.11.zip
 Finance-Bank-ES-Cajamadrid-0.04.zip
 Finance-Edgar-0.01.zip
 Games-LogicPuzzle-0.20.zip
 Games-Multiplayer-Manager-1.01.zip
 Graph-Maker-0.02.zip
 HTML-EasyTable-0.04.zip
 Inline-Octave-0.22.zip
 Joystick-1.01.zip
 LIMS-Controller-1.6b.zip
 LIMS-MT_Plate-1.17.zip
 Lingua-EN-Dict-0.20.zip
 Lingua-EN-VerbTense-3.00.zip
 link_NCBI.zip
 Memo32-1.01b.zip
 mGen-1.03.zip
 Microarray-0.45c.zip
 MIDI-Trans-0.15.zip
 modules/etext/etext.1.6.3.zip
 modules/SelfUnzip-0.01.zip
 modules/SOM-0.0601.zip
 Module-Versions-0.02.zip
 MRTG-Config-0.04.zip
 MSN-PersonalMessage-0.02.zip
 NCBI-0.10.zip
 NetIcecast-1.02.zip
 Net-IPAddress-1.10-PPM.zip
 Net-Server-POP3-0.0009.zip
 Notes/Notes.zip
 NPRG-0.31.zip
 Nums2Words-1.12.zip
 Object-Interface-1.1.zip
 os2/OS2-FTP-0_10.zip
 os2/OS2-UPM-0_10.zip
 os2/tk/Tk-OS2src-1.04.zip
 Parse-EventLog-0.7.zip
 ParseTemplate-0.37.zip
 PCX-Loader-0.50.zip
 Perl56/Win32-PerfMon.0.07-Perl5.6.zip
 Perl6-Interpolators-0.03.zip
 perlipse/Perlipse-0.02.zip
 Prima-prigraph-win32-1.06.zip
 Reduziguais.zip
 Regex-Number-GtLt-0.1.zip
 resched-0.7.2.zip
 rms.zip
 smg.zip
 System-Index-0.1.zip
 SystemTray-Applet-0.02.zip
 SystemTray-Applet-Win32-0.01.zip
 Template-Ast-0.02.zip
 Term-Getch-0.20.zip
 Term-Sample-0.25.zip
 Test-Version-0.02.zip
 Tie-Tk-Text-0.91.zip
 TimeConvert0.5.zip
 tinyperl-1.0-580-win32.zip
 Tk-DiffText-0.19.zip
 Tk-TOTD-0.20.zip
 Tkx-FindBar-0.06.zip
 UDPServersAndClients.zip
 VCS-StarTeam-0.08.zip
 VMS-Device-0_09.zip
 VMS-FlatFile-0.01.zip
 VMS-Queue-0_58.zip
 vms-user-0_02.zip
 Win32-ActAcc-1.1.zip
 Win32-Capture-1.1.zip
 Win32-Encode-0.5beta.zip
 Win32-Env-Path-0.01.zip
 win32-guidgen-0.04.zip
 Win32-GUI-Scintilla-1.7.zip
 Win32-HostExplorer-01.zip
 Win32-MCI-Basic-0.02.zip
 Win32-MediaPlayer-0.2.zip
 Win32-MIDI-0_2.zip
 Win32-MMF-0.09e.zip
 Win32-MultiMedia-0.01.zip
 Win32-OLE-CrystalRuntime-Application-0.08.zip
 Win32-OLE-OPC-0.92.zip
 Win32-PerlExe-Env-0.04.zip
 Win32-Printer-0.9.1.zip
 Win32-ShellExt-0.1.zip
 Win32/SimpleProcess/SimpleProcess_1.0.zip
 Win32-SqlServer-2.004.zip
 Win32-SystemInfo-0.11.zip
 Win32-TaskScheduler2.0.3.zip
 Win32-TieRegistry-0.25.zip
 Win32-TSA-Notify-0.01.zip
 WordPress-V1.zip
 WWW-MySpaceBot-0.01.zip
 WWW-PDAScraper-0.1.zip
 WWW-TwentyQuestions-0.01.zip
(113 rows)

Re: Why make up a Makefile.PL (was: Re: git tarballs / tarfile comments)

[David Golden]

On Wed, Sep 3, 2008 at 3:55 PM, Aristotle Pagaltzis <[EMAIL PROTECTED]> wrote:
>> * BinTree
>> * Counter
>
> Can't even find those.
>
>> * Apache::AuthenIMAP
>
> Last update: 2002.

They may not be indexed on search.cpan.org, but they exist in
the02packages.details.txt.gz file and they exist in actual CPAN
mirrors. E.g.:

http://cpan.hexten.net/authors/id/T/TO/TOMC/scripts/CS-Talk/source/dstructs/trees/BinTree.pm.gz

If you load the CPAN shell and type "test BinTree" you'll see what happens.

Note -- you have to disable CPAN::SQLite if you have it as it can't
handle these kinds of files.  The native CPAN indexing will find them,
however.

Apache-AuthenIMAP-0.03 is unauthorized and isn't indexed in
02packages.  Someone requesting the "Apache::AuthenIMAP" module will
get the zipped pm file.

-- David

Re: Why make up a Makefile.PL (was: Re: git tarballs / tarfile comments)

[Aristotle Pagaltzis]

* David Golden <[EMAIL PROTECTED]> [2008-09-03 21:20]:
> Examples:
> 
> * BinTree
> * Counter

Can’t even find those.

> * Apache::AuthenIMAP

Last update: 2002.

> Just to be on the safe side, however, earlier today I committed
> a patch to the CPAN trunk to bypass CPAN::Reporter entirely if
> a Makefile.PL has been generated by CPAN.pm.

Sounds good. Presumably those distros are all so old that their
authors are unlikely to have an interest in test reports
generated almost a decade after release.

Regards,
-- 
Aristotle Pagaltzis // 

Why make up a Makefile.PL (was: Re: git tarballs / tarfile comments)

[David Golden]

On Wed, Sep 3, 2008 at 1:38 PM, Aristotle Pagaltzis <[EMAIL PROTECTED]> wrote:
> But the FAIL did fail to point out the true source of the
> problem. And IMO this hints at a real problem that was mentioned
> in this thread, but was not really indicted: namely CPAN.pm's
> logic that if there is no Makefile.PL, it is a sane idea to make
> one up out of whole cloth. I can't believe anyone would think
> this could ever *ever* work, though Andreas does not strike me as
> the type to put harebrained heuristical magic in code. So I have
> to wonder what the justification for this behaviour might be. Is
> it really necessary or even helpful?

There are handful of things on CPAN that are just zipped .pm files.  I
think the generated Makefile.PL was intended to deal with them.

Examples:

* BinTree
* Counter
* Apache::AuthenIMAP

However, CPAN.pm only generates a Makefile.PL as a last resort (i.e.
no Build.PL and no Makefile.PL).

Just to be on the safe side, however, earlier today I committed a
patch to the CPAN trunk to bypass CPAN::Reporter entirely if a
Makefile.PL has been generated by CPAN.pm.

David

Re: git tarballs / tarfile comments

[Aristotle Pagaltzis]

* Graham Barr <[EMAIL PROTECTED]> [2008-09-03 18:35]:
> But the FAIL is record in the wrong place.

Anyone with a CPAN toolchain older than the most recent bleeding
edge version or with a couple-months-old tar binary (ie. everyone
except a number of people indistinguishable from zero) will still
encounter the problem. Your distribution is not backward
compatible with only-barely-outdated software.

That, IMO, is a legitimate flaw in your distribution: current
users with non-ancient system configurations will actually
encounter this problem. I would certainly suggest that you
re-release with a comment-less tarball.

So the FAIL did ding the right place. The inadequacy of Testers
in this case is that there weren’t *enough* FAILs to go around to
account for all the problems that your incident uncovered.
Without manual investigation, this might have gone undetected for
quite a while longer.

But the FAIL did fail to point out the true source of the
problem. And IMO this hints at a real problem that was mentioned
in this thread, but was not really indicted: namely CPAN.pm’s
logic that if there is no Makefile.PL, it is a sane idea to make
one up out of whole cloth. I can’t believe anyone would think
this could ever *ever* work, though Andreas does not strike me as
the type to put harebrained heuristical magic in code. So I have
to wonder what the justification for this behaviour might be. Is
it really necessary or even helpful?

Regards,
-- 
Aristotle Pagaltzis // 

Re: git tarballs / tarfile comments

[Eric Wilhelm]

# from David Cantrell
# on Wednesday 03 September 2008 06:14:

>> Or, do thousands of people need to learn to do:
>>   man git-tar-tree:
>>          git tar-tree v1.4.0^{tree} git-1.4.0 | gzip
>> >git-1.4.0.tar.gz Create a tarball for v1.4.0 release, but without a
>> global extended pax header.
>
>A better solution might be for EU::MM / Module::Build /
> Module::Install or whatever people use for creating their tarballs to
> learn about git's peculiarities.  I believe that EU::MM at least
> already knows something about CVS and SVN.

Uh... I wonder how this tarball was created.  './Build dist' doesn't 
care if you use git, mercurial, platinibus, or even goldenium.

--Eric
-- 
software:  a hypothetical exercise which happens to compile.
---
http://scratchcomputing.com
---

Re: git tarballs / tarfile comments

[David Cantrell]

On Wed, Sep 03, 2008 at 11:33:30AM -0500, Graham Barr wrote:
> On Sep 3, 2008, at 9:26 AM, David Golden wrote:
> > So do we count this as a win for CPAN Testers?  ;-)
> Sort of, due to the fact the bug did get fixed.
> But the FAIL is record in the wrong place. It gets counted as a FAIL  
> in the distribution because the tester had issues building it that  
> were not really the fault of the distribution.

IMO including Weird Stuff in the distribution that causes CPAN.pm to
throw a fit is a problem with the distribution.  Even though the latest
dev release of CPAN.pm will hide the problem, users who don't stay on
the bleeding edge (that's nigh-on all of them) will still see this
happen, and still wonder why on earth your module won't install.

Anyway, it counts as a win because all the relevant people now know that
there is a problem and what it is.  But it's only a win because the
module author queried the report.  Without having done that, only he
would know that there was something weird happening.

Dragging this back specifically onto the subject of QA - quality
assurance is an iterative process, with feedback.

If, like one of my previous employers, you make widgets, you test
completed widgets, you analyse how they fail, the analyst suggests
how to improve the manufacturing process to prevent a common failure,
and TPTB then ignore his report - then "Quality Assurance: you're
doing it wrong".

My manager's response to my analysis that they'd not be doing anything
is another case of "I don't care why it happens".

(Actually we didn't make widgets - we made things that tell other things
where to go Boom.  Scary.)

-- 
David Cantrell | Hero of the Information Age

All children should be aptitude-tested at an early age and,
if their main or only aptitude is for marketing, drowned.

Re: git tarballs / tarfile comments

[Graham Barr]

On Sep 3, 2008, at 9:26 AM, David Golden wrote:
On Wed, Sep 3, 2008 at 10:19 AM, David Cantrell  
<[EMAIL PROTECTED]> wrote:

On Wed, Sep 03, 2008 at 08:00:49AM +0200, Andreas J. Koenig wrote:

[git comment in tarballs]

CPAN 1.92_64 is uploaded with a workaround for broken tar
implementations.


Thanks, that appears to work.  At least, it Does The Right Thing for
perl-ldap-0.37.


So do we count this as a win for CPAN Testers?  ;-)


Sort of, due to the fact the bug did get fixed.


perl-ldap-0.37 uploaded 28 Aug 2008.  CPAN Testers FAILed it.  Author
complained.  People investigated.  Underlying problem in tar addressed
in CPAN 1.92_64 released on 3 Sep 2008.


But the FAIL is record in the wrong place. It gets counted as a FAIL  
in the distribution because the tester had issues building it that  
were not really the fault of the distribution. So I would still  
advocate changing these FAILs to UNKNOWNs


Graham.

Re: git tarballs / tarfile comments

[David Golden]

On Wed, Sep 3, 2008 at 10:19 AM, David Cantrell <[EMAIL PROTECTED]> wrote:
> On Wed, Sep 03, 2008 at 08:00:49AM +0200, Andreas J. Koenig wrote:
>> > [git comment in tarballs]
>> CPAN 1.92_64 is uploaded with a workaround for broken tar
>> implementations.
>
> Thanks, that appears to work.  At least, it Does The Right Thing for
> perl-ldap-0.37.

So do we count this as a win for CPAN Testers?  ;-)

perl-ldap-0.37 uploaded 28 Aug 2008.  CPAN Testers FAILed it.  Author
complained.  People investigated.  Underlying problem in tar addressed
in CPAN 1.92_64 released on 3 Sep 2008.

-- David

Re: git tarballs / tarfile comments

[David Cantrell]

On Wed, Sep 03, 2008 at 08:00:49AM +0200, Andreas J. Koenig wrote:
> > [git comment in tarballs]
> CPAN 1.92_64 is uploaded with a workaround for broken tar
> implementations.

Thanks, that appears to work.  At least, it Does The Right Thing for
perl-ldap-0.37.

-- 
David Cantrell | A machine for turning tea into grumpiness

  The test of the goodness of a thing is its fitness for use.  If it
  fails on this first test, no amount of ornamentation or finish will
  make it any better, it will only make it more expensive and foolish.
 -- Frank Pick, lecture to the Design and Industries Assoc, 1916

Re: git tarballs / tarfile comments

[David Cantrell]

On Tue, Sep 02, 2008 at 06:21:34PM -0700, Eric Wilhelm wrote:
> # from Jan Dubois
> >On Tue, 02 Sep 2008, David Cantrell wrote:
> >> $ tar tzvf perl-ldap-0.37.tar.gz
> >> ?rw-rw-rw- root/root52 2008-08-28 12:52:15 pax_global_header
> >>unknown file type `g'
> >It is not actually a file, but an extended header record containing
> >a comment (a GIT commit id in this case).  You need GNU tar 1.14 ...
> >Given that GIT is becoming more popular it would be good to teach
> >CPAN.pm to deal with these kind of tarballs even when they are
> >being processed by older tar versions.
> And then CPAN.pm will need to be upgraded ;-)
> >>This might also explain why the distribution doesn't appear on
> >>search.cpan.org.
> The tar on that machine (or PAUSE?) needs an upgrade?

If you think that getting people to upgrade CPAN.pm is hard, I dread to
think how people will react to being told to upgrade tar!  Doubly so for
people who don't use GNU tar.

> Or, do thousands of people need to learn to do:
>   man git-tar-tree:
>  git tar-tree v1.4.0^{tree} git-1.4.0 | gzip >git-1.4.0.tar.gz
>   Create a tarball for v1.4.0 release, but without a global
>   extended pax header.

A better solution might be for EU::MM / Module::Build / Module::Install
or whatever people use for creating their tarballs to learn about git's
peculiarities.  I believe that EU::MM at least already knows something
about CVS and SVN.

-- 
David Cantrell | top google result for "topless karaoke murders"

Godliness is next to Englishness

Re: git tarballs / tarfile comments

[Andreas J. Koenig]

> On Wed, 03 Sep 2008 05:55:19 +0200, [EMAIL PROTECTED] (Andreas J. Koenig) 
> said:

  > (I'll have a look what CPAN.pm can do about this ASAP.)

CPAN 1.92_64 is uploaded with a workaround for broken tar
implementations. Please upgrade:

  cpan> install ANDK/CPAN-1.92_64.tar.gz

Bugreport against Archive::Tar is filed.
http://rt.cpan.org/Ticket/Display.html?id=38932

Thanks for the heads up!
-- 
andreas

Re: git tarballs / tarfile comments

[Andreas J. Koenig]

> On Tue, 2 Sep 2008 18:21:34 -0700, Eric Wilhelm <[EMAIL PROTECTED]> said:

 >> a comment (a GIT commit id in this case).  You need GNU tar 1.14 to
 >> handle the extended header correctly.  Earlier versions will display
 >> a warning and extract the comment as a file.

 ew> Ah, interesting.

 >>> That first file is the problem.  Because the distribution doesn't
 >>> untar cleanly into a subdirectory, CPAN.pm creates a subdir for you
 >>> ... 
 >> 
 >> Given that GIT is becoming more popular it would be good to teach
 >> CPAN.pm to deal with these kind of tarballs even when they are
 >> being processed by older tar versions.

 ew> And then CPAN.pm will need to be upgraded ;-)

 >>> This might also explain why the distribution doesn't appear on
 >>> search.cpan.org.

 ew> The tar on that machine (or PAUSE?) needs an upgrade?

For the record: the tar on PAUSE is 1.16 since December 2006.

(I'll have a look what CPAN.pm can do about this ASAP.)

-- 
andreas

Re: git tarballs / tarfile comments

[Eric Wilhelm]

# from Jan Dubois
# on Tuesday 02 September 2008 17:42:

>On Tue, 02 Sep 2008, David Cantrell wrote:
>> $ tar tzvf perl-ldap-0.37.tar.gz
>> ?rw-rw-rw- root/root52 2008-08-28 12:52:15 pax_global_header
>>unknown file type `g'
>> drwxrwxr-x root/root 0 2008-08-28 12:52:15 perl-ldap-0.37/
>> ...
>
>It is not actually a file, but an extended header record containing
>a comment (a GIT commit id in this case).  You need GNU tar 1.14 to
>handle the extended header correctly.  Earlier versions will display
>a warning and extract the comment as a file.

Ah, interesting.

>> That first file is the problem.  Because the distribution doesn't
>> untar cleanly into a subdirectory, CPAN.pm creates a subdir for you
>> ... 
>
>Given that GIT is becoming more popular it would be good to teach
>CPAN.pm to deal with these kind of tarballs even when they are
>being processed by older tar versions.

And then CPAN.pm will need to be upgraded ;-)

>>This might also explain why the distribution doesn't appear on
>>search.cpan.org.

The tar on that machine (or PAUSE?) needs an upgrade?

Or, do thousands of people need to learn to do:

  man git-tar-tree:
 git tar-tree v1.4.0^{tree} git-1.4.0 | gzip >git-1.4.0.tar.gz
  Create a tarball for v1.4.0 release, but without a global
  extended pax header.

?

--Eric
-- 
As an old bass player friend of mine used to say: throw money, don't 
clap.
--Tony Parisi
---
http://scratchcomputing.com
---