Re: Performance of Data::Dump
On Fri, May 05, 2006 at 11:03:51AM -0700, Lyle Kopnicky wrote: > I'm using Data::Dump in a project to periodically write a snapshot of a > hash table to disk, so it can be recovered on failure. Unfortunately, it > can take 40 seconds to write out a 10,000-entry hash table (each entry > is itself a small hash table, one entry of which is itself a small hash > table). By writing it to a string first and then to a file, I found that > the vast majority of that time is in creating the string, and only 5 > seconds is in writing the file (using print). > > If I iterate through the table, writing each entry separately to the > file, I get a fair improvement. I guess this means that internally, > Data::Dump is using a lot of inefficient string appends. > > Nevertheless, I need to get this operation down to 5 seconds, preferably > 1 second if possible. Are there modules faster than Data::Dump? What > about MLDBM? I think Storable would be the normal thing to use. TomP -- --------- Tom Pollard [EMAIL PROTECTED] Schrodinger, Inc.646-366-9555 x102 - ___ Perl-Win32-Users mailing list Perl-Win32-Users@listserv.ActiveState.com To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
Re: pipes, IO::Select and blocking
On Mon, Mar 20, 2006 at 02:49:39PM -0800, Hon Shi wrote: > New to pipes - got one working - but now want to set it up > so the forked process won't block. I thought I'd try and use > IO::Selects's ->can_read(1) method. I've tried it many different > ways. What am I doing wrong? Under Windows, you can only use IO::Select() with socket handles. TomP -- ------------- Tom Pollard [EMAIL PROTECTED] Schrodinger, Inc.646-366-9555 x102 - ___ Perl-Win32-Users mailing list Perl-Win32-Users@listserv.ActiveState.com To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
Re: Threads
On Tue, Feb 28, 2006 at 10:51:39AM +0800, Foo Ji-Haw wrote: > But don't get too ambitious on Perl's ithreads. They are fun, but not as > rich as threads from other languages. They're also unstable under Unix, and will cause your script to crash randomly (though not frequently, in my experience.) This is not a problem for Win32 Perl, as far as I've been able to tell, but if you expect your scripts to be cross-platform, don't count on using ithreads on the Unix side. TomP -- ------------- Tom Pollard [EMAIL PROTECTED] Schrodinger, Inc.646-366-9555 x102 - ___ Perl-Win32-Users mailing list Perl-Win32-Users@listserv.ActiveState.com To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
Re: Net::FTP capture Debug output
On Tue, 8 Nov 2005, $Bill Luebkert wrote: I have a logging subroutine that writes time stamped output to the logfile. My goal would be to capture the NET::FTP output to a var so that it could be handed to my logging procedure. Any possibilities like that? It appears to write direct to STDERR, so I doubt that will work. You could tail a log file in your code and grab any new log data into your program though if necessary. You can use the IO::Capture module(s) to grab your own script's stdout or stderr for purposes like this. See: http://search.cpan.org/~reynolds/IO-Capture-0.05/ TomP ----- Tom Pollard [EMAIL PROTECTED] Schrodinger, Inc.646-366-9555 x102 - ___ Perl-Win32-Users mailing list Perl-Win32-Users@listserv.ActiveState.com To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
Re: [threads] Forced termination
On Thu, 25 Aug 2005, $Bill Luebkert wrote:
> >>win32 doesn't have signals and any that are implemented don't actually
> >>interrupt the receiving process. I believe they just use simple message
> >>passing to fake it - so your mileage may vary.
> >
> > My point was precisely that "they" have arranged that this idiom works
> > under Windows. Your mileage should not vary; my experience is that
> > it always works.
>
> Which idiom is that ? SIGALRM ? or signals in general ? or your
> eval example specifically ?
The use of an alarm to timeout an eval{} block definitely works. I don't
know why it works, but it does. I do get the impression that alarms are
handled in some special way by perl, because this idiom works on all
platforms we've tried it on (Linux, AIX, IRIX, MacOSX, Windows), even
though ordinary signal delivery to a threaded app isn't predictable (that
is, which thread gets the signal?) on some of those platforms.
This didn't work in ActiveState's perl 5.6.1, but as of the 5.8.0 release
it was working. In any case, we use the technique widely in our
networking code and have found it to work reliably.
TomP
-
Tom Pollard [EMAIL PROTECTED]
Schrodinger, Inc.646-366-9555 x102
-
___
Perl-Win32-Users mailing list
Perl-Win32-Users@listserv.ActiveState.com
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
Re: [threads] Forced termination
On Thu, 25 Aug 2005, $Bill Luebkert wrote: > win32 doesn't have signals and any that are implemented don't actually > interrupt the receiving process. I believe they just use simple message > passing to fake it - so your mileage may vary. My point was precisely that "they" have arranged that this idiom works under Windows. Your mileage should not vary; my experience is that it always works. TomP --------- Tom Pollard [EMAIL PROTECTED] Schrodinger, Inc.646-366-9555 x102 - ___ Perl-Win32-Users mailing list Perl-Win32-Users@listserv.ActiveState.com To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
Re: [threads] Forced termination
On Thu, 25 Aug 2005, ajpeck wrote:
> Is there a way to forcibly terminate a thread after a specified time
> regardless if the thread is finished executing.
The standard eval{}-timeout technique apparently works well for threads.
That is, wrap the code that might hang for 5 min in an eval{} block and
use an alarm to time it out, e.g.
eval {
local %SIG{ALRM} = { die "TIMEOUT\n"; };
alarm 5;
<...your code here...>
alarm 0;
}
giveup() if $@ =~ /^TIMEOUT/;
...where giveup() does whatver you need to do to exit from the thread
prematurely. In general, threads and signals are awkward to use together,
but this particular idiom has been unproblematic for us in our threaded
perl apps.
TomP
> I have threads which
> call certain network functions that have very long hard coded timeouts,
> such as 5 mins. A no response of 5 secs is sufficient for me to know
> that there is a problem and the nature of it is not needed (device
> switched off, network break, etc). These threads only write out a status
> result (threads::shared variable) at the very end of the thread
> execution, thus terminating the thread early from within the parent
> thread and with no result written is good enough for me to know that
> there is a problem.
>
> Is there a way to impliment timed thread termination, if so I would
> appreciate it very much for a very simple example.
---------
Tom Pollard [EMAIL PROTECTED]
Schrodinger, Inc.646-366-9555 x102
-
___
Perl-Win32-Users mailing list
Perl-Win32-Users@listserv.ActiveState.com
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
Re[3]: Archive::Zip (2)
On Thu, 28 Apr 2005, [koi8-r] áÒÔÅÍ á×ÅÔÉÓÑÎ wrote: > Thank you! That works. > > However, as the zip gets bigger the perfomance suffers aloud. Which is > no surprise for the programm has to reread and rewrite zip every cycle > round. A downside to the zip format is that there's a "central directory" that needs to be updated when files are added to the archive. The tar format doesn't have anything like that, and it should be possible to append new files to the end of a tar archive efficiently. Unfortunately Archive::Tar doesn't appear to support that sort of usage; as far as I can see, the only options for IO are to read an entire archive into memory, or to write a complete in-memory archive to disk. If I had to do this, and scaling to really large numbers of files was critical, I'd try using Compress::Zlib directly. You can open a file handle to a new gzip file and then stream data to it, which is compressed and flushed to the disk file on the fly. You'd have to come up with your own way of distinguishing one file from the next and you'd have to write your own reader for the resulting archive, but the performance should be great and writing the archive ought to be quite simple. TomP ------------- Tom Pollard [EMAIL PROTECTED] Schrodinger, Inc.646-366-9555 x102 - ___ Perl-Win32-Users mailing list Perl-Win32-Users@listserv.ActiveState.com To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
Re[2]: Archive::Zip (2)
On Wed, 27 Apr 2005, [koi8-r] áÒÔÅÍ á×ÅÔÉÓÑÎ wrote:
> Tom, that is exactly how the script works for a moment ;) And that is
> why I'm concerned ;) Because: I have to generate ALL the files before
> moving them into the archive. I can't do this:
>
> foreach my $i (@whatever) {
> $file = generate_file($i)
> add $file to zip
> delete $file
> }
No, but you can do this:
my $zip = Archive::Zip->new();
my $status = $zip->writeToFileNamed($archive_name);
die "Error: unable to create zip archive\n" unless $status == AZ_OK;
foreach my $i (@whatever) {
my $file = &generate_file($i)
$zip = Archive::Zip->new();
$zip->read($archive_name);
$zip->addFile($file);
$zip->overwrite();
unlink $file || print STDERR "Warning: unable to delete \"$file\"\n";
}
> This matters when $whatever == 10, doesn't it? ;)
I don't know whether read(), addFile(), overwrite() performs the append
efficiently or not, but I believe this is the most straightforward way
to do an append using Archive::Zip.
TomP
-
Tom Pollard [EMAIL PROTECTED]
Schrodinger, Inc.646-366-9555 x102
-
___
Perl-Win32-Users mailing list
Perl-Win32-Users@listserv.ActiveState.com
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
Re: Archive::Zip (2)
On Wed, 27 Apr 2005, [koi8-r] áÒÔÅÍ á×ÅÔÉÓÑÎ wrote:
> I'm using ActivePerl 5.8.4.810 on Windows 2000. I'm writing a script
> where I have 1 files generated in cycle. Finally I need them to be
> packed in the zip archive. It would be easy and convinient to move (not
> just add) them into archive in the same cycle. However, as I understood,
> Archive::Zip doesn't allow me this simple task. Is that correct, or I
> misread something?
It's easy enough to use Archive::Zip in place of an external archiver.
Instead of
system('zip -rm archive.zip file1 file2 file3');
use a subroutine that does the same thing.
&archive_files("archive.zip","file1","file2","file3");
sub archive_files {
my $archive_name = shift;
my @files = @_;
# Create archive
my $zip = Archive::Zip->new();
# Add files to archive
foreach my $file (@files) {
$zip->addFile(file);
}
# Write archive to disk
my $status = $zip->writeToFileNamed($archive_name);
return 0 unless $status == AZ_OK;
# Delete archived files
foreach my $file (@files) {
unlink $file || print STDERR "Warning: unable to delete \"$file\"\n";
}
return 1;
}
(Note: this is untested, but it should make it obvious how to go about
doing what it is you said you wanted to do.)
TomP
-
Tom Pollard [EMAIL PROTECTED]
Schrodinger, Inc.646-366-9555 x102
-
___
Perl-Win32-Users mailing list
Perl-Win32-Users@listserv.ActiveState.com
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
RE: regex confusion...
On Mon, 21 Feb 2005, Peter Eisengrein wrote:
> sorry about that. a bit too quick on the trigger finger. I have the
> following:
>
> @files = grep { /^ccs_[cas|pri|atsm]/ && -f "$dir/$_" } readdir(DIR);
>
> There are other files in $dir that start with ccs but I only want ccs_cas
> ccs_pri or ccs_atsm but for some reason it seems to also match a filename
> that starts with ccs_trsum. What am I doing wrong?
'[...]' specifies a character set, not a list of alternatives. You need
to use parentheses, as in
@files = grep { /^ccs_(cas|pri|atsm)/ && -f "$dir/$_" } readdir(DIR);
TomP
---------
Tom Pollard [EMAIL PROTECTED]
Schrodinger, Inc.646-366-9555 x102
-
___
Perl-Win32-Users mailing list
Perl-Win32-Users@listserv.ActiveState.com
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
___
Perl-Win32-Users mailing list
Perl-Win32-Users@listserv.ActiveState.com
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
Re: Simple question
On Sat, 19 Feb 2005, Alberto Fernandez wrote: > Hello, i am new in Perl and i have a doubt that i think is very easy to > result. > I have this portion code: > > my $string = "cccabccc"; > my @array =$string =~ /ab|a/g; > > I execute this and array contains 1 element with value "ab". > > I would like to know if there is some way for array contains all the > matches "ab" and "a".(container and contents). Not that I can think of. The regular expression matches just one substring. If you want to allow that any given art of the original target string might be matched by different patterns, you'll need to to match against each of those patterns separately. In this case, the following would work, my @array = ($string =~ /ab/g); push @array, ($string =~ /a/g); TomP ----- Tom Pollard [EMAIL PROTECTED] Schrodinger, Inc.646-366-9555 x102 - ___ Perl-Win32-Users mailing list Perl-Win32-Users@listserv.ActiveState.com To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
RE: Reg. expression help
On Wed, 16 Feb 2005, Erich Beyrent wrote:
> > Can someone help me with the syntax of deleting the last word of text
> string.
> > i.e. $string = 'my abc 123'
> > Like to have only 'my abc'
>
> #!/usr/bin/perl
>
> $string = 'some string';
> $string =~ s/(?<=\s)(.*?)$//g;
>
> print "String = $string\n";
That deletes everything after the first space, not just the last word.
Also the '(.*?)' construction is bizarre, as is the 'g' modifier at the
end; if you want to match the rest of the string, you just need '.*$'.
A good regexp for deleting the last word would be
$string =~ s/\s*\S+\s*$//;
Tom
-
Tom Pollard [EMAIL PROTECTED]
Schrodinger, Inc.646-366-9555 x102
-
___
Perl-Win32-Users mailing list
Perl-Win32-Users@listserv.ActiveState.com
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
Re: Parse::RecDescent grammar error?
On Fri, 12 Nov 2004, Christopher Hahn wrote:
> I am trying to use Parse::Recdescent on fairly simple data,
> but am getting stuck on capturing an escaped carriage return.
>
> I am not able to make too many assumptions about the structure of the data,
> or I would probably just us a regexp.
[...]
> Words: /[\w\s\.\,]+/ { print "$item[1]\n"; }
> EscCR: /n/{ print "$item[1]\n"; }
> EscQt: /\\\"/ { print "$item[1]\n"; }
> };
Here, /n/ matches only a single backslah, followed by an 'n'. To
match two backslashes followed by an 'n', you need /n/. I guess
this is because '\\' interpolates as a single backslash even in a
single-quoted string. Here, the grammar is a single quoted string, and so
one level of backslash interpolation occurs before the regexp is even
processed by Parse::RecDescent; another level of interpolation occurs when
the regexp is processed.
TomP
-
Tom Pollard [EMAIL PROTECTED]
Schrodinger, Inc.646-366-9555 x102
-
___
Perl-Win32-Users mailing list
[EMAIL PROTECTED]
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
RE: regex in <>
On Fri, 5 Nov 2004, Beckett Richard-qswi266 wrote: > use strict; > use warnings; > my $pat = "abcdijklmnopuvwxyz"; > print "\$pat is $pat\n"; > $pat =~ /<(.*)>.*<(.*)>/; > my ($one, $two) = ($1, $2); > print "\$one is $one\n\$two is $two\n"; That does work in this case, but it's not terribly robust. If there were a third angle-bracketed substring, for instance, (or only one) this would no longer match them correctly. I'd always use one of the following: 1) $pat = /<([^>]*)>.*<([^>]*)>/; This always matches the first two angle-bracketed substrings, ignoring any others there might follow. 2) my @matches = ($pat =~ /<([^>]*)>/g); This matches any number of angle-bracketed substrings, returning them in a list. TomP > > -Original Message- > > pattern: abcdijklmnopuvwxyz > > > > >From the above pattern, How to find the parameter $1=efgh > > and $2=qrst within <>. What is the regex to written in perl? - Tom Pollard [EMAIL PROTECTED] Schrodinger, Inc.646-366-9555 x102 - ___ Perl-Win32-Users mailing list [EMAIL PROTECTED] To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
Re: how to have a perl script delete itself...
On Fri, 15 Oct 2004, Rhesa Rozendaal wrote: > Tom Pollard wrote: > > On Fri, 15 Oct 2004, Rhesa Rozendaal wrote: > > > >>unlink $0 or die "Error during self-destruct: $!"; > > > > I'd be surprised if that worked under Windows. Unlike Unix, I didn't > > think Windows allowed you to delete a file that was in use. > > What makes you think I didn't test it? My apologies for the implied slight. > And what makes you think the script is in use? A perl script is not an > executable. A good point, which I missed. Thanks for teaching me something today, TomP ----- Tom Pollard [EMAIL PROTECTED] Schrodinger, Inc.646-366-9555 x102 - ___ Perl-Win32-Users mailing list [EMAIL PROTECTED] To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
Re: how to have a perl script delete itself...
On Fri, 15 Oct 2004, Rhesa Rozendaal wrote: > unlink $0 or die "Error during self-destruct: $!"; I'd be surprised if that worked under Windows. Unlike Unix, I didn't think Windows allowed you to delete a file that was in use. TomP ------------- Tom Pollard [EMAIL PROTECTED] Schrodinger, Inc.646-366-9555 x102 - ___ Perl-Win32-Users mailing list [EMAIL PROTECTED] To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
Re: Help with program
On Sun, 10 Oct 2004, James wrote:
> I am writing a program to read from an input file and
> verify or not if the Quantity equal to the number of
> parts. The problem is some of the parts can be in 2 or
> 3 lines. Fromthe data file, the quantity match the
> number of parts but I cannot find the logic to write a
> program. Please help
The three problems with the original script were that:
(1) @pts was never re-initialized, so it just kept growing,
(2) the comparison between $qty and @pts can't be done until you
know you've processed all of the CONT lines for an item, and
(3) the number of items in @pts is length(@pts), or scalar(@pts),
or 1+$#pts, but not $#pts.
Here's a modified copy of the script that addresses those issues:
[...]
while () {
chomp;
next if /^\s*ITEM/;
if ( m!^\s*\d+! ) {
&check_qty($item,$qty,@pts) if $item;
($item, $qty, $part) = split ' ', $_, 3;
@pts = split(',',$part);
}
elsif ( s!^\s+CONT\s*!! ) {
push @pts, split(',',$_);
}
}
&check_qty($item,$qty,@pts);
sub check_qty {
my($item,$qty,@pts) = @_;
if ( $qty != 1+$#pts ) {
printf "Quantity does not match parts for item $item\n";
}
}
[...]
>
> use strict;
> use warnings;
>
>
> my ($item,$qty,$part,$part2);
> my @pts;
>
> while ()
> {
>
>chomp;
>next if /^\s*ITEM/;
>
>if ( m!^\s*\d+! )
> {
> ($item, $qty, $part) = split ' ', $_, 3;
> foreach ( split /,/, $part )
>{ push ( @pts, $_ ); }
> }
>elsif ( m!^\s+CONT! )
> {
> ( undef, $part2) = split ' ', $_, 2;
> foreach ( split /,/, $part2 )
>{ push ( @pts, $_ ); }
> }
>
> if ( $qty != $#pts )
> {
>print "Quantity does not match parts\n";
> }
> }
>
> __DATA__
> ITEMQTY PARTS
> 1 1 P3
> 2 1 MP4
> 3 4 J15,MP6,RC1,MP8,
> 4 4 MP9,RC19,MP11,MP1,
> 5 15 P1,P2,P15,P16,P17,
> CONT AP18,MP19,MP20,MP21,MP42,
> CONT BP23,MP24,MP25,MP26,MP27,
> 6 8 MP28,MP29,MP30,MP31,MP32,
> CONT AR3,AR34,MP35,
> 7 2 PS6,PS5
>
>
>
>
> __
> Do you Yahoo!?
> Take Yahoo! Mail with you! Get it on your mobile phone.
> http://mobile.yahoo.com/maildemo
> ___
> Perl-Win32-Users mailing list
> [EMAIL PROTECTED]
> To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
>
--
-
Tom Pollard [EMAIL PROTECTED]
Schrodinger, Inc.646-366-9555 x102
-
___
Perl-Win32-Users mailing list
[EMAIL PROTECTED]
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
