Re: [OT] clp.moderated, was- Re: Backticks as fast as XS
On 27 Sep 2001, Joe Schaefer wrote: [beginners lists] [2]: IIRC, they are at http://learn.perl.org/ yup, thousands of subscribers. Incredibly amounts of (helpful) mails. http:[EMAIL PROTECTED]/ http:[EMAIL PROTECTED]/ - ask ps. we had some network trouble around noon Thursday (PST) on the network where onion (serving learn.perl.org and others) is located, that's why you couldn't connect. =) -- ask bjoern hansen, http://ask.netcetera.dk/ !try; do(); more than a billion impressions per week, http://valueclick.com
RE: Backticks as fast as XS
-Original Message-
From: Doug MacEachern [mailto:[EMAIL PROTECTED]]
i'm guessing part of the difference in your code is due to
fprintf having
a pre-allocated buffer, whereas the SV's SvPVX has not been
pre-allocated
and gets realloc-ed each time you call sv_catpv. have a look
at the code
below, fprintf is faster than sv_catpvn, but if the SvPVX is
preallocated,
sv_catpvn becomes faster than fprintf:
timethese(1_000, {
fprintf = sub { TickTest::fprintf() },
svcat = sub { TickTest::svcat() },
svcat_pre = sub { TickTest::svcat_pre() },
});
Benchmark: timing 1000 iterations of fprintf, svcat, svcat_pre...
fprintf: 9 wallclock secs ( 8.72 usr + 0.00 sys = 8.72
CPU) @ 114.68/s (n=1000)
svcat: 13 wallclock secs (12.82 usr + 0.00 sys = 12.82
CPU) @ 78.00/s (n=1000)
svcat_pre: 2 wallclock secs ( 2.75 usr + 0.00 sys = 2.75
CPU) @ 363.64/s (n=1000)
Very interesting. I'll try that (I wish you'd been listening to clp.mod when
I posted - other perl gurus weren't much help).
Matt.
_
This message has been checked for all known viruses by Star Internet
delivered through the MessageLabs Virus Scanning Service. For further
information visit http://www.star.net.uk/stats.asp or alternatively call
Star Internet for details on the Virus Scanning Service.
[OT] clp.moderated, was- Re: Backticks as fast as XS
Matt Sergeant [EMAIL PROTECTED] writes:
-Original Message-
From: Doug MacEachern [mailto:[EMAIL PROTECTED]]
i'm guessing part of the difference in your code is due to
fprintf having a pre-allocated buffer, whereas the SV's SvPVX
has not been pre-allocated and gets realloc-ed each time you
call sv_catpv. have a look at the code below, fprintf is faster
than sv_catpvn, but if the SvPVX is preallocated,
sv_catpvn becomes faster than fprintf:
timethese(1_000, {
fprintf = sub { TickTest::fprintf() },
svcat = sub { TickTest::svcat() },
svcat_pre = sub { TickTest::svcat_pre() },
});
Benchmark: timing 1000 iterations of fprintf, svcat, svcat_pre...
fprintf: 9 wallclock secs ( 8.72 usr + 0.00 sys = 8.72
CPU) @ 114.68/s (n=1000)
svcat: 13 wallclock secs (12.82 usr + 0.00 sys = 12.82
CPU) @ 78.00/s (n=1000)
svcat_pre: 2 wallclock secs ( 2.75 usr + 0.00 sys = 2.75
CPU) @ 363.64/s (n=1000)
Very interesting. I'll try that (I wish you'd been listening to clp.mod when
I posted - other perl gurus weren't much help).
Matt.
OT type=rant
IMO clp.mod is a sinkhole for perl-related discussions, and like
Abigail[1], despite its noble intentions I think clp.mod fails to
serve the Perl community in any useful way. IME, this mailing list (and
p5p when appropriate) is one of the best places to ask about perl
performance-related issues (that aren't covered by Stas' guide,
of course :). You could also don the high rubber boots and try
clp.misc, which IMO seems to be a bit improved since the creation
of the beginners' [2] mailing lists.
[1]: (long, unwrapped url)
http://groups.google.com/groups?hl=engroup=comp.lang.perl.miscrnum=1selm=slrn820ckc.ls3.abigail%40alexandra.delanet.com
Personally I had similar experiences with clp.moderated,
so I don't read it anymore either.
[2]: IIRC, they are at
http://learn.perl.org/
but I can't connect to it at the moment.
/OT
--
Joe Schaefer
Backticks as fast as XS
Robin Berjon thought I should post this as a heads-up to anyone thinking what I thought: XS or pure perl code will always be faster than backticks or system() calls. Wrong. I spent some time converting some of our backtick programs to XS code here, and the result was absolutely zero difference in performance. I posted to c.l.p.moderated about this. The thread is here: http://groups.google.com/groups?hl=enframe=rightth=1cbfb00db194a925seekm= eb3031b9.0109100209.7c85168%40posting.google.com#link1 The OS was Linux (tested on kernel's 2.2 and 2.4). I think it's probably fairly OS dependant, but I figure most people are on linux. Matt. _ This message has been checked for all known viruses by Star Internet delivered through the MessageLabs Virus Scanning Service. For further information visit http://www.star.net.uk/stats.asp or alternatively call Star Internet for details on the Virus Scanning Service.
RE: Backticks as fast as XS
-Original Message- From: Doug MacEachern [mailto:[EMAIL PROTECTED]] On Wed, 26 Sep 2001, Matt Sergeant wrote: Robin Berjon thought I should post this as a heads-up to anyone thinking what I thought: XS or pure perl code will always be faster than backticks or system() calls. Wrong. matt your benchmark is severly flawed. for starters, your xs and external program do not do the same thing. your xs has the overhead of sv_catpv. As does backticks, surely? If you can tell me a way to make the code faster, damn I'll do it as we have a *lot* of emails to process :-) and who knows what else. Nothing else. I detailed this in the thread. if you want proof that there is overhead using backticks, compare the difference of calling an xsub that does _nothing_ vs. a backticked program that does _nothing_. That's not really what I was trying to say. Add some actual code to your test, and some printf's (and sv code to the XS) and the differences diminish fairly rapidly. I was trying to say don't make the _assumption_ that XS or pure Perl code will be faster. For example, I think a lot of people should re-evalute their fears of system calls to sendmail, especially where sendmail is qmail's implementation :-) [unfortunately this is one area not many people are willing to test, as they don't want to end up sending real emails, and a failed address path may be much quicker than a real one] There's also benefits of automatic cleanup with system calls, whereas XS can easily leak memory or leave files open (especially if it's not an external library - there's a lot of crap C coders around, and I include myself in that list :-). I just think it's something that needs less of a blanket statement. Matt. _ This message has been checked for all known viruses by Star Internet delivered through the MessageLabs Virus Scanning Service. For further information visit http://www.star.net.uk/stats.asp or alternatively call Star Internet for details on the Virus Scanning Service.
Re: Backticks as fast as XS
On Wed, 26 Sep 2001, Matt Sergeant wrote:
Robin Berjon thought I should post this as a heads-up to anyone thinking
what I thought: XS or pure perl code will always be faster than backticks
or system() calls.
Wrong.
matt your benchmark is severly flawed. for starters, your xs and
external program do not do the same thing. your xs has the overhead of
sv_catpv. and who knows what else. if you want proof that there
is overhead using backticks, compare the difference of calling an
xsub that does _nothing_ vs. a backticked program that does _nothing_.
test.c:
int main(int argc, char **argv, char **env)
{
return 1;
}
TickTest.xs:
#include EXTERN.h
#include perl.h
#include XSUB.h
MODULE = TickTest PACKAGE = TickTest
void
foo()
CODE:
test.pl:
use blib;
use TickTest ();
use Benchmark;
timethese(100_000, {
backtick = sub { `./test` },
xs = sub { TickTest::foo() },
});
results:
Benchmark: timing 10 iterations of backtick, xs...
backtick: 292 wallclock secs (18.68 usr 43.93 sys + 142.43 cusr 84.00 csys = 289.04
CPU) @ 1597.19/s (n=10)
xs: -1 wallclock secs ( 0.25 usr + 0.00 sys = 0.25 CPU) @ 40.00/s
(n=10)
(warning: too few iterations for a reliable count)
RE: Backticks as fast as XS
On Wed, 26 Sep 2001, Matt Sergeant wrote:
As does backticks, surely? If you can tell me a way to make the code faster,
damn I'll do it as we have a *lot* of emails to process :-)
maybe, i don't know in what way your code uses sv_catpv.
and who knows what else.
Nothing else. I detailed this in the thread.
yeahbut, i have not seen the code.
That's not really what I was trying to say. Add some actual code to your
test, and some printf's (and sv code to the XS) and the differences diminish
fairly rapidly.
right, that's the flaw, you're benchmarking fprintf vs sv_catpv
I was trying to say don't make the _assumption_ that XS or pure Perl code
will be faster.
and what i'm trying to say is that if both the xs code and external
program are doing the same thing, xs will be heaps faster than backticking
a program. your xsub and external program are not doing the same thing.
i'm guessing part of the difference in your code is due to fprintf having
a pre-allocated buffer, whereas the SV's SvPVX has not been pre-allocated
and gets realloc-ed each time you call sv_catpv. have a look at the code
below, fprintf is faster than sv_catpvn, but if the SvPVX is preallocated,
sv_catpvn becomes faster than fprintf:
timethese(1_000, {
fprintf = sub { TickTest::fprintf() },
svcat = sub { TickTest::svcat() },
svcat_pre = sub { TickTest::svcat_pre() },
});
Benchmark: timing 1000 iterations of fprintf, svcat, svcat_pre...
fprintf: 9 wallclock secs ( 8.72 usr + 0.00 sys = 8.72 CPU) @ 114.68/s (n=1000)
svcat: 13 wallclock secs (12.82 usr + 0.00 sys = 12.82 CPU) @ 78.00/s (n=1000)
svcat_pre: 2 wallclock secs ( 2.75 usr + 0.00 sys = 2.75 CPU) @ 363.64/s (n=1000)
#include EXTERN.h
#include perl.h
#include XSUB.h
static FILE *devnull;
MODULE = TickTest PACKAGE = TickTest
BOOT:
devnull = fopen(/dev/null, w);
void
fprintf()
CODE:
{
int i;
char buffer[8292];
for (i=0; isizeof(buffer); i++) {
fprintf(devnull, a);
}
}
void
svcat()
CODE:
{
int i;
char buffer[8292];
SV *sv = newSV(0);
for (i=0; isizeof(buffer); i++) {
sv_catpvn(sv, a, 1);
}
SvREFCNT_dec(sv);
}
void
svcat_pre()
CODE:
{
int i;
char buffer[8292];
SV *sv = newSV(sizeof(buffer)+1);
for (i=0; isizeof(buffer); i++) {
sv_catpvn(sv, a, 1);
}
SvREFCNT_dec(sv);
}
