Re: [OT] clp.moderated, was- Re: Backticks as fast as XS
On 27 Sep 2001, Joe Schaefer wrote: [beginners lists] [2]: IIRC, they are at http://learn.perl.org/ yup, thousands of subscribers. Incredibly amounts of (helpful) mails. http:[EMAIL PROTECTED]/ http:[EMAIL PROTECTED]/ - ask ps. we had some network trouble around noon Thursday (PST) on the network where onion (serving learn.perl.org and others) is located, that's why you couldn't connect. =) -- ask bjoern hansen, http://ask.netcetera.dk/ !try; do(); more than a billion impressions per week, http://valueclick.com
RE: Backticks as fast as XS
-Original Message- From: Doug MacEachern [mailto:[EMAIL PROTECTED]] i'm guessing part of the difference in your code is due to fprintf having a pre-allocated buffer, whereas the SV's SvPVX has not been pre-allocated and gets realloc-ed each time you call sv_catpv. have a look at the code below, fprintf is faster than sv_catpvn, but if the SvPVX is preallocated, sv_catpvn becomes faster than fprintf: timethese(1_000, { fprintf = sub { TickTest::fprintf() }, svcat = sub { TickTest::svcat() }, svcat_pre = sub { TickTest::svcat_pre() }, }); Benchmark: timing 1000 iterations of fprintf, svcat, svcat_pre... fprintf: 9 wallclock secs ( 8.72 usr + 0.00 sys = 8.72 CPU) @ 114.68/s (n=1000) svcat: 13 wallclock secs (12.82 usr + 0.00 sys = 12.82 CPU) @ 78.00/s (n=1000) svcat_pre: 2 wallclock secs ( 2.75 usr + 0.00 sys = 2.75 CPU) @ 363.64/s (n=1000) Very interesting. I'll try that (I wish you'd been listening to clp.mod when I posted - other perl gurus weren't much help). Matt. _ This message has been checked for all known viruses by Star Internet delivered through the MessageLabs Virus Scanning Service. For further information visit http://www.star.net.uk/stats.asp or alternatively call Star Internet for details on the Virus Scanning Service.
[OT] clp.moderated, was- Re: Backticks as fast as XS
Matt Sergeant [EMAIL PROTECTED] writes: -Original Message- From: Doug MacEachern [mailto:[EMAIL PROTECTED]] i'm guessing part of the difference in your code is due to fprintf having a pre-allocated buffer, whereas the SV's SvPVX has not been pre-allocated and gets realloc-ed each time you call sv_catpv. have a look at the code below, fprintf is faster than sv_catpvn, but if the SvPVX is preallocated, sv_catpvn becomes faster than fprintf: timethese(1_000, { fprintf = sub { TickTest::fprintf() }, svcat = sub { TickTest::svcat() }, svcat_pre = sub { TickTest::svcat_pre() }, }); Benchmark: timing 1000 iterations of fprintf, svcat, svcat_pre... fprintf: 9 wallclock secs ( 8.72 usr + 0.00 sys = 8.72 CPU) @ 114.68/s (n=1000) svcat: 13 wallclock secs (12.82 usr + 0.00 sys = 12.82 CPU) @ 78.00/s (n=1000) svcat_pre: 2 wallclock secs ( 2.75 usr + 0.00 sys = 2.75 CPU) @ 363.64/s (n=1000) Very interesting. I'll try that (I wish you'd been listening to clp.mod when I posted - other perl gurus weren't much help). Matt. OT type=rant IMO clp.mod is a sinkhole for perl-related discussions, and like Abigail[1], despite its noble intentions I think clp.mod fails to serve the Perl community in any useful way. IME, this mailing list (and p5p when appropriate) is one of the best places to ask about perl performance-related issues (that aren't covered by Stas' guide, of course :). You could also don the high rubber boots and try clp.misc, which IMO seems to be a bit improved since the creation of the beginners' [2] mailing lists. [1]: (long, unwrapped url) http://groups.google.com/groups?hl=engroup=comp.lang.perl.miscrnum=1selm=slrn820ckc.ls3.abigail%40alexandra.delanet.com Personally I had similar experiences with clp.moderated, so I don't read it anymore either. [2]: IIRC, they are at http://learn.perl.org/ but I can't connect to it at the moment. /OT -- Joe Schaefer
RE: Backticks as fast as XS
-Original Message- From: Doug MacEachern [mailto:[EMAIL PROTECTED]] On Wed, 26 Sep 2001, Matt Sergeant wrote: Robin Berjon thought I should post this as a heads-up to anyone thinking what I thought: XS or pure perl code will always be faster than backticks or system() calls. Wrong. matt your benchmark is severly flawed. for starters, your xs and external program do not do the same thing. your xs has the overhead of sv_catpv. As does backticks, surely? If you can tell me a way to make the code faster, damn I'll do it as we have a *lot* of emails to process :-) and who knows what else. Nothing else. I detailed this in the thread. if you want proof that there is overhead using backticks, compare the difference of calling an xsub that does _nothing_ vs. a backticked program that does _nothing_. That's not really what I was trying to say. Add some actual code to your test, and some printf's (and sv code to the XS) and the differences diminish fairly rapidly. I was trying to say don't make the _assumption_ that XS or pure Perl code will be faster. For example, I think a lot of people should re-evalute their fears of system calls to sendmail, especially where sendmail is qmail's implementation :-) [unfortunately this is one area not many people are willing to test, as they don't want to end up sending real emails, and a failed address path may be much quicker than a real one] There's also benefits of automatic cleanup with system calls, whereas XS can easily leak memory or leave files open (especially if it's not an external library - there's a lot of crap C coders around, and I include myself in that list :-). I just think it's something that needs less of a blanket statement. Matt. _ This message has been checked for all known viruses by Star Internet delivered through the MessageLabs Virus Scanning Service. For further information visit http://www.star.net.uk/stats.asp or alternatively call Star Internet for details on the Virus Scanning Service.
Re: Backticks as fast as XS
On Wed, 26 Sep 2001, Matt Sergeant wrote: Robin Berjon thought I should post this as a heads-up to anyone thinking what I thought: XS or pure perl code will always be faster than backticks or system() calls. Wrong. matt your benchmark is severly flawed. for starters, your xs and external program do not do the same thing. your xs has the overhead of sv_catpv. and who knows what else. if you want proof that there is overhead using backticks, compare the difference of calling an xsub that does _nothing_ vs. a backticked program that does _nothing_. test.c: int main(int argc, char **argv, char **env) { return 1; } TickTest.xs: #include EXTERN.h #include perl.h #include XSUB.h MODULE = TickTest PACKAGE = TickTest void foo() CODE: test.pl: use blib; use TickTest (); use Benchmark; timethese(100_000, { backtick = sub { `./test` }, xs = sub { TickTest::foo() }, }); results: Benchmark: timing 10 iterations of backtick, xs... backtick: 292 wallclock secs (18.68 usr 43.93 sys + 142.43 cusr 84.00 csys = 289.04 CPU) @ 1597.19/s (n=10) xs: -1 wallclock secs ( 0.25 usr + 0.00 sys = 0.25 CPU) @ 40.00/s (n=10) (warning: too few iterations for a reliable count)
RE: Backticks as fast as XS
On Wed, 26 Sep 2001, Matt Sergeant wrote: As does backticks, surely? If you can tell me a way to make the code faster, damn I'll do it as we have a *lot* of emails to process :-) maybe, i don't know in what way your code uses sv_catpv. and who knows what else. Nothing else. I detailed this in the thread. yeahbut, i have not seen the code. That's not really what I was trying to say. Add some actual code to your test, and some printf's (and sv code to the XS) and the differences diminish fairly rapidly. right, that's the flaw, you're benchmarking fprintf vs sv_catpv I was trying to say don't make the _assumption_ that XS or pure Perl code will be faster. and what i'm trying to say is that if both the xs code and external program are doing the same thing, xs will be heaps faster than backticking a program. your xsub and external program are not doing the same thing. i'm guessing part of the difference in your code is due to fprintf having a pre-allocated buffer, whereas the SV's SvPVX has not been pre-allocated and gets realloc-ed each time you call sv_catpv. have a look at the code below, fprintf is faster than sv_catpvn, but if the SvPVX is preallocated, sv_catpvn becomes faster than fprintf: timethese(1_000, { fprintf = sub { TickTest::fprintf() }, svcat = sub { TickTest::svcat() }, svcat_pre = sub { TickTest::svcat_pre() }, }); Benchmark: timing 1000 iterations of fprintf, svcat, svcat_pre... fprintf: 9 wallclock secs ( 8.72 usr + 0.00 sys = 8.72 CPU) @ 114.68/s (n=1000) svcat: 13 wallclock secs (12.82 usr + 0.00 sys = 12.82 CPU) @ 78.00/s (n=1000) svcat_pre: 2 wallclock secs ( 2.75 usr + 0.00 sys = 2.75 CPU) @ 363.64/s (n=1000) #include EXTERN.h #include perl.h #include XSUB.h static FILE *devnull; MODULE = TickTest PACKAGE = TickTest BOOT: devnull = fopen(/dev/null, w); void fprintf() CODE: { int i; char buffer[8292]; for (i=0; isizeof(buffer); i++) { fprintf(devnull, a); } } void svcat() CODE: { int i; char buffer[8292]; SV *sv = newSV(0); for (i=0; isizeof(buffer); i++) { sv_catpvn(sv, a, 1); } SvREFCNT_dec(sv); } void svcat_pre() CODE: { int i; char buffer[8292]; SV *sv = newSV(sizeof(buffer)+1); for (i=0; isizeof(buffer); i++) { sv_catpvn(sv, a, 1); } SvREFCNT_dec(sv); }