Perl-friendly message queue-like system
Mongers of Perl, I have a bunch of servers doing mail ilftering. I would like them to send tiny messages about the results of said filtering to a central point. I would then like something at the central point to pop messages off the queue and update a database. Think of it as sort of a centralised collection point for statistical data. If the odd message gets eaten, it really doesn't matter. I understand something like RabbitMQ or ActiveMQ might do what I want, however the former's website doesn't mention Perl and the latter I am terrified of, having used it. My Google-fu is failing me, what Perl-friendly system might I use for such a thing? Thanks -- Martin A. Brooks http://antibodyMX.net/ - antispam antivirus email filtering.
Re: Perl-friendly message queue-like system
On Thu, Sep 22, 2011 at 9:15 AM, Martin A. Brooks mar...@antibodymx.netwrote: My Google-fu is failing me, what Perl-friendly system might I use for such a thing? If it's definitely perl-only, and will never talk to anything else, you might want to consider TheSchwartz: https://metacpan.org/release/TheSchwartz Or one of its close relatives: https://metacpan.org/module/TheSchwartz::Simple https://metacpan.org/module/TheSchwartz::Moosified We've used TheSchwartz in production for about two years and haven't had any problems with the module. -- Chisel e: chi...@chizography.net w: http://chizography.net
Re: Perl-friendly message queue-like system
Hi, I've used ActiveMQ with a Perl STOMP client in the past. Works well, but at the time ActiveMQ was leaking memory quite a bit, I don't know if it's still the case now. The RabbitMQ site doesn't feature examples in Perl, but I've recently played with Net::RabbitMQ, and it seems to do the job alright. Cheers! J. On 22 September 2011 09:15, Martin A. Brooks mar...@antibodymx.net wrote: Mongers of Perl, I have a bunch of servers doing mail ilftering. I would like them to send tiny messages about the results of said filtering to a central point. I would then like something at the central point to pop messages off the queue and update a database. Think of it as sort of a centralised collection point for statistical data. If the odd message gets eaten, it really doesn't matter. I understand something like RabbitMQ or ActiveMQ might do what I want, however the former's website doesn't mention Perl and the latter I am terrified of, having used it. My Google-fu is failing me, what Perl-friendly system might I use for such a thing? Thanks -- Martin A. Brooks http://antibodyMX.net/ - antispam antivirus email filtering. -- Jerome Eteve. http://sigstp.blogspot.com/ http://twitter.com/jeteve
Re: Perl-friendly message queue-like system
On 22 September 2011 09:15, Martin A. Brooks mar...@antibodymx.net wrote: I have a bunch of servers doing mail ilftering. I would like them to send tiny messages about the results of said filtering to a central point. I would then like something at the central point to pop messages off the queue and update a database. Think of it as sort of a centralised collection point for statistical data. If the odd message gets eaten, it really doesn't matter. I understand something like RabbitMQ or ActiveMQ might do what I want, however the former's website doesn't mention Perl and the latter I am terrified of, having used it. Why not simply use RRDtool if you want statistical reporting - it's light, fast and easy to use http://oss.oetiker.ch/rrdtool/tut/rrd-beginners.en.html Regards, Peter
Re: Perl-friendly message queue-like system
Which RabbitMQ site are you talking about? Here're the RabbitMQ+Perl links, from the RabbitMQ devtools page: http://www.rabbitmq.com/devtools.html#perl-dev AnyEvent::RabbitMQ looks good: https://github.com/cooldaemon/AnyEvent-RabbitMQ Ciao Richard -- Richard Foley Ciao - shorter than AufWiederSehen! http://www.rfi.net/books.html Hi, I've used ActiveMQ with a Perl STOMP client in the past. Works well, but at the time ActiveMQ was leaking memory quite a bit, I don't know if it's still the case now. The RabbitMQ site doesn't feature examples in Perl, but I've recently played with Net::RabbitMQ, and it seems to do the job alright. Cheers! J. On 22 September 2011 09:15, Martin A. Brooks mar...@antibodymx.net wrote: Mongers of Perl, I have a bunch of servers doing mail ilftering. I would like them to send tiny messages about the results of said filtering to a central point. I would then like something at the central point to pop messages off the queue and update a database. Think of it as sort of a centralised collection point for statistical data. If the odd message gets eaten, it really doesn't matter. I understand something like RabbitMQ or ActiveMQ might do what I want, however the former's website doesn't mention Perl and the latter I am terrified of, having used it. My Google-fu is failing me, what Perl-friendly system might I use for such a thing? Thanks -- Martin A. Brooks http://antibodyMX.net/ - antispam antivirus email filtering.
Re: Perl-friendly message queue-like system
On 22 Sep 2011, at 09:49, Richard Foley wrote: AnyEvent::RabbitMQ looks good: https://github.com/cooldaemon/AnyEvent-RabbitMQ And works well, I'm using it fairly extensively. Cheers t0m
Re: Perl-friendly message queue-like system
On 22 Sep 2011, at 09:15, Martin A. Brooks wrote: I have a bunch of servers doing mail ilftering. I would like them to send tiny messages about the results of said filtering to a central point. I would then like something at the central point to pop messages off the queue and update a database. Think of it as sort of a centralised collection point for statistical data. If the odd message gets eaten, it really doesn't matter. [snip] My Google-fu is failing me, what Perl-friendly system might I use for such a thing? I'd consider using Spread, even though the Perl bindings for it are a bit unloved, see the review. http://search.cpan.org/~jesus/Spread-3.17.4.4/ The underlying Spread toolkit is a very reliable and well-designed *group messaging* framework. Regrettably the Perl bindings are for an entire major version behind the latest Spread toolkit release, but I've had very good results using Spread 3.17 for a reliable, general-purpose log aggregation tool for well over 5 years and in particular, it can handle widely distributed nodes over TCP, rather than the easy case of everything in the same broadcast domain. I would guess it works extremely well for the case you're describing. It's not a message queueing system in that it's *not* intended to queue messages indefinitely, but it's good for aggregating messages reliably, which sounds like your main goal. A key point to remember is that message *writers* don't need to join groups to write to them. If your app joins a group, your inbound message buffers will fill up very quickly if you don't read them and writers, in this context, don't care about reading from other nodes. You're meant to run a Spread daemon on every machine in the group and local clients connect to the local daemon which then manages the message delivery to the other Spread daemons on the other hosts, which in turn manage the message delivery to their local clients. - Mark
Re: Perl-friendly message queue-like system
On 22 Sep 2011, at 10:22, Dirk Koopman wrote: On 22/09/11 09:15, Martin A. Brooks wrote: Mongers of Perl, I have a bunch of servers doing mail ilftering. I would like them to send tiny messages about the results of said filtering to a central point. I would then like something at the central point to pop messages off the queue and update a database. Think of it as sort of a centralised collection point for statistical data. If the odd message gets eaten, it really doesn't matter. ZeroMQ seems to be fairly sane. http://www.zeromq.org/bindings:perl Dirk The *name* of ZeroMQ always annoys me. ITS NOT A QUEUE. Its a Protocol/socket library, if you want to actually have any kind of queueing you need to write your own 'broker'
Re: Perl-friendly message queue-like system
On 22 September 2011 11:06, Mark Blackman m...@blackmans.org wrote: On 22 Sep 2011, at 09:15, Martin A. Brooks wrote: I have a bunch of servers doing mail ilftering. I would like them to send tiny messages about the results of said filtering to a central point. I would then like something at the central point to pop messages off the queue and update a database. Think of it as sort of a centralised collection point for statistical data. If the odd message gets eaten, it really doesn't matter. See also previous discussion on this topic... http://london.pm.org/pipermail/london.pm/Week-of-Mon-20110523/thread.html See ActiveMQ (was Re: Devel::Cover with Moose?) thread Leo
Re: Perl in shared hosting environments
Hello. 2011/09/21 12:04:47 +0100 Dirk Koopman d...@tobit.co.uk = To l...@cuckoo.org : DK Then you can switch between mod_perl / FastCGI / Starman / Twiggy to your DK hearts content (we found Starman is REALLY fast). DK DK I am happy to be educated, but I found Plack introduced a load of DK dependencies that I did not want, it *is* another layer which cannot DK help but reduce speed - which may not matter - but did to me at the time. Then you may find FCGI::Spawn https://github.com/petr999/fcgi-spawn/tree/devel -- Peter Vereshagin pe...@vereshagin.org (http://vereshagin.org) pgp: A0E26627
Re: Perl-friendly message queue-like system
On 22 Sep 2011, at 11:06, Mark Blackman wrote: I'd consider using Spread, even though the Perl bindings for it are a bit unloved, see the review. IME (YMMV etc) spread works well when it works, but is a nightmare to debug when it doesn't. I'm personally against it, but then there are people who are using it very successfully. shrug Cheers t0m
Re: Perl-friendly message queue-like system
On 22 Sep 2011, at 12:00, Tomas Doran wrote: On 22 Sep 2011, at 11:06, Mark Blackman wrote: I'd consider using Spread, even though the Perl bindings for it are a bit unloved, see the review. IME (YMMV etc) spread works well when it works, but is a nightmare to debug when it doesn't. I'm personally against it, but then there are people who are using it very successfully. shrug Ok, I wouldn't be surprised to find that 0MQ is a big step up from Spread. Low frequency (10/sec/node) log aggregation isn't very demanding, so perhaps that's why we've seen no pain. - Mark
Re: Perl-friendly message queue-like system
On Thu, 22 Sep 2011 10:28:40 +0200, Jérôme Étévé jerome.et...@gmail.com wrote: The RabbitMQ site doesn't feature examples in Perl Now it says [More] w/ links to Perl examples too, instead of .NET. -- Cosimo
Re: Perl-friendly message queue-like system
On Thu, 2011-09-22 at 09:15 +0100, Martin A. Brooks wrote: I have a bunch of servers doing mail ilftering. I would like them to send tiny messages about the results of said filtering to a central point. I would then like something at the central point to pop messages off the queue and update a database. Think of it as sort of a centralised collection point for statistical data. If the odd message gets eaten, it really doesn't matter. Would a simple syslog based solution do the job? I still have a simple system in place that uses syslog to distribute messages as necessary with the target server throwing them into a pipe and a very simple perl script taking messages from the pipe and updating an authentication database. It is very simple and works very well.
Re: Perl-friendly message queue-like system
On Sep 22, 2011, at 10:15 AM, Martin A. Brooks wrote: Mongers of Perl, I have a bunch of servers doing mail ilftering. I would like them to send tiny messages about the results of said filtering to a central point. I would then like something at the central point to pop messages off the queue and update a database. Think of it as sort of a centralised collection point for statistical data. If the odd message gets eaten, it really doesn't matter. I understand something like RabbitMQ or ActiveMQ might do what I want, however the former's website doesn't mention Perl and the latter I am terrified of, having used it. My Google-fu is failing me, what Perl-friendly system might I use for such a thing? Maybe syslogging to a central host is what you really need? And then process the syslog logs ? Liz
Perl Skills Test
Interesting question from a training client: Do you know of a general Perl skills test (on-line or paper) that we could give to our Perl developers before you turn up. The idea is that the output could feed into the training you are doing for us. I don't[1]. Does anyone else have any ideas? I suppose I could write something. But I don't really have time. I don't want to get into the whole certification issue. That's not what this is about at all. Dave... [1] Well, other than Brainbench but that costs money and the quality of the questions is somewhat dubious.
Re: Perl Skills Test
What about the good old Perl Purity Test? I reckon some bits are out of date, but some others might still be useful. J. On 22 September 2011 14:14, Dave Cross d...@dave.org.uk wrote: Interesting question from a training client: Do you know of a general Perl skills test (on-line or paper) that we could give to our Perl developers before you turn up. The idea is that the output could feed into the training you are doing for us. I don't[1]. Does anyone else have any ideas? I suppose I could write something. But I don't really have time. I don't want to get into the whole certification issue. That's not what this is about at all. Dave... [1] Well, other than Brainbench but that costs money and the quality of the questions is somewhat dubious. -- Jerome Eteve. http://sigstp.blogspot.com/ http://twitter.com/jeteve
Re: Perl Skills Test
On 22 Sep 2011, at 14:14, Dave Cross wrote: [1] Well, other than Brainbench but that costs money and the quality of the questions is somewhat dubious. I was about to suggest that. If we have a brainbench subscriber here, could we cull some quesitons and sanitise them? :)
Re: Perl-friendly message queue-like system
On 22 Sep 2011, at 12:33, Jason Clifford wrote: On Thu, 2011-09-22 at 09:15 +0100, Martin A. Brooks wrote: I have a bunch of servers doing mail ilftering. I would like them to send tiny messages about the results of said filtering to a central point. I would then like something at the central point to pop messages off the queue and update a database. Think of it as sort of a centralised collection point for statistical data. If the odd message gets eaten, it really doesn't matter. Would a simple syslog based solution do the job? I still have a simple system in place that uses syslog to distribute messages as necessary with the target server throwing them into a pipe and a very simple perl script taking messages from the pipe and updating an authentication database. It is very simple and works very well. I almost suggested syslog also. :) My impression is that the only real requirement is avoiding calling fsync() on the database server once per log message (i.e. aggregating several seconds worth and doing a bulk insert), so I can imagine something like this being perfect.. There are other advantages of having a 'real' message queue, and if you're likely to later want one for other things then that's entirely fair. But if this is the only problem like this you're likely to have to want to solve, I'd be very tempted to just steal Jason's code. :) Cheers t0m
Re: Perl Skills Test
On 22 September 2011 14:14, Dave Cross d...@dave.org.uk wrote: Interesting question from a training client: Do you know of a general Perl skills test (on-line or paper) that we could give to our Perl developers before you turn up. my employer -2 had a test they would use as a vetting tool. It was only about a half a page, but perl being what it is there was more than one way to answer all the questions: one elegant, and probably several that were hard work, the idea being to find more of the elegant options. Either way, if your response was deemed to be not too useless, you're answers would be discussed at interview. -- Nonnullus unus commodo reddo is mihi. ABC*D1EFGHIJK2.LMNO3*4PQRST*ITUBE-STANDARD-ANTI-BULLSHEIT-EMAIL*U.56X
Re: Perl-friendly message queue-like system
On Thu, Sep 22, 2011 at 02:48:32PM +0100, Tomas Doran wrote: There are other advantages of having a 'real' message queue ... I have yet to come across a situation where you needed a real message queue and can't just use a table in a database (replicated/clustered if necessary) with an auto-incrementing id. -- David Cantrell | Godless Liberal Elitist What is the difference between hearing aliens through the fillings in your teeth and hearing Jesus in your heart?
Re: Perl-friendly message queue-like system
On 22/09/11 15:56, David Cantrell wrote: On Thu, Sep 22, 2011 at 02:48:32PM +0100, Tomas Doran wrote: There are other advantages of having a 'real' message queue ... I have yet to come across a situation where you needed a real message queue and can't just use a table in a database (replicated/clustered if necessary) with an auto-incrementing id. FLAME ALERT!
Re: Perl-friendly message queue-like system
On 22 September 2011 15:56, David Cantrell da...@cantrell.org.uk wrote: On Thu, Sep 22, 2011 at 02:48:32PM +0100, Tomas Doran wrote: There are other advantages of having a 'real' message queue ... I have yet to come across a situation where you needed a real message queue and can't just use a table in a database (replicated/clustered if necessary) with an auto-incrementing id. But then you have to use a real database that can scalably generate auto-inc primary keys. Historically, MySQL didn't until about a year ago. Given the use case is for statistical reporting on what an app has been doing, rrdtool does the entire job really simply with no need for any fancy queueing software at all. We used it to measure activity at BBC WS and it worked well. We also used ActiveMQ which leaks memory and needs restarting unless you are really careful how you configure and use it. Regards, Peter
Re: Perl Skills Test
On 22 September 2011 14:14, Dave Cross d...@dave.org.uk wrote: Interesting question from a training client: Do you know of a general Perl skills test (on-line or paper) that we could give to our Perl developers before you turn up. At an interview with Auntie a few years ago, they gave me a Perl script and asked me to highlight all the problems/errors in it. Not sure if they were flattering me, but I found quite a few they didn't seem to be aware of.. Chris
Re: Perl-friendly message queue-like system
Hello, Given the use case is for statistical reporting on what an app has been doing, rrdtool does the entire job really simply with no need for any fancy queueing software at all. The problem is not the database I store stuff in, it's getting stuff to the database in the first place. The final data store will almost certainly be postgres, I see no reason to use anything else. I don't want to use any kind of remote syslog as something then has to parse potentially quite a lot of unstructured data and then be idempotent if fed the same log again for some reason. Thanks for all the suggestions, I shall take a look at each. Regards -- Martin A. Brooks http://antibodyMX.net/ - antispam antivirus email filtering.
Re: Perl-friendly message queue-like system
On Thu, Sep 22, 2011 at 04:38:24PM +0100, Martin A. Brooks wrote: The problem is not the database I store stuff in, it's getting stuff to the database in the first place. The final data store will almost certainly be postgres, I see no reason to use anything else. I don't want to use any kind of remote syslog as something then has to parse potentially quite a lot of unstructured data Perl, meet data; data, Perl. and then be idempotent if fed the same log again for some reason. It should be possible to mark each datum uniquely at the first stage of logging, via some combination of hostname, datestamp and (if necessary) application name or process id. That should be plenty; any script feeding a database should be capable of not re-inserting the same data if it is already present. MQ-style solutions don't magically give you any greater protection from duplicate data if you fail to identify the data uniquely from the start. And how do you intend to replay the data you're passing into MQ? I guess you'd have to log it... -- Bruce Vajazzle - giving new meaning to the phrase I'll scratch your eyes out.
Re: Perl-friendly message queue-like system
Hi Bruce, - Original Message - From: Bruce Richardson itsbr...@workshy.org To: london pm london.pm@london.pm.org Sent: Thursday, 22 September, 2011 5:08:02 PM Subject: Re: Perl-friendly message queue-like system MQ-style solutions don't magically give you any greater protection from duplicate data if you fail to identify the data uniquely from the start. And how do you intend to replay the data you're passing into MQ? I guess you'd have to log it... Well that was entirely the point really. I don't actually care if a little bit of data gets dropped on the floor and I would never expect or want to replay data. A message is emitted, stuffed into a queue and then processed. If any part of this fails for 1 message in every few thousand (we're handling something like 120,000 mail connections per day), meh. It's not billing data, it really doesn't matter. As it happens, the data is already handled locally in a transactional way. Each of my mail filtering nodes runs a postgres database that exim logs to. Rather than trying to merge N databases to get some pretty stats, I figured just having each node fling crumbs of data, monkey-like, off to some server would be a nice way of seeing that _something_ has happened. -- Martin A. Brooks http://antibodyMX.net/ - antispam antivirus email filtering.
[ANNOUNCE] London Perl Workshop: Newsletter #2
(If you are receiving this any you use a language different to Perl, or are new to programming, fear not and please read on :) ) Hi All, In this newsletter: 1. New to Perl, use another language? 2. Workshops and events 3. Further call for speakers 4. Before and after conference events 5. Conference flyers 6. Facebook Page 7. DuckDuckGo t-Shirts and DuckDuckgo logo on the day 1. New to Programming, new to Perl, use another language? If you are new to the Perl language, or are a practitioner of another programming language then we would love to welcome you to the London Perl Workshop this year. The London Perl Workshop is a free event held on Saturday, November 12th, at the University of Westminster, Cavendish Campus, London. And when we say free' we do mean free as in Beer. There are: free attendance, free sponsor goodies, free coffee/cakes in the morning, free presentations by world-renowned speakers and new speakers, free workshops (see next section) and then free beer (usually 1-3+ rounds dependent on number of attendees) and food at the social event after the conference. We would also like to welcome those of you taking your first steps into the higher languages and to programming, come along to a conference that embraces newbies, be prepared for spirited debate and to have your understanding stretched, but in a friendly way with a beer and a cake of choice :). The reason we would especially like to welcome you this year (you are welcome any time) is that there is a glut of companies in the Perlverse seeking new programmers, some of these companies seek experienced Perl personnel, but many of them also seek experienced polyglot programmers as experience of more than one language brings benefits and they have on the job training where you can use your existing skills while learning new ones, there are also companies seeking graduates and have graduate internship programs so this is the right time to brush up your skills in Perl and see what has changed, and trust me a lot has. So take the plunge, at the very least you can see some of the new things in a rival/new language and get a free drink. Visit: www.londonperlworkshop.org.uk and register today ;). 2. Workshops and events To continue from previous announcements I can now confirm that we have three workshops/training sessions planned and accepted and two more in the pipeline. Ian Norton will be inviting all newcomers to Perl to come along to a training session intended to give you a first taste in how to use the language. Dave Cross will be teaching us what has changed in Perl since 5.10 up until 5.14 (and 5.15.3) and what the future holds for 5.16 and beyond. Tatsuhiko Miyagawa will be presenting a workshop on Plack, CPANM and more (there are other great modules that spiral from this man's mind). There are two other workshops under negotiation one on Dancer and one on Dependency Injection, Test::Spec/Test::BDD::Cucumber, more information will be announced as these are all clarified and scheduled. 3. Further call for speakers I must note that it is only four weeks until the call for papers closes for this year and so I expect a flurried rush of activity as you all now scramble to submit to make sure your presentation is in well before the deadline so that we can come to appreciate it for the gem of magnificence it truly is. In other words, please submit earlier rather than later as it helps us to start building up a picture of the day. Thanks. 4. Before and after conference events As always the Wiki is up and running and I will fill in the details of the after event as soon as they are finalised. We will be attending the same public house as we did last year and I expect they will put on the same level of fabulous beers and food as they did then, we have received generous contributions from sponsors and are open to more if other sponsors wish to come forward. The principal sponsor for the evening however has been selected. If someone wishes to organise the pre-conference event then please choose somewhere central London that we can attend and chew over the following days activities. I realise that London.pm has a drinks tzar but i don't expect to push this onerous charge onto him unless he truly wishes it. 5. Conference flyers I have created a conference flyer and would like your help, please print off some copies and place them in appropriate locations at work, colleges, universities as soon as you can, I would love to see a lot of new faces at LPW this year as there is a real surge of people needed in our growing job market and community. The flyer already prepared is A4 (though can be reduced to A5) and is suitable to print on a home/small office printer in either colour or black and white. Please find a pdf at: http://www.mdk.me.uk/community/lpw2011/lpw-2011-flyer1.pdf 6. Facebook Page The London Perl Workshop now has a Facebook Page to be promoted using
Re: Perl Skills Test
On Thursday, September 22, 2011, Jones, Christopher c.jo...@ucl.ac.uk wrote: On 22 September 2011 14:14, Dave Cross d...@dave.org.uk wrote: Interesting question from a training client: Do you know of a general Perl skills test (on-line or paper) that we could give to our Perl developers before you turn up. At an interview with Auntie a few years ago, they gave me a Perl script and asked me to highlight all the problems/errors in it. Not sure if they were flattering me, but I found quite a few they didn't seem to be aware of.. I've been on the other side of that interview. I was astonished by how bad the script was.
Re: Perl-friendly message queue-like system
The problem is not the database I store stuff in, it's getting stuff to the
database in the first place. The final data store will almost certainly be
postgres, I see no reason to use anything else.
If volume is not a problem, sure.
We already had 10GB in our PostgreSQL db and didn't want any more coming
from generating rolling 5 minute, 15 minutes, 30 minutes, hourly (etc.)
stats - real time graphs for our monitoring screens with historic period
comparison across a range of measures .
In case it's of use for someone else here's an easy way to do it.
With RRD you create a logging database file like
$ rrdtool create transfer.rrd \
--start N --step 300 \
DS:transfer_in_secs:GAUGE:1200:0:600 \
DS:files_transferred:GAUGE:1200:0:20 \
DS:transfer_errors:GAUGE:1200:0:20 \
DS:average_errors:COMPUTE:transfer_errors,files_transferred,/,100,* \
RRA:AVERAGE:0.5:1:3600 \
RRA:MAX:0.5:1:3600 \
RRA:MIN:0.5:1:3600 \
RRA:AVERAGE:0.5:4:9600 \
RRA:AVERAGE:0.5:24:6000
that logs 3 data points and computes one more
# in your app's logging configuration set up a Log4perl logger to write RRD
entries
use Log::Log4perl;
my $transfer_rrd_db_path = ./transfer.rrd;
Log::Log4perl-init(
...
log4perl.category.TRANSFERRRD = INFO, TransferRRD,
log4perl.appender.TransferRRD = Log::Log4perl::Appender::RRDs,
log4perl.appender.TransferRRD.dbname = $transfer_rrd_db_path,
log4perl.appender.TransferRRD.layout =
Log::Log4perl::Layout::PatternLayout,
log4perl.appender.TransferRRD.layout.ConversionPattern = N:%m,
...
);
# now in your code when you do some work
get_logger('TRANSFERRRD')-info(
$time_elapsed . ':' .
$files_transferred . ':' .
$total_errors
);
then you can create a graph (if that's what you want) with
$ rrdtool graph ./transfer.rrd \
--start -1day --title=Average transfer time --vertical-label secs \
--height 200 --width 400 \
DEF:var_transfer_in_secs=transfer.rrd:transfer_in_secs:AVERAGE \
DEF:var_files_transferred=transfer.rrd:files_transferred:AVERAGE \
DEF:var_transfer_errors=transfer.rrd:transfer_errors:AVERAGE \
LINE1:var_transfer_in_secs#ff:transfer time in seconds \
AREA:var_files_transferred#33BB22:files transferred \
AREA:var_transfer_errors#A1A1A1:transfer errors
More at
http://oss.oetiker.ch/rrdtool/doc/rrdtool.en.html
Regards, Peter
http://perl.dragonstaff.co.uk
Re: Perl in shared hosting environments
On Wed, Sep 21, 2011 at 4:04 AM, Dirk Koopman d...@tobit.co.uk wrote: Check out Plack: http://plackperl.org/ - https://metacpan.org/module/Plack http://blog.plackperl.org/2011/08/plack-basics-for-perl-websites-yapceu-2011.html Then you can switch between mod_perl / FastCGI / Starman / Twiggy to your hearts content (we found Starman is REALLY fast). I am happy to be educated, but I found Plack introduced a load of dependencies that I did not want, Plack is the reference implementation of PSGI handlers and middleware, and it only has a few dependencies that could potentially be already dependent in most cases if you're doing a web development in perl (such as URI and LWP), but that mileage might vary. The nice thing about Plack/PSGI however is that you *decouple* your application code from the runtime environment such as mod_perl and FastCGI - you write your application to target the PSGI interface, then your app will run on any environments that supports PSGI (via Plack handlers) such as mod_perl, FastCGI, Starman etc. with absolutely zero changes. it *is* another layer which cannot help but reduce speed - which may not matter - but did to me at the time. Switching to Plack/PSGI actually gives better performance in many cases, particularly because PSGI uses Perl's native data types such as hash reference, array reference instead of objects with hundreds of method calls. Wrapping the interface with a nice OO is the job of framework, not PSGI. It does *not* get in the way. For my simple (and naive) Hello World application testing, Starman, Starlet and Feersum web servers got a better performance than CGI.pm (with mp2 wrapper) + mod_perl, for example. http://www.reddit.com/r/perl/comments/h6qqr/the_psgi_is_the_limit/ has an amusing comment thread about the performance and benefit of PSGI over existing wrappers like CGI.pm, started by shi4 - you might need to click the thread to collapse to read the whole thread. You also get a lot (160+ modules) of nice middleware available. More software to, at least potentially, get in the way or add unnecessary dependencies or unwanted constraints. PSGI interface and middleware works consistent as an interface between the application and servers. It doesn't get in the way. -- Tatsuhiko Miyagawa
Re: Perl in shared hosting environments
On Wed, Sep 21, 2011 at 8:49 AM, Randal L. Schwartz mer...@stonehenge.com wrote: Dirk == Dirk Koopman d...@tobit.co.uk writes: Dirk Reason include: You forgot: * My application only requires content generation, and I can completely ignore the other 14 phases of serving, because I don't want custom redirects, authentication, authorization, mime interpretation, and/or logging and other things written in the language of my choice: Perl. This is where PSGI stands out. If you write a couple of mod_perl handlers that work on these 14 phases of serving, then it will *only* work with mod_perl, putting aside the fact that mod_perl1 handlers won't work with mod_perl 2. If you write PSGI middleware that does URL rewrites, authentication, authorization, custom redirects, logging etc. as PSGI middleware, then you can use that with ANY of PSGI supported web servers (Starman, mod_perl, FastCGI etc.) and ANY of PSGI supported frameworks (Catalyst, Dancer, Mojolicious etc.) You can see the examples of middleware by searching on CPAN: https://metacpan.org/search?q=plack%3A%3Amiddleware Seriously... *nothing* can compete with mod_perl as far as its reach into Apache. *Nothing*. With mod_perl, you can inject new behavior at every level of decision making that Apache does. FastCGI is replacing just *one* of those 14 stages. True: you can further its reach into *Apache* and nothing else. -- Tatsuhiko Miyagawa
