Poe and FTP
Hi I have been looking at Poe for some time but now I believe I have an application for it. I would welcome some general feedback on my comments. Currently we have (a number of) FTP scripts. They each work by sending files sequentially to a remote FTP servers via one of a number of methods. Net::FTP Net::SFTP Net::SFTP::Foreign::Compat I am trying to rationalise these scripts since they are rather in a mess! Each script is called by a different process so in theory we can have multiple copies of the same script running at the same time sending to the same or different FTP servers. I can't work out if Poe would be a suitable alternative. On the one hand, sending files sequentially to the same FTP server would seem to be limited by the throughput of the pipe to the server. Running Poe with multiple processes all trying to use the same pipe should be no faster since we are presumably limited by the pipe bandwidth. On the other hand, if one of the files had a problem and timed out, then other files may not be delayed and could in theory still be transferred (depending upon the reason for the problem). I can see that if I was sending files to different FTP servers then Poe would be of use since then I would be sending files down different pipes, but this is no better that having several processes, one for each server. I have also looked at the FTP components. Poe::Component::Net::FTP I understand that if I were to use the 'put' command then it would block other Poe components until the file was sent. For this reason I would have to use the lower level write command which adds complexity. Has anyone implemented a none-blocking put for this module? Poe::Component::Client::FTP I think this has the same problem, there is no high level put command. I am not yet convinced that for my application Poe is a suitable advantage, but I am willing to be convinced :-) Regards Ian This email is confidential and intended solely for the use of the individual to whom it is addressed. Any views or opinions presented are solely those of the author and do not necessarily represent those of RedBee Media Metadata. If you are not the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing, or copying of this email is strictly prohibited. If you have received this email in error please notify the sender. Red Bee Media Metadata is a trading name of Broadcasting Dataservices Limited. Registered in England and Wales No.: 2554733. Registered Office: 201 Wood Lane, London W12 7TP, UK. Broadcasting Dataservices Limited is a wholly owned subsidiary of Red Bee Media Limited.
RE: Poe and FTP
Typical. Having posted a question, I then find similar points in the archive. Sorry. However, I think some of my points may still be relevant and are not covered in the thread 'How does POE speed up operation in reality?'. Chris Fedde. Mentioned the use of a single host process (presumably a daemon) that could fork to manage a number of transfer processes. I assume this would only be a benefit if it connected to a number of different hosts, rather than a single host so that transfers down one pipe would not be blocked by the bandwidth of the slowest pipe. No benefit that I can see if there are multiple files to just one FTP server. Rocco Caputo. Made the excellent point that Poe is appropriate where something would otherwise block other work being done. In my example, all I want to do is to send files to a single server via FTP. There is nothing else to do (other than perhaps a log file output) which could be blocked making Poe unnecessary. Regards Ian -Original Message- From: Ian Docherty [mailto:ian.doche...@bds.tv] Sent: 07 May 2009 08:05 To: poe@perl.org Subject: Poe and FTP Hi I have been looking at Poe for some time but now I believe I have an application for it. I would welcome some general feedback on my comments. Currently we have (a number of) FTP scripts. They each work by sending files sequentially to a remote FTP servers via one of a number of methods. Net::FTP Net::SFTP Net::SFTP::Foreign::Compat I am trying to rationalise these scripts since they are rather in a mess! Each script is called by a different process so in theory we can have multiple copies of the same script running at the same time sending to the same or different FTP servers. I can't work out if Poe would be a suitable alternative. On the one hand, sending files sequentially to the same FTP server would seem to be limited by the throughput of the pipe to the server. Running Poe with multiple processes all trying to use the same pipe should be no faster since we are presumably limited by the pipe bandwidth. On the other hand, if one of the files had a problem and timed out, then other files may not be delayed and could in theory still be transferred (depending upon the reason for the problem). I can see that if I was sending files to different FTP servers then Poe would be of use since then I would be sending files down different pipes, but this is no better that having several processes, one for each server. I have also looked at the FTP components. Poe::Component::Net::FTP I understand that if I were to use the 'put' command then it would block other Poe components until the file was sent. For this reason I would have to use the lower level write command which adds complexity. Has anyone implemented a none-blocking put for this module? Poe::Component::Client::FTP I think this has the same problem, there is no high level put command. I am not yet convinced that for my application Poe is a suitable advantage, but I am willing to be convinced :-) Regards Ian This email is confidential and intended solely for the use of the individual to whom it is addressed. Any views or opinions presented are solely those of the author and do not necessarily represent those of RedBee Media Metadata. If you are not the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing, or copying of this email is strictly prohibited. If you have received this email in error please notify the sender. Red Bee Media Metadata is a trading name of Broadcasting Dataservices Limited. Registered in England and Wales No.: 2554733. Registered Office: 201 Wood Lane, London W12 7TP, UK. Broadcasting Dataservices Limited is a wholly owned subsidiary of Red Bee Media Limited.
Re: Porting POE to Moose
On 4/27/09 6:07 PM, sungo wrote: On (04/27 16:53), Evan Carroll wrote: You're actually worse than ugly, you're hideous. You're the ugly bastard brother of the ugly duck. Evan, there really are more polite and professional ways to announce a new project. Sending us an email that effectively reads You all fucking suck but you should help me on replacing your code does NOT make me want to work with you. Had you presented this any other way, I might have come helped you. However, you decided to present this by being a complete asshole about it. Moose is fantastic and you're doing it a great disservice by being a giant cockmaster. So, this occasional-core-contributor has the following to say: die in a fire. To berate someone for not being more polite and then to do the same yourself? How does that help anyone? Robert
Re: Bug in PoCo::Child when PID wraps round (fwd)
Hi,
just checking one more time: is anyone looking after PoCo::Child, or is
it orphaned?
TIA,
Tony.
On Wed, 15 Apr 2009, Tony Wildish wrote:
Hi,
I sent this to the bug-poe-component-ch...@rt.cpan.org list some time ago
but it looks like that list is dead. Is there someone here managing
PoCo::Child?
Cheers,
Tony.
-- Forwarded message --
Date: Tue, 24 Mar 2009 12:56:33 +0100 (CET)
From: Tony Wildish wild...@mail.cern.ch
To: bug-poe-component-ch...@rt.cpan.org
Subject: Bug in PoCo::Child when PID wraps round
Hi,
we have been using PoCo::Child in a project for a while now and have just
uncovered a bug. The internal bookkeeping for the association of wheelIDs
and PIDs is wrong, with the result that when the OS PID wraps round, the
wrong wheelID will be given to the 'done' event.
I'm using PoCo::Child version 1.39, perl 5.8.5, kernel
2.6.9-78.0.8.EL.cernsmp.
You can see the bug by inspection of the code. At line 192 we have:
my $id = $wheel-ID;
$self-debug(qq/run(): @$cmd, wheel=$id, pid=/ . $wheel-PID);
$self-{$PKG}{pids}{$wheel-PID} = $id;
so the {pids} are inserted keyed on $wheel-PID. But at line 268, the
clean up deletes as follows:
delete $self-{$PKG}{wheels}{$id};
delete $self-{$PKG}{pids}{$id};
so it's deleting keyed on wheel-ID, not $wheel-PID.
When the PIDs wrap round, the protection in sig_child is broken by this:
sub sig_child {
my ($kernel, $self, $pid, $rc) = @_[KERNEL, OBJECT, ARG1, ARG2];
my $id = $self-{$PKG}{pids}{$pid} || ;
# child death signals are issued by the OS and sent to all
# sessions; we want to handle only our own children
return unless $id;
Because the {pids} hash had the wrong entry deleted in the cleanup, when
the PIDs wrap, an old $id will be found where none should be. So sig_child
will not return here, and will cause 'done' to be fired with the old
wheelID, which is plainly wrong.
Also, the cleanup around line 268/269 should remove {CLOSED} and
{SIGCHLD} entries for the wheel too. That's less serious because it won't
cause a problem until wheel-IDs wrap round, which will take a lot longer,
but it's still a leak.
Cheers,
Tony.
Mapping package states with different code ref names
For some reason..when I execute the find method, which should really
execute the '_process_request' method the find isn't replaced with a
reference to _process_request
# CODE ATTEMPT #
my $session = $class-__spawn({easyDBI_alias = 'easydbi_alias'});
$kernel-post('PoeDevice', find = { id = 1, find_event =
\my_method_handler } );
##
## find and any keys of %command should all be mapped to _process_request
but they aren't any ideas why?
sub __spawn {
my $class = shift;
my %exp_commands = map { $_ = 1 } qw(find);
my $session = POE::Session-create(
package_states =
[
(__PACKAGE__) = [
'_start' = '_start',
'_stop' = '_stop',
(map { $_ = '_process_request' } keys %COMMANDS)
],
($class) = [
'find' = '_process_request'
]
],
## pass args into _start:
args = [...@_],
heap = {
class = $class
},
);
return $session;
}
