Re: Signal delivery in POE and time warp.
On Friday, Aug 29, 2003, at 01:05 US/Eastern, Rocco Caputo wrote:
On Thu, Aug 28, 2003 at 05:27:22PM -0400, Dmitri Tikhonov wrote:
I was looking at POE::Kernel again to try to figure out this problem,
and
then I re-visited this thread. It hit me that because of semantic
differences between alarm() and delay(), POE::Kernel would have to use
_two_ time scales instead of one it uses right now (system time). In
an
event loop, delay() wants correct time increments independent of
system
time. I wrote a proof-of-concept time() subroutine to be used in
POE::Kernel that simply read number of seconds from /proc/uptime. No
matter what I did with system time, children were reaped at
appropriate
moments. Fine.
Sounds good, but also full of problems. It will require a lot of
maintenance to support different monotonic timer functions for each
operating system.
Maybe the better thing to do is to use a standard API such as
POSIX::times as a default. This solves the problem uniformly for
platforms that have *proper* implementations of POSIX. This is
probably preferable to using a non-portable Linux specific interface
such as /proc/uptime. In cases where a monatomic increasing
POSIX::times does not exist, support can be added for a platform
specific monatomic timer.
Now let's consider alarms. Alarms that are called with time = time()
+ 5 are
not truly alarms -- they're delays (of course, because of the way
alarm is
set, the argument specifies actual time). Alarms that specify a
future time
point independent of current time must use system time to operate.
An alarm
is expected to be triggered when the clock strikes, thus it is
supposed to
wait for system time to catch up to it. I don't see how delays can be
implemented using alarms or vice versa (ideas?).
I have none. It's useful to define recurring delays in terms of
$some_time + $some_seconds, because that avoids time drift.
sub do_timer_thing {
# do some code here
$kernel-alarm( do_timer_thing = $heap-{next_time} += 5 );
}
That does something every 5 seconds, without clock drift.
It also uses alarm semantics rather than delay semantics, which means
it's susceptible to time shifts. Assuming different ST_ALARM and
ST_DELAY events, you can't make a delay that doesn't succumb to clock
drift.
Implementation of the two scales, it seems, would require two time
fields in
event struct: ST_ALARM and ST_DELAY (instead of ST_TIME). Depending
on which
one is set, the appropriate time scale should be used. I am not sure
what
other consequences this would bring both inside and outside of
POE::Kernel,
but this approach might be worth a try?
Is there any particular advantage of replacing ST_TIME w/ ST_ALARM?
ST_TIME uses the old system time based semantic. Isn't what we are
trying to do here simply to add a delay based semantic? How about
keeping ST_TIME and adding ST_DELAY?
Pete
Re: Signal delivery in POE and time warp.
On Friday, Aug 29, 2003, at 14:29 US/Eastern, Dmitri Tikhonov wrote: It also uses alarm semantics rather than delay semantics, which means it's susceptible to time shifts. Assuming different ST_ALARM and ST_DELAY events, you can't make a delay that doesn't succumb to clock drift. I am not familiar with this issue. Why would there be a clock drift? Aren't we to measure monotonic time using clock cycles? And if there is a clock drift, how bad is it? How long would a delay have to be to experience this drift? I believe Rocco is speaking of clock drifts between the system timer or kernel clock and real time. Take a look at http://www.tldp.org/HOWTO/Clock-2.html for an explanation. Wouldn't it be nice if such monotonic clock source was in ANSI C? What would you say if I wrote an experimental module for monotonic clock (only for Linux for now, based on uptime), and it would be an option to POE::Kernel to either use Time::HiRes::time() or Monotonic::time()? This would be very useful for solving my problem, because my daemon only cares about delays. Dual time scale implementation of POE::Kernel can wait (or be somewhere on a branch). If you take a look at the postings dated back in April, you will find that such API already exists, it's POSIX::times. There is no need to reinvent the wheel. Pete
Re: Signal delivery in POE and time warp.
Dmitri Tikhonov wrote: On Mon, 1 Sep 2003, Peter Chen wrote: wait (or be somewhere on a branch). If you take a look at the postings dated back in April, you will find that such API already exists, it's POSIX::times. There is no need to reinvent the wheel. Yeah, that's what I'd do. You have to convert the value differently based on your OS (128 ticks per second for FreeBSD?) - Dmitri. I believe you can use the constant CLOCKS_PER_SEC for that. Pete
Re: Signal delivery in POE and time warp.
sungo wrote: Peter Chen wrote: Is there any particular advantage of replacing ST_TIME w/ ST_ALARM? ST_TIME uses the old system time based semantic. Isn't what we are trying to do here simply to add a delay based semantic? How about keeping ST_TIME and adding ST_DELAY? so, let me get this straight... we're proposing a rewrite of POE's internal timing mechanisms so that people who don't run ntp won't get screwed when they manually adjust the time on their system once a year? does this seem a little silly to anyone else? we're talking about adding lots of complexity for very little gain. at least in my mind. i say we tag the documentation with a big DONT DO THAT and move on. The issue is actually much more severe. If you reread the postings back in April, you will find that the initial discovery came from a POE based daemon failing to respond to any request after the system time was adjusted. Subsequent investigation revealed that POE's internal timing mechanism is vulnerable to system time adjustements (whether it is done manually or by NTP). Depending on the actual adjustment, one of the symptoms is that sigchild will cease to be delivered. One implication is that any daemon that uses POE::Wheel::Run becomes unreliable. I believe Rocco understands this, and this is why he was looking into using POSIX::times. The only catch is that his development platform is FreeBSD and FreeBSD seems to have a strange implementation of POSIX::times which is not monotonically increasing. IMHO, the goal is not so much to rewrite POE's internal timing mechanism per se, instead the proposal is to supplement it with a delay based semantic in addition to the exisitng system time based semantic. This delay based semantic would allow sigchild to be delivered regardless of system time changes. Pete
Re: Signal delivery in POE and time warp.
On Tuesday, Sep 2, 2003, at 17:28 US/Eastern, Rocco Caputo wrote: I have heard conflicting stories about whether the time shift is significant. Jeff Bisbee posted that systems based on UTC don't see shifts in time() when DST/Standard arrive. Matt Cashner is convinced that NTP prevents the problem from occurring. It's an area I admit knowing very little about, and I'm loath to bring down my development machine to simulate Daylight Savings/Standard time shifts, manually, or by tricking NTP. I've made some small changes, of perhaps five minutes either way, and discovered that times() does not behave as claimed. Perhaps I can ask the question slightly differently. Is it acceptable for POE to fail to deliver sigchild because of a system time change? Here is a simple way of illustrating this, start a process using POE::Wheel::Run, then immediately set the system time backwards for two hours. I have observed that in this case sigchild will not be delivered until two hours later. This is somewhat of a nasty surprise, wouldn't you say? Pete
Re: POE useragent and https
On Fri, 2002-11-08 at 15:16, [EMAIL PROTECTED] wrote: I was trying to make it work with HTTPS, and after a few rounds gave up on it. If turning non-blocking connects off does not help, fork a process and run LWP or LWP::PUA in it. Or try PoCoCl::HTTP. There may be a different way to create or handle sockets that could improve things a bit, but otherwise I don't think it's an LWP or LWP::PUA or PoCoCl::UA problem. The underlying SSL libraries just don't like non-blocking sockets at all. You can try filing bugs against them. Interesting, in fact, I have done that. I have a component that runs in POE::Wheel::Run that calls LWP, and use Cache::Cache to pass back the response. I consider it a kludge though. The curious part is, if the problem is the underlying SSL library. Then how does PoCoCL::HTTP work with it at all? You see the conflict in this logic. After all, the same SSL libray is used. But the other thing is, the way LWP and LWP::PUA are designed it's pretty much impossible to make them truly POE-friendly. For example, an FTP request will block for the entire duration of the connection establishment phase. So a future version of PoCoCl::UA should do away with the LWP dependency entirely. To do away w/ LWP totally may not be so easy. One reason is that it has become somewhat of a standard. People expect the same API's for the request, response, and UserAgent. I wonder whether it's possible to change the low level code so a fix similar to what PoCoCl::HTTP uses can be adapted. Pete
Re: POE useragent and https
On Thu, 2002-11-07 at 21:47, Rocco Caputo wrote: On Thu, Nov 07, 2002 at 11:05:28AM -0800, Lenny Rachitsky wrote: Has anyone gotten POE::Component::Client::UserAgent to work with SSL/HTTPS pages? I've disabled nonblocking, and Net::SSL is being used, but I still get a failure after a few bytes are read. Is there anything else I can do, or am I out of luck? Any help at all would be very much appreciated. We discussed this in IRC, and I spent a long look at the bowels of LWP to no avail. Maybe someone else can figure it out, but I'm totally at a loss. POE::Component::Client::HTTP works, however. Perhaps we can build on each other's experience. Several of us have looked into this problem. Is POE::Component::Client::UserAgent still based on LWP::ParallelUA? I recall a few months ago, we got in touch with the authors, and one of them was working on a fix. Pete
FYI - Mandrake rpm, perl-POE-0.23-1mdk
FYI, I have contributed perl-POE-0.23 rpm to Mandrake. It's released. Pete ---BeginMessage--- [Contrib-RPM] --=-=-= Name: perl-POE Relocations: (not relocateable) Version : 0.23 Vendor: MandrakeSoft Release : 1mdk Build Date: Thu Oct 3 14:58:20 2002 Install date: (not installed) Build Host: bi.mandrakesoft.com Group : Development/Perl Source RPM: (none) Size: 305856 License: GPL or Artistic Packager: Lenny Cartier [EMAIL PROTECTED] URL : http://search.cpan.org/search?dist=POE Summary : POE module for perl Description : POE module for perl. POE is a mature framework for creating multitasking programs in Perl. It has been in active development since 1998. It has been used in mission-critical systems such as internetworked financial markets, file systems, commerce and application servers. --=-=-= * Wed Oct 23 2002 Lenny Cartier [EMAIL PROTECTED] 0.23-1mdk - for Peter Chen [EMAIL PROTECTED] : - 0.23 -- http://www.linux-mandrake.com/en/cookerdevel.php3 ---End Message---
Re: PoCo-Server-TCP, socket shutdown, and SOAP
On Mon, 2002-09-09 at 15:42, Rocco Caputo wrote:
Here's a way to close files above $^F. It's not pretty, not tested,
and may not be very portable. The code would go into Wheel::Run, just
before the function is called in the child process.
# Assuming 255 open handles max.
use POSIX qw(close);
for my $fd ($^F+1..255) {
POSIX::close($fd);
}
It should probably be made optional but on by default, probably
controlled by a CloseOnExec option.
I tried this, and it works, no more file descriptor leaks. :-)
However, it did not solve the problem of hanging TCP connections as I
hoped. I even turned on TRACE_DEFAULT. POE::Kernel is deleting the
file descriptor in _internal_select:
!!! deleting fileno (5) ...
Yet the socket still did not shutdown. I am scratching my head
wondering whether this is caused by some interaction with other modules.
The fail-safe approach seems to be shutdown($socket,2) in my code,
which means I am stuck w/ locally patching PoCo::TCP::Server to keep
$socket on $heap, *sigh*.
Pete
Re: POE::Wheel::Run and missing CloseEvent
On Sun, 2002-09-08 at 14:14, Rocco Caputo wrote: Please try http://poe.dynodns.net/~troc/tmp/peter-chen-wheel-run.perl (temporary URL on my dial-up machine), or you can use it as the base for your own test case. I still can't reproduce the problem here, even with that running for almost 15 minutes (6300 child processes). Maybe it's something specific to your runtime environment? Thank you. I shall try this. Pete
Re: PoCo-Server-TCP, socket shutdown, and SOAP
On Mon, 2002-09-09 at 12:07, Rocco Caputo wrote: If DESTROY is being called, it's almost certain that the socket is being destroyed from POE's point of view. Sockets still won't close if something outside POE has copies of them. That includes child processes, so this may be related to the Wheel::Run issues in the other thread. Eureka! Thank you for pointing this out. Now it all makes sense. In order to handle individual requests, my daemon forks child processes (which are Perl functions run by POE::Wheel::Run), these child processes held on to the sockets of all connections at the time. That's why the clients were hanging until all those children exit, *sigh*. Now I just have to figure out how to close all file descriptors $^F in the child process as you suggested. Pete
Re: PoCo-Server-TCP buffer flushing; possible option() deprecation
On Fri, 2002-08-30 at 18:19, Rocco Caputo wrote:
I opted against using option() for this right now. Moving trace to
_trace (and the others) is a public interface changes that will take
months to implement. Moving the component options to option() would
also take a long time. Rather than wait for all that, I added a
shutdown_on_error value to the heap.
That's also bad, but I don't think it's as bad. Changes have been
committed to cvs; let me know how it works.
Is there some way of setting this option within
PoCO::Server::TCP-new() method?
The current code as of Revision 1.26 requires $HEAP-{shutdown_on_error}
to be set in a connection handler session. The only mean would be to
set this option in one of the Client* state handlers (most likely
ClientError). Would it be cleaner to have this available directly from
new()? For example,
POE::Component::Server::TCP-new
(
Alias = $alias,
Port = $port,
Address = $address,
Error = $error,
Shutdown_on_Error = 0,
ClientError = $client_error,
ClientInput = $client_input,
);
So it's more declarative?
Pete
Re: Cygwin bug.
On Sat, 2002-08-31 at 18:24, Rocco Caputo wrote: I just received a similar problem report and suggested a solution that might also work for you. Namely, sockets should have their close-on-exec flags set to true so they don't bleed into child processes. If this sample code works for you, I'd like to consider setting the close-on-exec for sockets in SocketFactory. Please let me know. I second that. In several instances, my SOAP server couldn't be restarted because one of the child process unintentionally held on to the socket. I was looking for a way to set the option. The problem was that the current PoCo::Server::TCP interface does not expose the underlying socket. Setting the option directly in POE::Wheel::SocketFactory will solve the problem nicely. Pete
Re: POE::Wheel::Run and missing CloseEvent
On Thu, 2002-08-29 at 13:40, Rocco Caputo wrote: I haven't, but I don't use POE::Wheel::Run as much as you do. Is it possible that occasionally the SIGCHLD is being delivered before the STDOUT/STDERR pipes close, so the wheel is being destroyed before it can emit a CloseEvent? I did check for that. As you have guessed, some of my applications make heavy use of this, so I am explicit about the conditions in which a Wheel::Run is considered done and can be deleted. In this particular instance, I only delete a wheel when both sigchild and CloseEvent are received. As I mentioned before, 99% of the time this works perfectly. Yet I have a few reproducible cases where a CloseEvent is never emitted. Even though presently I have a workaround, at some point, I will have to track this down. This reminds me of another idea that we were bouncing around, having a component to manage the states for Wheel::Run. Pete
POE::Wheel::Run and missing CloseEvent
Does POE::Wheel::Run always emit a CloseEvent? I have encountered a strange situation where POE::Wheel::Run seems to omit emitting a CloseEvent. This happens when I do a rpm -Uvv --percent with a bunch of rpm's. Most of the time this works just fine, but occasionally the CloseEvent never appears. This of course turns up as a sporadic bug, since I am using both sigchild and CloseEvent to indicate that the process has completed and there is no more output. Has anybody else seen this? TIA. Pete
Re: PoCo-SOAP and PoCo-Server-TCP buffer flushing
On Tue, 2002-08-20 at 16:34, Rocco Caputo wrote: On Mon, Jul 15, 2002 at 04:27:26PM -0400, Peter Chen wrote: Thank you for confirming this, and fixing it. Please let me know when you have a release, and I will roll out another RPM. I seem to have fixed it, but I don't remember when. The fix isn't the same as your code, but it seems equivalent. Could you test it to make sure that it fixes your SOAP server and doesn't break anything else? Will do. Is there going to be another release? I probably ought to make another rpm with the fix. Mandrake has accepted quite a few of the POE related rpm's that I contributed. So far there are perl-POE, perl-POE-Component-Logger, perl-Log-Dispatch, perl-Log-Dispatch-Config, etc. Pete
Re: PoCo-SOAP and PoCo-Server-TCP buffer flushing
On Wed, 2002-08-21 at 13:52, Peter Chen wrote: On Tue, 2002-08-20 at 16:34, Rocco Caputo wrote: I seem to have fixed it, but I don't remember when. The fix isn't the same as your code, but it seems equivalent. Could you test it to make sure that it fixes your SOAP server and doesn't break anything else? Will do. I just tested this with some of the simplest cases. After 6 clients connecting to the PoCo::SOAP server concurrently making a total of 2000 requests, every thing seems to work. The next step is to test this in a bigger application. I do have a question regarding the change in PoCo::Server::TCP (revision 1.24 from CVS server) though. It appears that if it gets a read error 0 (the client stops writing to the socket), it will shut down the handler session. I am not sure whether this is the desired behavior. I can imagine situations where the user may want to have more control over when the handler session is shutdown. For example, what happens when a read error 0 is received by the server before the server completes handling the request? In several instances, upon ClientInput, I use POE::Wheel::Run to handle some long running commands. If the server shuts down the handler session upon a read error 0, it will never have a chance to send back a response. If this a behavior that we wish to preserve for backward compatibility, perhaps it can be achieved by adding an additional option to indicate whether the handler session should automatically shutdown upon a read error? Pete
Re: Happy Birthday, POE!
On Fri, 2002-08-16 at 15:42, Rocco Caputo wrote: August 15, 2002 marked the fourth anniversary of POE's first public release. As lead designer and developer of the project, I would like to thank the hundreds of people who have helped make POE as great as it is today. Unfortunately, I cannot thank you individually. Thank you, everybody! -- Rocco Caputo / [EMAIL PROTECTED] / poe.perl.org / poe.sf.net If you have a wish list on Amazon or some site of the sort, I will be happy to send you/POE a birthday gift. POE has been good to me. :-) Pete
Re: PoCo::SOAP and non-POE-safe SOAP requests
On Sat, 2002-07-27 at 07:49, Rocco Caputo wrote: The only reason I can think of to use POE_safe might be if you want to support older versions of Perl. Otherwise the attributes sound fine. Yes, I am leaning toward using attributes as well. You could do this a third way: PoCo::SOAP can emit events which are handled by other sessions. The other sessions can decide whether to block or not. I think PoCo::Server::HTTP does this. That's true. One catch is that most likely a large number of functions that PoCo::SOAP needs to dispatch to are not POE aware. In this case, the only safe way to handle them may be using Wheel::Run. Pete
PoCo::TCP::Server, socket shutdown, and SOAP::Lite
How does one shutdown a socket created by PoCo::TCP::Server?
I am noticing some SOAP::Lite client hangs. Netstat shows that the
server is at a close wait state, and the client is at fin wait state.
Saving the socket passed into the the handler session in $heap-{socket}
and later calling shutdown($heap-{socket}, 1) in the ClientFlushed
state solved the problem for me. It seems that just deleting the
Wheel::ReadWrite is not sufficient to shutdown the socket. Also the
assumption seems to be that the client will disconnect first.
This is somewhat of a problem with SOAP::Lite. Apparently (based on my
observation, not the code) a SOAP::Lite client will wait for a response
until it detects a read EOF. This means the server must do
shutdown($socket, 1) first.
I am not sure whether my deduction is completely accurate though. It
surprises me that such cases haven't come up, which makes me wonder
whether I am missing something here.
Perhaps others with more experience with POE and socket programming can
shed some light on the subject. Thanks in advance.
Pete
FYI - POE 0.22 rpm for Mandrake on rpmfind.net
FYI, POE 0.22 rpm for Mandrake has made it to rpmfind.net: http://rpmfind.net/linux/rpm2html/search.php?query=perl-poe Pete
Re: ANN: POE 0.22 released
On Thu, 2002-07-04 at 01:28, Rocco Caputo wrote: Version 0.22 of the POE networking and multitasking framework has been released. This is mainly a bugfix and minor feature update from 0.21. Thank you. Incidentally, I have uploaded perl-POE-0.22 rpm to Mandrake contrib. Hopefully, it will show up soon on rpmfind.net. Pete
Re: Sigchild handler and its return
On Tue, 2002-06-25 at 21:41, Rocco Caputo wrote: You are correct. Children of a session will always receive notification of a signal even though the parent session has handled it. Thank you. I see. In that case, I guess the paradigm is for every session to keep track of the pid's it is interested in, and ignores the rest? Pete
Re: Sigchild handler and its return
On Wed, 2002-06-26 at 11:08, Rocco Caputo wrote: The paradigm has recently shifted regarding POE::Wheel::Run. Its new CloseEvent should now be a reliable and convenient way to tell when a child has gone away. So far I have been conservative by checking both CloseEvent and the arrival of SIGCHLD (mainly for the exit status). I have two concerns: 1. Event sequence. It's not clear to me whether it's possible for SIGCHLD to arrive after CloseEvent. 2. Exit status. In many cases, CloseEvent alone is insufficient. It only tells one that the child process has no more output, but one does not know whether the child has exited abnormally. For example, it's possible that the child process was killed, in which case, one would need to know that the output might be incomplete. I separate theses exits into five categories: success, failure, error, timeout, and abort. success - exit code 0, no signal, no coredump. failure - non-zero exit code, may have a signal, or a coredump. error - the child process never started, check ErrorEvent. timeout - child killed by the system, because it timed out. abort - child killed by the system based on a user request. To make my life easier, since one of my application makes use of POE::Wheel::Run extensively, I wrote a module to encapsulate these states. At one point I was also concerned whether there would be a problem deleting a wheel before its child process is reaped by waitpid. Based on a quick look in POE::Kernel::_invoke_state, it looks as if the child process will be reaped regardless of whether the wheel still exists. Pete
Re: Sigchild handler and its return
On Wed, 2002-06-26 at 13:05, Rocco Caputo wrote: This is an excellent point. If you've come up with a better interface for child processes, we can do some harsh revising. I'd really like to see Wheel::Run redone as a component anyway. Then it can manage its own signal handlers. We do seem to be heading that way. I like it. In addition to what's available in the Wheel::Run interface, I may suggest the following for consideration: 1. StdoutCloseEvent 2. StderrCloseEvent. Both can be detected through ErrorEvent already. These are more convenient, and more explicit. 3. SigchldEvent, a more explicit interface for sigchld handler, which is of particular importance for this component. It will be nice to only receive this event for child processes generated by this component, and use the return value from the handler to decide whether the signal should be propagated. 4. CompleteEvent, receives same arguments as sigchld handler. Instead of the developers keeping track of whether both CloseEvent and SIGCHLD have been received, they can use CompleteEvent. 5. KillEvent, analogous to wheel-kill, takes a signal number or name as the argument. 6. Timeout, specifies the number of seconds to timeout, upon which TimeoutEvent is dispatched. Undef or 0 means there is no timeout. 7. TimeoutEvent. See above. These will cover most of the functionalities that I added in my WheelState module. There are a few convenience functions, that may be handy: 1. exit_status, returns the exit status of the child process. 2. exit_code, returns $exit_status 8 3. exit_signal, returns $exit_status 127 4. exit_coredump, returns $exit_status 128 All 4 return undef if the child process hasn't exited. I suspect some CPAN modules may have already defined this, but I haven't found one. Pete
Re: POE 0.20 released to CPAN, web
On Sat, 2002-06-08 at 02:15, Rocco Caputo wrote: Version 0.20 of the POE networking and multitasking framework has been released. Thanks to everyone who helped make this possible. That's great. I have submitted the corresponding RPM to Mandrake. It should show up within the week. I noticed a couple peculiarities while making the rpm: 1. CHANGES file is empty. It's touched in lib/Makefile-5005.pm, but that doesn't seem to explain why it's empty. 2. NEEDS file has no mention of Storable module which seems to be inconsistent with lib/Makefile-5005.pm: In NEEDS: POE::Filter::Reference wants FreezeThaw POE::Filter::Reference wants Compress::Zlib In Makefile-5005.pm Optional modules for marshaling/serializing data. = [ -default = 0, 'Storable' = '', 'Compress::Zlib' = '', ], Finally in POE::Filter::Reference, If SERIALIZER is omitted or undef, the Reference filter will try to use Storable. If storable isn't found, it will try FreezeThaw. And finally, if FreezeThaw is not found, it will die. This seems to indicate that POE::Filter::Reference wants either Storable or FreezeThaw. In all cases, thank you, Rocco. Pete
RPM's appearing on rpmfind.net
It seems that the typical Mandrake process for contributing a package to the time it appears on rpmfind.net takes 2-3 days. The first POE related rpm, perl-Filter-1.28 has appeared on rpmfind.net. http://rpmfind.net//linux/RPM/cooker/contrib/RPMS/perl-Filter-1.28-3mdk.i586.html Now I have the process down. We are ready to roll when POE-0.20 is released. Pete
beginning of POE RPM's
FYI - I have started uploading source RPM's to Mandrake following its process (yeah, there are a few steps). The first one is perl-Filter which includes Filter::Util::Call that POE needs. My plan is to upload POE-0.19, and other POE related modules in this couple weeks, and eventually POE-0.20 when it's released. Pete
Re: beginning of POE RPM's
On Tue, 2002-06-04 at 12:43, Rocco Caputo wrote: On Tue, Jun 04, 2002 at 12:36:24PM -0400, Peter Chen wrote: FYI - I have started uploading source RPM's to Mandrake following its process (yeah, there are a few steps). The first one is perl-Filter which includes Filter::Util::Call that POE needs. Thank you, Pete. You are welcome. I figure instead of maintaining my own rpm's, I may as well go the distance, so other people may benefit from them. Also, I tried to stay away from vendor specific features in the rpm specfiles, so other distributions may use these too, for example, RedHat. Pete
Re: POE RPM's
On Sat, 2002-06-01 at 11:24, Al Tobey wrote:
try out /usr/lib/rpm/cpanflute or /usr/lib/rpm/cpanflute2
Yeah, cpanflute and cpanflute2 are wonderful tools.
This should work for most any CPAN module.
The world is a happy place ...
There are some minor vendor specific idiosyncrasy though.
cpanflute* generate %file list using find. Mandrake RPM Howto
specifically discourages this.
http://www.linux-mandrake.com/en/howtos/mdk-rpm/building.html#AEN267
Note that you should never use find to build a list of files to
include but explicitely list all files (this'll show up bugs in new
versions).
There are also convention differences. For example, Mandrake tends to
use %real_name for the name fo the module, whereas PLD uses %module.
Also Mandrake's rpm macros include definitions of paths to binaries such
as make, perl as %{__make} and %{__perl}.
Another is that some times the specfiles generated need to be tweaked
because of bugs in MakeMaker. For example, some makefile variables need
to be explicitly overridden. With perl modules, the frequent offenders
of this category are INSTALLMAN1DIR, INSTALLMAN3DIR, INSTALLSITEMAN1DIR,
and INSTALLSITEMAN3DIR.
There is a thread about this in May on the rpm mailing list.
http://www.redhat.com/mailing-lists/rpm-list/msg06859.html
The moral of the story is that while it's straight-forward to generate a
rpm specfile for a CPAN module, it does take additional effort and
understanding to make it conform to a specific distribution's
conventions (if one so desires).
Pete
Re: Pattern for converting procedures to events
On Tue, 2002-05-28 at 18:11, [EMAIL PROTECTED] wrote:
I cannot give final POE code since I'm not quite sure about the syntax that
would be used to specify place types. The net could be described similar to
the small sum test (just list all the transitions together with their arcs,
the arcs' expressions and the guard and code expressions). For the example
above this could be
stepN = {
arc_in = {
placeN-1 = {
type = 'single', # just check one token at a time
expression = sub { shift; }
# just consumes one token, enabled as soon as there is one token
# (that means one work-packet
}
},
This is not really more than just writing the steps. But these would occur in
the correct order, and you could even execute several of these concurrently.
Is this ready for public consumption? I downloaded the first snapshot
and took a peak. The code is well organized. I like what I see. It
seems to be a very promising tool for describing and coding a state
machine. It would make my tutorial as simple as writing down the steps
as you mentioned.
The only barrier is the initial learning curve of CPN. Then again,
having a clearly defined procedure to convert a state machine into code
makes it well worth the effort.
Pete
Re: Pattern for converting procedures to events
On Fri, 2002-05-24 at 19:03, Matt Cashner wrote: are you familiar with POE::Component::SubWrapper? if not, you might want to become so. it seems like it would be a great win for you. Thank you for pointing this out. I did. What I find is that it requires many intermediate states in cases where a procedure includes many steps, which makes it tedious. For example, if the procedure has 4 steps, then one would need 4 corresponding callback events. Before you know it, the session is cluttered with many of these intermediate events. This is another issue that troubles me at times. There seems to be a tendency for events to proliferate. And all events handled by a session are exposed externally. There are many internal states of a session that one may not wish to expose. Consequently, I often find myself thinking of ways to reduce the number of events. Pete
Re: Pattern for converting procedures to events
On Fri, 2002-05-24 at 13:05, [EMAIL PROTECTED] wrote: (event-listener) - [step1] - (step2 args) - [step2] ... - [step n] using my coloured Petri net implementation you could either use the graph editor or write a simple hash describing the net. So it either gives you a good graphical presentation of complex nets or a short and easy to write/read code presentation. Do you have some POE code that may demonstrate this? I reread the introductory CPN paper a couple times. I must admit that I do not understand CPN well enough. Pete
Re: best practices for handling responses/postbacks
On Fri, 2002-05-24 at 13:14, [EMAIL PROTECTED] wrote:
On Fri, May 24, 2002 at 10:34:36AM -0400, Peter Chen wrote:
On Fri, 2002-05-24 at 00:02, Rocco Caputo wrote:
Postbacks are cumbersome for message passing in larger programs.
I also notice that postbacks decrease the transparency of the messages
being passed. When posts are used, it's straight forward to tell what
messages are passed back to the sender. Postbacks on the other hand
mask this information.
I wouldn't agree to this. They enable the posting session to have
local communication ends. They don't need to know about the receiving sessions
interface (except the sender-supplied arguments). I would consider this as
a good thing. The bad side is that they are not powerful enough. You can
use them in good ways, but they don't automatically steer developers in this
direction.
It's not clear to me which approach has a clear advantage. That's why I
asked the question. :-) In practice, I tend to use postbacks more often
because I do not need to keep track of the sender as you described.
However, in some ways, postbacks are somewhat awkward.
Another idiosynchrocy about postbacks that often caught beginners is the
arguments (in fact, this was a bug in the example program fragment I
included, *sigh*). In the event handler, $_[ARG0] is [@state_args] and
$_[ARG1] is [@event_args] instead. This is very different from simple
posts.
If I were to use postbacks:
$postback = $session-postback(got_response);
$kernel-post(worker, work_request, $postback);
The event handler for got_response may look like
sub got_response {
my ($state_args, $event_args) = @_[ARG0, ARG1];
# ...
}
But that means, I can no longer write
$kernel-post(got_response, @result);
So, I often end up with
$callback = sub { $kernel-post($session, got_response, @_); };
$kernel-post{worker, work_request, $callback);
Pete
Re: Pattern for converting procedures to events
On Thu, 2002-05-23 at 21:19, Matt Cashner wrote: when you get done with this, would you be agreeable to it being included in poed? (poed is the languishing project to rewrite the poe docs.) all (well most) of our docs are under the bsd license. let me know what you think :) I am honored. :-) Yes. Let me know how to submit this, and I shall. Pete
Re: best practices for handling responses/postbacks
On Fri, 2002-05-24 at 00:02, Rocco Caputo wrote: Postbacks are cumbersome for message passing in larger programs. I also notice that postbacks decrease the transparency of the messages being passed. When posts are used, it's straight forward to tell what messages are passed back to the sender. Postbacks on the other hand mask this information. Another idiosynchrocy about postbacks that often caught beginners is the arguments (in fact, this was a bug in the example program fragment I included, *sigh*). In the event handler, $_[ARG0] is [@state_args] and $_[ARG1] is [@event_args] instead. This is very different from simple posts. Since the creation of postbacks, two changes have occurred to make them nearly obsolete for inter-session (and session/component) messaging. I take it that this means in most cases, using simple posts is preferred? This issue haunts my every waking moment. :) Seriously, a long time ago several people said POE needed a consistent messaging system. As I've used it more, it's become painfully clear they were right. A consistent messaging system will certainly make learning POE easier. I have rough designs for a few ways to pass messages between components. None of them feels quite right, so we still don't have a standard. If I were to write a messaging manifesto summarizing them, would people be interested in discussing them towards the goal of creating one tidy system? Absolutely. IMHO, at a minimum, it will provide a platform for discussion, and opportunities for developers to rethink their messaging passing mechanisms. Pete
Re: Pattern for converting procedures to events
On Fri, 2002-05-24 at 00:47, Rocco Caputo wrote: I think after writing about two of those, I'd go slightly mad and create a function that built these for me. If it turned out that I was doing this a lot, I would probably assume it's a generic pattern and write a new type of Session to do it. As you may have gathered from some of my recent postings, I am putting together some docs on learning POE, so in the future my coworkers may take over the maintenance of some of my POE code. Most of the developers I work with are much more accustomed to procedural programming. Event driven programming presents a substantial paradigm shift and consequently a steep learning curve. This makes it harder for them to adopt POE. Ideally, I would provide them with guide lines (e.g., message passing mechanism) and frequently used patterns in the form of howto's and faq's (many references to POE cookbook). Converting procedures to events turns out to be one of the first and most common patterns, because procedures are what they are used to. It will be convenient to have a dedicated type of Session to handle this. In its absence, a clear and well defined pattern will suffice. I can see that your sample code is much clearer and intuitive than mine. I was never quite satisfied with having nested postbacks. I find it confusing to many people. Also, the fact that they look chronologically reversed is also counter intuitive. Thank you very mcuh. Pete
Re: POE::Wheel::Run EOF again
On Wed, 2002-05-22 at 18:41, Rocco Caputo wrote: Do you specifically need to know when STDOUT is closed, or is it more important to know that the child is done sending information at all? The later would be sufficient, even though it would be nice to have finer grain control. For example, I am reading the STDOUT of /bin/rpm -qa. What I need to know is when to emit an event to tally the output received. In this instance, I do not care about STDERR. However, I can imagine some applications that do. If it's the latter, I have just committed a new version of Wheel::Run that includes a CloseEvent. The CloseEvent will be fired when the child process has closed its last output filehandle. I understand. I suspect in many applications, this is sufficient. It's much cleaner than counting the number of read errors. Thank you. Pete
Re: Recognizing end of file on POE::Wheel::Run
On Thu, 2002-05-23 at 14:15, Rocco Caputo wrote: It is slightly inaccurate. CloseEvent indicates when the child process has closed its STDOUT and STDERR handles. STDIN may still be available for input. Thank you for the clarification. I see my misunderstanding regarding how STDIN is handled. In all cases, the new facilities that you have put in place should resolve my issues nicely. Thanks again. Pete
best practices for handling responses/postbacks
I would like to solicit your insight on the best practices in handling responses/postbacks. Specifically, I am wondering what are the pro's and con's of using a response event vs. a response postback. For example, $kernel-post($worker_session, $work_request_event, $work_response_event) vs. $postback = $session-postback($work_response_event); $kernel-post($worker_session, $work_request_event, $postback); In the various components I have surveyed (PoCoCl::DNS, PoCoCl::HTTP, PoCoCl::Ping, PoCoCl::UA, PoCoCL::POP3, PoCo::SubWrapper), both styles are used. A couple advantages of using postbacks that I can think of are 1. The ability to include state arguments. 2. More generalized, i.e., it works with functions outside of POE. I am certain that many of you have more profound insights in the matter. TIA. Pete
Pattern for converting procedures to events
I am working on a tutorial explaining how to poeize a procedural
program, in the hope of making it easier for my coworkers to pick up
POE.
Since I have not worked with POE for that long, I am wondering whether
there is an easier and more elegant way of doing this. This is what I
have so far, starting with the most basic. I would appreciate your
feedback.
Given the following function foo:
sub foo {
step1() or return (undef, 'step1 failed');
step2() or return (undef, 'step2 failed');
step3() or return (undef, 'step3 failed');
step4();
}
What's the easiest way to poeize this into the following form:
$kernel-post($session, 'foo', $response_postback);
Assume event foo, step1, ... step4 map to state poe_foo, poe_step1, ...
poe_step4.
sub poe_foo {
my($kernel, $session, $response) = _[ KERNEL, SESSION, ARG0 ];
my $postback3 = $session-postback('step4', $response, $response);
my $postback2 = $session-postback('step3', $postback3, $response);
my $postback1 = $session-postback('step2', $postback2, $response);
$kernel-yield('step1', $postback1, $response);
}
sub poe_step1 {
my($kernel, $session, $success_cbk, $failure_cbk) =
_[ KERNEL, SESSION, ARG0, ARG1 ];
my $result = step1;
$result? $success_cbk-($result) :
$failure_cbk-(undef, 'step1 failed');
}
...
Pete
Re: POE::Wheel::Run EOF again
On Wed, 2002-05-22 at 00:33, Chris Fedde wrote: I saw behavior like this with 0.18 on FreeBSD 4.5 and Solaris 2.8. After 0.19 the sigchld consistently arrives after two error events marking failed nonblocking read on stderr and stdout. Unfortunately, I don't share the same experience. I observe the following sequence read error with error number 0 sigchild 2nd read error with error number 0 Apparently, the first read error comes from STDERR, and the second one is STDOUT. My current kludge is to count the number of read errors. When two of them are received, one can be assured that both STDERR and STDOUT are flushed. This approach was suggested on the mailing list last December as well. To my knowledge there is still no way to map the error events back to file handles. Let me make sure I understand this. Do you mean there is no way to know which file handle caused the read error? Pete
Re: POE::Wheel::Run EOF again
On Tue, 2002-05-21 at 18:58, Chris Fedde wrote: I have not had time to clean it up yet. But my test code now behaves predictably: All the output arrives before sigchld. I can't tell which file handles the errors are on. But all the data has been read before the sigchld which was the big deal for me. Is this algorithmic, i.e., the code ensures that STDOUT is flushed before delivering sigchld? If it's not, how do we know this behavior is deterministic? I am wondering whether the observed behavior is system dependent. I am seeing a different behavior on my test system (PII-350, Mandrake 8.1, perl 5.6.1, POE 0.19). While doing a /bin/rpm -qa, which should return 804 lines, sigchld is consistently delievered before all 804 lines are received. The number of lines missing ranges widely up to 200+. We all care about having all the data read. However, counting on sigchld being delivered after STDOUT is flushed does not appear to be reliable. Pete
Re: POE and daemonization
On Thu, 2002-05-02 at 14:51, Tavin Cole wrote: what is the advantage of self-daemonizing programs over solutions like nohup and daemontools? First of all, thank you for pointing out daemontools. I guess daemontools takes care of daemonization. Most of the Linux distributions I have worked with, e.g., RedHat, Mandrake, for some reason does not include daemontools. My understanding is that the convention for rc scripts in /etc/rc.d/init.d is to have the daemons daemonize themselves. i often find them to be a royal PITA when you actually want to run them under daemontools and they don't provide a command-line switch to disable detaching.. Good point. I shall include such a command-line switch. :-) Thanks again. Pete
POE and daemonization
What is the convention for handling daemonization in a POE application? In particular, what I am wondering is whether typically one daemonizes before $poe_kernel-run() or do the daemonization in one of the application manager sessions. For example, I am considering using what I call a StoreKeeper session to handle application initialization with AppConfig, start up other helper sessions, etc. Following that paradigm, some where in the _start event handler of StoreKeeper, it probably should also daemonize (using Net::Server::Daemonize). Another approach is to only enter POE's event loop when the daemon is fully initialized (validated command arguments, read configuration files) and daemonized. Is there a better way? Thanks in advance. Pete
Re: POE and daemonization
On Wed, 2002-05-01 at 03:05, Andrew A. Chen wrote: I am unsure how kosher this solution is, but it's functional as far as I know. I jacked the daemonize() routine out of perlfaq and called it just before $poe_kernel-run(); See below. -a daemonize(); $poe_kernel-run(); exit 0; Yes, I found references to the function you included. http://dbforums.com/archive/95/2001/12/1/224495 Net::Server::Daemonize seems to take a more rigorous approach. What I am tripping over is whether it makes sense to daemonize within POE's event loop instead, and whether that would cause any adverse effects. For example, let's say I like to use POE::Component::Logger and I would like to keep the logging API consistent. In this case, it seems to require me to enter POE's event loop as soon as possible, and having a session do the initialization work and possibly log errors. Pete
Re: Pocol::UserAgent and https
On Mon, 2002-04-22 at 21:25, Rocco Caputo wrote:
There's a good reason for this, I think. At least, it's a half-good
reason. Net::SSLeay requires sockets to be in blocking mode. I'm not
sure why, but it probably has something to do with the underlying
protocol.
Yes, I noticed that. In Net::HTTPS distributed with libwww-5.64. There
are such comments:
package Net::HTTPS;
# $Id: HTTPS.pm,v 1.2 2001/11/17 02:05:31 gisle Exp $
$VERSION = 0.01;
# The underlying SSLeay classes fails to work if the socket is
# placed in non-blocking mode. This override of the blocking
# method makes sure it stays the way it was created.
sub blocking { } # noop
Here is another posting in which Gisle pointed out that nonblocking mode
is disabled for SSL.
http:[EMAIL PROTECTED]/msg03265.html
Interestingly enough, open_ssl itself seems to allow non-blocking mode.
There is an example using select and nonblocking sockets in pages
279-282 of Eric Rescorla's SSL and TLS book (ISBN 0-201-61598-3).
Perhaps PoCo::UserAgent can create the socket in blocking mode? I'm
not sure what sort of effect that will have on multitasking within
POE.
Yeah, it seems to work better w/ SSL when I change the
$object-nonblock(1) to nonblock(0).
sub spawn
{
my $class = @_ ? shift : 'POE::Component::Client::UserAgent';
$class = ref $class || $class;
my $object = $class - SUPER::new;
bless $object, $class;
$object - nonblock (0);
The question as you pointed out is whether this will have some adverse
effects. For example, is it possible to block during reading and
writing after select indicates that it's ready to be read or written?
If it does block, would setting timeout (to something much shorter than
180 seconds) for PoCoCl::UserAgent help?
Pete
Re: POE SOAP server and asynchronous calls
On Mon, 2002-04-22 at 21:41, Rocco Caputo wrote: On Thu, Apr 18, 2002 at 05:01:44PM -0400, Peter Chen wrote: This may be fine depending on what handle($input) does. I wouldn't worry about breaking up handle() if it's quick, for instance. This thread actually ties into the other thread about PoCoCl::UserAgent. :-) In my case, most SOAP requests can be handled quickly, returning values of variables, etc. I have made them synchronous calls using $poe_kernel-call(). However, there are a couple instances where it needs to make a https connection to retrieve some data. This is where PoCoCl::UserAgent comes into the picture. Since the SOAP clients are CGI's that display returned information, to improve the responsiveness, I decided to circumvent the issue by returning a fault code right away indicating that the client should check back again, while an https connection would be initiated by the SOAP server. Combining this with client pull (using refresh), the result was reasonable. However, since SOAP works over HTTP, I'm led to believe that it's a synchronous RPC protocol. Yes, and this is where I find a steep learning curve, how to reframe the synchronous procedural programming that I was used to in an asynchronous event driven environment. The application seems to lend itself to either the first or third solution (blocking handle() or forking). I am beginning to notice that there are more and more cases, where forking is necessary. For example, if I want to play it safe, I would probably fork and make the https connection using LWP, so I don't have to worry about it blocking other POE processing. Of course, all the time, Java folks would probably use threads w/o even thinking twice, *sigh*. Thanks for your response. It was helpful. Pete
Re: Pocol::UserAgent and https
On Tue, 2002-04-23 at 13:58, Rocco Caputo wrote: It can take a long time to build a TCP socket and even longer to determine the socket can't be built. Connecting in non-blocking mode will let other things happen while TCP works out the details. Once the socket is built, it can be made to block again. So I take it that what you are saying is that connect could block for a long time. I've used POE::Wheel::SocketFactory to create plain sockets without blocking. Once the connection succeeds, I've made it secure by turning blocking back on and tie()'ing a modified Net::SSLeay::Handle class to the socket. See http://poe.dynodns.net/~troc/tmp/poing-ssl.perl. It requires POE 0.19. Thank you. I did take a look, very nifty. This probably cuts down most of the hangs. I wonder whether there is a way to incorporate this into PoCoCl::UserAgent. Pete
Pocol::UserAgent and https
Has anybody tested https with Pocol::UserAgent? When I turn on LWP::Debug by use LWP::Debug qw(+); It shows that it's always using a IO::Socket::INET instead of Net::SSL. LWP::Parallel::UserAgent::init_request - (undef, [undef], LWP::Parallel::Protocol::https=HASH(0x84c1ef4), 180, 1) POE::Component::Client::UserAgent::_add_out_socket POE::Component::Client::UserAgent=HASH(0x84d4370) IO::Socket::INET=GLOB(0x8505058) This of course causes problems later. Pete
Re: Pocol::UserAgent and https
After digging deeper, I found out the issue stems from LWP::Prallel::Protocol::http. In LWP::Prallel::Protocol::http::connect, it explictly creates IO::Socket::INET when it's in nonblocking mode, this overrides the original https protocol, and effectively disable any https support in nonblocking mode. Consequently, PoCoCl::UserAgent does not support https either. Pete
POE SOAP server and asynchronous calls
I have been working with Fletch's POE::Component::SOAP. It's wonderful
to be able to get a SOAP server up and running with very little code.
I am tripping over some conceptual issues with the synchronouse calls.
Perhaps some of you who are more familiar with POE paradigm can
enlighten me.
Currently PoCo::SOAP makes a synchronous SOAP::Server::handle call to
deal with the SOAP requests. I am wondering how to make it more POE
compliant, perhaps some how breaking this up in a fashion similar to
PoCo::SubWrapper. It's not obvious to me how to go about this.
Here is an extract of the function that handles incoming requests.
sub got_req {
my( $self, $heap, $input, $wheel_id ) = _[ OBJECT, HEAP, ARG0, ARG1
];
$input .= '/SOAP-ENV:Envelope';
my $response = $self-{_dispatch}-handle( $input );
$self-{_wheel}-put( $response );
}
It seems that it may be necessary to move
$self-{_wheel}-put( $response );
to another state, got_resp, after the result of the SOAP call is
available. I am somewhat at loss regarding the next step.
Thanks in advance.
Pete
