Re: benchmarking various event loops with and without anyevent
On Apr 28, 2008, at 03:21, Marc Lehmann wrote: In all fairness, I want to point out that, after _multiple_ rounds of longish e-mail exchanges, Rocco Caputo could not solve the problems that forced AnyEvent to use this design, nor did he enlighten me on how to work around the specific problems that I mentioned to him that forced this design decision(*). Addressed in <[EMAIL PROTECTED]>. Please respond there. He did not come up with any further evidence for a problem, either (just repeatedly stating that the design is broken. The only argument he brought up was: one of your design goals is to be reasonably efficient, POE does not do it reasonably efficient, so your design is broken, which is an outright absurd logic). Marc and I are disagreeing with what I wrote in the message included in <[EMAIL PROTECTED]>. No amount of he- said/no-he-said will resolve it at this point, so I refer the reader back to the actual exchange. Everyone: Your suggestions to improve my communication are greatly appreciated. Please comment off-list, if you can. In fact, it seems his problem is indeed the AnyEvent API and not the interface module to POE, i.e. the "broken" means I should not provide events in the form AnyEvent does, which is of course counterproductive to the goal of AnyEvent of being compatible to multiple event loops (I can't provide different APIs to different event loops...). I explicitly stated otherwise in the message included in <[EMAIL PROTECTED] >. It's the sentence beginning with "We should not need to change AnyEvent::Impl::POE's public interface". So I conclude that even the POE author is unable to provide a (strongly) more efficient approach, which, according to his own words, would make him worse then the average first-time POE user. Considering our previous discussions on the matter, I feel that your conclusion is premature. Also, you seem to be saying that one solution can simultaneously perform equally as well as another and worse than it. Which quantum computers have you ported AnyEvent to lately? :) -- Rocco Caputo - [EMAIL PROTECTED]
Re: benchmarking various event loops with and without anyevent
On Apr 28, 2008, at 06:24, Marc Lehmann wrote:
[most important points first]
In your case, I would create a single persistent POE::Session
instance
that serviced all the watchers.
I would, too, but I cannot find a way to do that with POE: sessions
without active watchers will be destroyed, forcing the session to have
active resources will make the program never-ending.
I already told you that I tried this approach, and why I couldn't
get it
working.
I might use something like an explicit reference count to keep the
session alive. I would create a proxy object that, when DESTROYed,
would post a shutdown message to the session. Or if AnyEvent knows
when a program is to exit, I would have it explicitly shutdown the
session as part of its finalization.
The shutdown handler would remove the explicit reference count,
allowing the session to stop.
It's similar to the technique you use in AnyEvent::Impl::POE:
sub DESTROY {
POE::Kernel->post (${${$_[0]}}, "stop");
}
Except it would be done once at the very end, rather than for each
event watcher.
Perhaps this isn't necessary. You're not using run(), so technically
you're free to go at any time. If your program must exit while
watchers are active, then you could force the issue by sending a
global UIDESTROY signal (designed to tell POE when it must
unconditionally stop), and calling run():
$poe_kernel->signal($poe_kernel, "UIDESTROY");
$poe_kernel->run();
On the other hand, your AnyEvent::Impl::POE proxy objects could also
hold references to the singleton session, and if they release those
references when they clean up their POE::Kernel resources, the session
should be "empty" by the time they all destruct. In that case, the
UIDESTROY signal should not be needed. run() will return after
removing the "empty" session.
In short, your AnyEvent::Impl::POE objects would be of the form:
sub new {
# (AnyEvent::Impl::POE setup goes here)
# set up the POE::Kernel watcher
$poe_kernel->something(something);
# make a note of the watcher in this object
$self->{something} = $record_of_the_poe_watcher;
return $self;
}
sub DESTROY {
my $self = shift;
# ... release the POE::Kernel watcher
$poe_kernel->something($self->{something}, something);
}
If you expect the user to be creating their own POE::Session
instances, then you'd need to call() AnyEvent::Impl::POE to make sure
the watchers are created in the right context.
sub new {
# (AnyEvent::Impl::POE setup goes here)
# set up the POE::Kernel watcher
$poe_kernel->call("anyevent_impl_poe", "something", something);
# make a note of the watcher in this object
$self->{something} = $record_of_the_poe_watcher;
return $self;
}
And DESTROY would tell the session when to clear the watcher.
You may need to add a new AnyEvent::Impl method to explicitly stop
POE, especially if your public API allows users to exit with active
watchers.
sub shutdown {
$poe_kernel->signal($poe_kernel, "UIDESTROY");
POE::Kernel->run();
}
As an added bonus, shutting down this way satisfies the run() warning.
I know this isn't a full solution, but I hope you still find it helpful.
You kepe repeating how it could be designed better, but you never
actually
say how to solve the fundamental problems and bugs within POE that
keep it
from being implementable.
I would be welcome to discuss the code more than each-other. If we
can agree on this, perhaps we can get down to more important matters.
See above.
As I said, if possible, I can only imagine the design becoming
vastly more
complex because I would have to create sdessions on demand and be
able to
react to my session beign turned down at unopportune times.
I don't understand why this design is necessary. Please help me
understand your design constraints, so that I may focus on designs
that will work.
What we seem to agree on by now is that such a design is not trivial
to do
with POE.
Also, remember that the benchmarks show that session creation is not
the big problem, running the sesions is - of course, there could be
inefficiencies in POE handling large number of sessions, but that
means
just that - POE doesn't scale well.
While my suggestions are not as trivial as your current design, I
don't think the end design will be as complex as you expect.
Thank you for your feedback. I'm sorry that POE doesn't meet your
needs. When I have the chance, I'll profile POE while running your
benchmark and see what I can do.
As the documentation mentions, AnyEvent doesn't enforce itself on a
module, unlike POE - a module using POE is not going to work with
other
event loops, because it monopolises the process.
This means that a module using POE forces all its users to also use
POE.
This is factually incorrect. For example, POE::Loop::Glib allows POE
to be embedded into applications like vim and irssi. The
application's functionality is not impaired, n
Re: benchmarking various event loops with and without anyevent
[most important points first] > In your case, I would create a single persistent POE::Session instance > that serviced all the watchers. I would, too, but I cannot find a way to do that with POE: sessions without active watchers will be destroyed, forcing the session to have active resources will make the program never-ending. I already told you that I tried this approach, and why I couldn't get it working. You kepe repeating how it could be designed better, but you never actually say how to solve the fundamental problems and bugs within POE that keep it from being implementable. As I said, if possible, I can only imagine the design becoming vastly more complex because I would have to create sdessions on demand and be able to react to my session beign turned down at unopportune times. What we seem to agree on by now is that such a design is not trivial to do with POE. Also, remember that the benchmarks show that session creation is not the big problem, running the sesions is - of course, there could be inefficiencies in POE handling large number of sessions, but that means just that - POE doesn't scale well. On Mon, Apr 28, 2008 at 04:36:42AM -0400, Rocco Caputo <[EMAIL PROTECTED]> wrote: > Most people on the planet don't know Perl, or even how to program a > computer. No amount of documentation will help them. :) Good (or any) documentation is widely known to be almost a _requirement_ for learning how to program or use a software package. > In the future, you may wish to include me on your list of people to > consult about designing applications for POE. I may known a thing or > two about the topic. :) I am not interested in designing applications for POE - I am interested in making it possible to write event-based modules that are interoperable between various event loops such as POE. As the documentation mentions, AnyEvent doesn't enforce itself on a module, unlike POE - a module using POE is not going to work with other event loops, because it monopolises the process. This means that a module using POE forces all its users to also use POE. AnyEvent does not do that, as long as it supports the event model actually in use (it is not an event loop itself!): A module that uses AnyEvent works with both Qt, POE, IO::Async (once its backend is implemented), EV and so on. This is a fundamental difference between POE and AnyEvent, it has nothing to do with event loop backend modules, of which POE also emplys a few, but comes form the fact that you have to call POE::Kernel->run and give up your process to it (just like with EV::loop etc.) > Are you aware that I'm gradually rewriting POE's documentation? If > you could describe what you don't like in a useful way, I may be able > to do something about it. Since I described it already (and you know that) it means you find the way I did it "not useful". Thats a strawman argument. If you don't like my criticism or don't understand it, ask. > I would love to have the opportunity to suggest a different design, You always ahd the opportunity, you are not using it. > Obviously I cannot expect you to know everything about POE. Likewise, > you cannot expect me to magically know when you started writing > AnyEvent::Impl::POE. It doesn't matter when I started writing AnyEvent::Impl::POE at all. What matters is that you made unfounded (and as you now admit, wrong) statements about it (and its author). It is fine with me if you don't understand AnyEvent, it is somewhat fine with me if you make strong (But wrong) statements about it, but don't expect anybody to put much faith in them, or you ability to make useful statements. > Even if you announced it somewhere, I may not have been looking. The > first I heard of it was here, when you announced your benchmarks. Yes, so? > In general, if you need someone's attention online, the most effective > and polite way is to contact them directly. I did not need your attention? > about this, then I'm sorry that I missed it. Are you sure your > message wasn't lost in transit? Which message? I was only conveying some benchmark results, to give people an idea of the overheads of AnyEvent of various event loop implementations in the hope of beign useful. I was also hoping that people might give AnyEvent some try, as it's design doesn't force a module author using it into a specific event loop. > Assuming that N is the same between the equivalent POE and > AnyEvent::Impl::POE program: > > S(N*M) > S(N) for M > 1. > > QED :P I couldn't really follow you here, and I am not sure what you have proven. To me it certainly looks as if it was "POE cannot support the AnyEvent API efficiently" (at leats not in a simple and straightforward way). I knew that already. > >I can only imagine making some very complex on-demand instantiating and > >re- check wether the session still exists on each watcher creation. > > Your imagination comes up with such incredible things. Don't lose > t
Re: benchmarking various event loops with and without anyevent
On Apr 28, 2008, at 03:21, Marc Lehmann wrote: On Sat, Apr 26, 2008 at 08:13:27AM -0400, Rocco Caputo <[EMAIL PROTECTED] > wrote: each event watcher. Anyone who knows POE can tell you this is one of the least efficient designs possible. In fact, this design is worse than the average for first-time POE users. [He then called my model fundamentally broken in private mail and the documentation rude and unprofessional, without bringing up any evidence] Dear perl-loop@perl.org, and anyone reading this in the archives. I apologize for my part in this unfolding thread. As Marc mentions, I have been trying to take it to private e-mail. However I feel that Marc has misrepresented to the list what I said to him in private. At this point, it's easier for me to repost what I actually said rather than paraphrase it. Again, I'm sorry you had to be involved. -- Rocco Caputo - [EMAIL PROTECTED] Begin forwarded message: From: Rocco Caputo <[EMAIL PROTECTED]> Date: April 27, 2008 23:29:18 EDT To: Marc Lehmann <[EMAIL PROTECTED]> Subject: Re: benchmarking various event loops with and without anyevent On Apr 27, 2008, at 01:53, Marc Lehmann wrote: On Sun, Apr 27, 2008 at 01:15:49AM -0400, Rocco Caputo <[EMAIL PROTECTED] > wrote: I have read your code and documentation for AnyEvent::Impl::POE. Your module's design is fundamentally broken, and your code is probably more to blame than POE. Oh, btw, be careful with such strong idioms such as "fundemantally broken": so far, there is no evidence that it is broken at all, only inefficient. (If you think it really is fundamentally _broken_ then you better back up your statements). I hope to show that I intended no offense. Reasonable scalability (CPU and memory) seems to be one of AnyEvent's design goals. I base this impression on the fact that you're benchmarking your code in terms of speed and size. "Reasonable" is subject to interpretation, but I think we agree that AnyEvent::Impl::POE is neither as fast nor as small as it should be. Therefore, while AnyEvent::Impl::POE operates correctly, it does not fulfill some of AnyEvent's design goals. AnyEvent::Impl::POE's greatest inefficiencies stem from one fundamental design choice: the 1:1 relationship between watcher instances and POE::Session instances. In your own words: "AnyEvent has to create one POE::Session per event watcher, which is immensely slow and makes watchers very large." One point of contention may be whether this is a design or implementation flaw. The problem is inherent in the way one class (AnyEvent::Impl::POE) interacts with another (POE::Session). Class interaction is a software design issue. It can be modeled in software design languages such as UML. Re-implementing the same entity relationship more efficiently, or in a faster language such as C, would not resolve the scalability problem. Therefore, AnyEvent::Impl::POE is flawed in design rather than implementation. Therefore, AnyEvent::Impl::POE's design prevents it from meeting some of AnyEvent's design goals. Therefore, AnyEvent::Impl::POE's design is broken. Unfortunately most of AnyEvent::Impl::POE's design stems from its flawed interaction with POE::Session. We should not need to change AnyEvent::Impl::POE's public interface, but we will need to rethink and revise nearly all aspects of its interaction with POE. Therefore, AnyEvent::Impl::POE's design flaw is a fundamental one. Therefore, AnyEvent::Impl::POE's design is fundamentally broken. Again, let me repeat that empty insults that obviously are founded by paranoia will not have any positive effect on your standing with me (I mean, I won't hate you or anything, but you make yourself an idiot in my eyes very quickly by repeatedly not beinging up any evidence...). Of course, I understand that if you mistook my comments about POE as rude, there is a natural tendency to "insult back". I hope I have shown reasonable evidence that my assertion is neither empty nor intrinsically insulting. Without the intent to insult, there can be no intent to "insult back". In the end, any offense you have taken may be of your own manufacture. Could there be cultural differences to overcome? In this light, your assertion of my paranoia is unfounded, unjust and offensive. Your view that I'm acting like an idiot is no better. You are of course entitled to your opinions, but those two are not appropriate for polite conversation. ... I'm sorry that I haven't responded promptly to your e-mail. I get the impression that you expect the worst from me, so I feel the need to choose my words in a slow and painful (and often futile) effort to minimize being misunderstood. As a result, I cannot write to you as often as I would like. Nor have I had the opportunity to work with you on the module in question (something I would much rather be doing). I hope we can reach a m
Re: benchmarking various event loops with and without anyevent
On Apr 27, 2008, at 00:56, Marc Lehmann wrote: On Sat, Apr 26, 2008 at 08:13:27AM -0400, Rocco Caputo <[EMAIL PROTECTED] > wrote: each event watcher. Anyone who knows POE can tell you this is one of the least efficient designs possible. It is the only design that I could get working, even after consulting a few people and implementing some workarounds for the bugs in POE. In any case, you have to consider that most people on this planet don't know POE, and even if, they don't know it that well. Since the documentation for POE is in such a bad state, thats the obvious way to fix that. Most people on the planet don't know Perl, or even how to program a computer. No amount of documentation will help them. :) In the future, you may wish to include me on your list of people to consult about designing applications for POE. I may known a thing or two about the topic. :) Are you aware that I'm gradually rewriting POE's documentation? If you could describe what you don't like in a useful way, I may be able to do something about it. In fact, this design is worse than the average for first-time POE users. If a better design is possible, it is not known to me, and you haven't suggested one either, so talk is cheap. I'd be happy to get a more efficient design for POE but nobody could come up with one that also worked reliably through multiple iterations of run and also does not keep the POE kernel from returning. I would love to have the opportunity to suggest a different design, but most of my discretionary time is spent addressing the constant misunderstandings between us. If we can first resolve them, I'll have that much more time to work on the design. Obviously I cannot expect you to know everything about POE. Likewise, you cannot expect me to magically know when you started writing AnyEvent::Impl::POE. Even if you announced it somewhere, I may not have been looking. The first I heard of it was here, when you announced your benchmarks. In general, if you need someone's attention online, the most effective and polite way is to contact them directly. If you did contact me about this, then I'm sorry that I missed it. Are you sure your message wasn't lost in transit? As for the design: First-time POE users tend to design programs where the number of sessions scales linearly with the number of objects that handle events. If S(1) is the total overhead imposed by a single session, then S(N) is the overhead imposed by the average naïve POE user. N is the number of objects handling events. AnyEvent::Impl::POE creates a new POE::Session for every event watcher. It's not uncommon for an object to use more than one event watcher (I/O and timeout, for example). So we can model the session overhead in an AnyEvent::Impl::POE program as S(N*M), where N is the number of objects handling events, and M is the average number of event watchers per object. Assuming that N is the same between the equivalent POE and AnyEvent::Impl::POE program: S(N*M) > S(N) for M > 1. QED :P If a better design *is* possible (which I don't really doubt), then it needs to be vastly more complex, or it needs some non-obvious trick. I can only imagine making some very complex on-demand instantiating and re- check wether the session still exists on each watcher creation. Your imagination comes up with such incredible things. Don't lose that. :) The "trick" is to minimize the number of sessions used. Your benchmarks and comments in your documentation implied that you knew that sessions imposed overhead. I wrongly assumed the solution would be obvious. In your case, I would create a single persistent POE::Session instance that serviced all the watchers. The watchers themselves would be small proxies that controlled POE::Kernel watchers within that session's context. I use this design in POE::Stage. If you're not averse to looking at experimental code, you can find it on the CPAN. It also does a lot of other, unrelated things, so you may have difficulty separating the magic you need from the voodoo you don't. Comments are welcome, if they're useful. (It is possible that I was fooled by the docs as well, so if there is a better way, it likely isn't documented, so who could blame me). I won't blame you. But I will point out that your documentation says you're already familiar with using undocumented POE features. :) Instead of going around and accuse people of making bad designs, it would be much better to improve the documentation for POE, so that said people don't have to go around and start guessing... I don't appreciate your vague, negative comments about the documentation. Please describe problems in a useful way, or your expectation that they be fixed is unreasonable. You seem to be saying that your design for AnyEvent::Impl::POE is based on guesswork. If so, I overestimated your
Re: benchmarking various event loops with and without anyevent
On Sat, Apr 26, 2008 at 08:13:27AM -0400, Rocco Caputo <[EMAIL PROTECTED]> wrote: > each event watcher. Anyone who knows POE can tell you this is one of > the least efficient designs possible. In fact, this design is worse > than the average for first-time POE users. [He then called my model fundamentally broken in private mail and the documentation rude and unprofessional, without bringing up any evidence] In all fairness, I want to point out that, after _multiple_ rounds of longish e-mail exchanges, Rocco Caputo could not solve the problems that forced AnyEvent to use this design, nor did he enlighten me on how to work around the specific problems that I mentioned to him that forced this design decision(*). He did not come up with any further evidence for a problem, either (just repeatedly stating that the design is broken. The only argument he brought up was: one of your design goals is to be reasonably efficient, POE does not do it reasonably efficient, so your design is broken, which is an outright absurd logic). In fact, it seems his problem is indeed the AnyEvent API and not the interface module to POE, i.e. the "broken" means I should not provide events in the form AnyEvent does, which is of course counterproductive to the goal of AnyEvent of being compatible to multiple event loops (I can't provide different APIs to different event loops...). So I conclude that even the POE author is unable to provide a (strongly) more efficient approach, which, according to his own words, would make him worse then the average first-time POE user. This is absurd, so I conclude that the original claim has been disproven. (And yes, I did ask multiple times to come up with how to better design the interface to POE, or how to solve the lifetime-issues with POE). (*) the specific problems are (taken directly from my mail to rocco): - the session must not go away, or there must be an easy way to recreate it when the kernel kills it. - the session itself must not keep the kernel "alive"/running (preferably without going away). -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\
