Ryan Chan wrote:
I am profiling my POE program which fetch HTML pages from remote site.
The result show that 80% of time are spent on this file "POE/Loop/Select.pm"
Is it normal and expected behavior?
Hello,
Without knowing what kind of profiling you are doing ( Devel::DProf?
NYTProf? ) and the code you are writing, it's hard to answer your
question. However, you are in luck as I'm the author of
POE::Devel::Benchmarker and has been benchmarking POE on and off over
the years. What you are seeing is very normal - let me explain a little
why it's happening :)
Your app fetches HTML pages, right? So you are presumably using a
HTTP client - and it's making a lot of network requests, right? What's
happening here is your app spends a lot of time waiting for network
input before it can process them. All this waiting takes up wallclock
time, and that's what your profiler is seeing. The 80% number you see
does not mean the loop is crunching away at the CPU! ( obviously, a
busy-wait loop is a BAD idea, ha! )
If you wanted to reduce the time spent waiting, then you need to
increase the number of parallel HTTP requests. That way, your app spends
more of it's time processing the HTML instead of waiting for network
input. There's plenty of methods to achieve that, and you can ask on
this list or browse the CPAN modules to do it.
Tip: The POE developers have spent years optimizing POE the best
they can. You can safely ignore the core POE modules from your profiling
( POE::Loop::Select, POE::Kernel, etc ) and focus on your own code when
profiling. Also, there are numerous Loop adapters ( POE::Loop::IO_Poll,
POE::XS::Loop::EPoll, etc ) that could take advantage of your platform
or workload to reduce the overhead. As always, benchmark/test your code
against the various loops to see if one of them is best for your
workload. As taken from the POE::Kernel pod: "By default POE uses its
select() based loop to drive its event system. This is perhaps the least
efficient loop, but it is also the most portable. POE optimizes for
correctness above all."
Have fun coding POE apps!
~Apocalypse
