Thanks. Well, it should be possible to use sendfile which is zerocopy, that should increase the performance even more. On Sep 30, 2014 9:17 AM, "Marcel Wirtz" <[email protected]> wrote:
> Hello Sergey > > I did some additional valgrind tests with -O3 and -g (needed by valgrind) > and I am surprised. The unpatched version still spends ~70% of the CPU > instructions in memcpy (Not surprising, it is the same version of memcpy) > and the total instruction count is reduced to about 83% of the unoptimized > version. But the performance in the test without valgrind had been better > by far. The patched version needs only ~14% of the CPU instructions of the > unpatched version, but the difference in performance in the runtime test is > not that big. May be, now IO is the bottleneck. But it's obviously still > better not to use memcpy if you do not need to. > > All test have been done with Mongoose 5.4. > > Am Donnerstag, 25. September 2014 14:36:19 UTC+2 schrieb Marcel Wirtz: >> >> The environment is: >> Ubuntu 14.04 64bit with the most recent kernel >> C-Library: glibc 2.19 >> Compiler: GCC 4.8.2 >> All available updates of Ubuntu have been installed. >> >> I was surprised myself. My first thought was, you call it to often, but >> that's not the case. read() and send() are cheap, because they would use >> DMA if available. The memcpy implementation used is >> __memcpy_sse2_unaligned, I really have no idea, why this is so slow. >> >> Valgrind should not be a problem, the generated output should not contain >> instructions from the tool and it should not modify anything from a normal >> program run. And the CPU load and bad transfer rates even occure without it. >> >> I tested the file example again with -O3 and no -g, performance is far >> better (No surprise here), but still worse than the patched version. >> Transfer rates are good, but CPU load is about 20% lower with the patch. I >> will do a new comparison with Valgrind next week. Btw: In your examples >> Makefile you set -g in line 1 and again in line 16. >> >> >> >> Am Donnerstag, 25. September 2014 10:39:04 UTC+2 schrieb Sergey Lyubka: >>> >>> Thanks Marcel. >>> >>> I am really (mean, really) surprised to see that memcpy() call was a >>> bottleneck. >>> Are we sure that valgrind instrumentation doesn't interfere here? >>> >>> Could you elaborate more on you environment please - OS, C library, >>> compiler, etc? >>> If you build with full optimization on, do you see the same CPU load? >>> >>> >>> On Thu, Sep 25, 2014 at 8:09 AM, Marcel Wirtz <[email protected]> wrote: >>> >>>> Thanks Sergey, >>>> >>>> the patch works great so far. I tested the file example with your patch >>>> applied and, as predicted, the CPU instruction count went down by about >>>> 60%. CPU load as shown by top also decreased by about the same factor and >>>> download speed has increased, especially with multiple parallel downloads. >>>> 2 parallel downloads have the same total transfer rate as a single >>>> download, without the patch it decreased rapidly, limited by the CPU usage. >>>> >>>> An analysis with callgrind now shows FD_* macros as the biggest impact >>>> on CPU load, and i don't think, there's a way to save anything there. >>>> >>>> >>>> Am Freitag, 19. September 2014 23:57:17 UTC+2 schrieb Sergey Lyubka: >>>>> >>>>> Thanks Macel. >>>>> Could you try on attached patch please? >>>>> >>>>> >>>>> >>>>> On Thu, Sep 18, 2014 at 12:19 PM, Marcel Wirtz <[email protected]> >>>>> wrote: >>>>> >>>>>> The problem is definetly not IO related (And no other software I know >>>>>> has such a high CPU load transferring data from a file to a socket). >>>>>> >>>>>> A short analysis of the problem: >>>>>> >>>>>> in transfer_file_data() read is called to read data from the file to >>>>>> a buffer (0.29% of the total CPU instructions). Then ns_send() is called. >>>>>> Here the buffer is copied to another buffer (59.27% CPU instructions in >>>>>> memcpy). send() is called by ns_write_to_socket() and uses 0.49% of the >>>>>> CPU >>>>>> instructions. So the real problem is not reading from the file or writing >>>>>> to the socket, but the internal copy of the buffer in ns_send(). If there >>>>>> would be a way not to copy every byte of the file internaly before >>>>>> sending, >>>>>> the CPU load should be smaller (about 60%, which would be better by far). >>>>>> >>>>>> Am Mittwoch, 17. September 2014 09:54:11 UTC+2 schrieb Sergey Lyubka: >>>>>>> >>>>>>> Well if that is memcpy, then I don't find it surprising. >>>>>>> Server is mostly doing data copying, reading from file and copying >>>>>>> to the socket. >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Sat, Sep 13, 2014 at 12:11 PM, Marcel Wirtz <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Yes >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "mongoose-users" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> To post to this group, send email to [email protected]. >>>>>>>> Visit this group at http://groups.google.com/group/mongoose-users. >>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>> >>>>>>> >>>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "mongoose-users" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To post to this group, send email to [email protected]. >>>>>> Visit this group at http://groups.google.com/group/mongoose-users. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "mongoose-users" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To post to this group, send email to [email protected]. >>>> Visit this group at http://groups.google.com/group/mongoose-users. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- > You received this message because you are subscribed to the Google Groups > "mongoose-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/mongoose-users. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "mongoose-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/mongoose-users. For more options, visit https://groups.google.com/d/optout.
