On Sunday, January 12, 2014 7:13:18 PM UTC-8, Joe Van Dyk wrote: > > On Sunday, January 12, 2014 6:48:26 PM UTC-8, Daniel Farina wrote: >> >> On Sun, Jan 12, 2014 at 6:40 PM, Joe Van Dyk <[email protected]> wrote: >> > from "perf top": >> > >> > 75.38% [kernel] [k] hypercall_page >> > 3.39% python2.7 [.] 0x179be3 >> > 3.18% libc-2.15.so [.] 0x140931 >> > 2.86% libcrypto.so.1.0.0 [.] 0x66dbf >> > 2.78% python2.7 [.] PyEval_EvalFrameEx >> > 1.77% liblzo2.so.2.0.0 [.] lzo1x_decompress_safe >> > 1.02% [kernel] [k] copy_user_generic_string >> > 0.73% liblzo2.so.2.0.0 [.] lzo_adler32 >> > 0.59% python2.7 [.] _PyObject_GenericGetAttrWithDict >> > 0.28% libc-2.15.so [.] epoll_ctl >> >> Yeah, this is the kind of trace I don't know what to do about. I >> suspect it's from making too many syscalls, but I'm not sure, and >> cursory search doesn't give me a slam dunk either, roughly the same as >> last time. Someone who knows Xen a bit is going to have to divine >> something or I have to put aside a block of time to figure out what it >> means, which is why I didn't go ahead and Just Fix It in the past. >> >> Another profitable use of time might be to run the program through one >> of the Python-level profilers. Lately I've been thinking I should >> commit "hidden" (as to not clutter the help messages) '--debug'-family >> options, including that one. >> > > More random data points... > > I was originally using Ubuntu 10.04 on a m1.large instance. I switched > over to Ubuntu 12.04 on a m1.xlarge. Performance improved a tad, about 10 > MB/s. > > I tried s3cmd, I could get about 25MB/s. Although weirdly, CPU usage was > high as well. > > I then made one of the tar partition files public and used curl and wget > to download it. I could get 37MB/s (which is what I'd expect, I think). > But, they also had a high cpu percentage in top. Seems weird maybe? > > I ran 'perf top' on the 12.04 system while hardly anything was running, I > see: > 99.93% [kernel] [k] hypercall_page > at the top. >
and curl https://s3-url | lzop -d -c | tar -x could handle about 35 MB/s. -- You received this message because you are subscribed to the Google Groups "wal-e" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
